#pragma section-numbers 2 <> = Pypy Plan = '''Status: In Progress''' '''Main proponents: [[Pierre-YvesDavid]], MaciejFijalkowski, MartijnPieter, BryanO'Sullivan''' Pypy is a python interpreter with a Tracing JIT compiler, using it could significantly boost our performance in some situation. == Description == The Python language is powerful but slow. Using a Jit compiler can give us a lot of that speed back without having to translate large part or Mercurial into C. However there is multiple challenge ahead: * Startup and JIT Warm-up time are an issue for the mercurial command line typically short running type, * Pypy is unable to use our CPython extension code, missing multiple of the optimisation we implemented. All kind of stuff can go here, solution description / alternative solution etc == Current approach == * We plan to mitigate startup and warmup time using [[ChgPortingPlan]], * We'll also investigate way to share/reuse JIT tracing information from one run to another, but it is a long shot, * We are investigating using CFFI to call our carefully optimised C code, The CPython implementation should stay around in parallel, * In some case improving the pure python code should be enough. == Failing Tests == * Failed test-devel-warnings.t: output changed * Failed test-strict.t: timed out * Failed test-setdiscovery.t: output changed * Failed test-clone-uncompressed.t: output changed * Failed test-doctest.py: output changed * Failed test-revset.t: output changed == CFFI experiment == Using the CPython API in C makes it hard for pypy to do nice optimisation. Calling our C code through [[cffi|https://cffi.readthedocs.org/en/latest/]] would fix that. However, cffi just allow function call, not building full compatible Python object directly from C. As a result we probably need to keep a CPython API and a CFFI version of our code in parallel. CPython will still use it's carefully build object using the CPython API and PyPy will use pure Python object calling C function through CFFI as it should be able to optimise the pure Python object overhead away. All our C code must be audited and sorted in the following categories === Python Version is Just Fine === The pure version is already as fast as the C version. * … Link to example: 'to be added' === Python Version need rework === The pure version should be as fast as the C version, but is currently not. * `reachableroot` * … Link to example: 'to be added' === Simple Function that need cffi version === * `mpatch.patches` Link to example: 'to be added' === Complex Class that need cffi version === * `lazymanifest` ? * `index` ? * … Link to example: 'to be added' === C Code calling Python code back === This will be fun. * … Link to example: 'to be added' == Benchmark == We plan to have a look at the performance of the following command. * `hg log` * `hg log -p` * `hg bundle -t gzip -a` - the main component of server side clone * `hg unbundle` - the main component of client side clone * `hg blame` * updating - `hg up null` followed by `hg up tip` * performance of http server (hg serve) * hg commit * hg amend * hg rebase * hg status == Roadmap == * {X} have tests pass, * {X} add support for a cffi module policy, * {X} demonstrate cffi usage for each "category": * {X} Simple function, * {X} Simple Class, * {X} Code with roundtrip between Python and C, * {X} Updated Python code (no CFFI). * {X} have a basic set of benchmark * {X} have some plan to re-use//share JIT tracing data * {X} have a C version for all critical code * fill me. * {X} implement that plan == See Also == * [[ChgPortingPlan]] ---- CategoryDeveloper CategoryNewFeatures