|
⇤ ← Revision 1 as of 2020-10-16 10:47:40
Size: 198
Comment: Just a stub for now
|
Size: 1944
Comment: first non trivial draft
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 5: | Line 5: |
| Will be written for the 5.6Sprint | Will be completed within the 5.6Sprint Author: GeorgesRacinet == Summary == For my own testing needs outside of the core, I've started to write integration tests, using a small testhelper Python module and the pytest framework. The resulting tests have been fast to run, easy to write and to plug in CI. I'm hereby proposing to upstream that helper module, so that extensions and other external code can leverage it without wondering about compatibility, and perhaps start to introduce integration tests in the core. == Why integration tests? == - only if user CLI experience or full end-to-end is not the point of the test - they're fast - easy to debug (pdb inline or post-mortem) - lots of tooling out of the box == the testhelpers module == (to be completed) It is a good time to upstream it, because it is currently part of Heptapod, which is dropping Python 2 support - speed rather than a complete lib - ease some pains (bytes) without preventing explicit assertions about them == Demo: revsets == (chosen because it is quite long and arguably not of much interest for CLI stability) In preparation for the 5.6 sprint, I've translated roughly a half of `test-revset.t`. On my workstation, that half of `test-revset.t` takes between 25 and 30 seconds. The translated version runs in 0.3 seconds. The code can be seen [[https://foss.heptapod.net/heptapod/py-heptapod/-/commits/topic/default/testhelpers-revset-demo|here]] (currently alongside testhelpers.py, i.e. with the Heptapod Python code). Here is a [[https://foss.heptapod.net/heptapod/py-heptapod/-/pipelines/12469|CI pipeline]] that ran it for Mercurial 5.5, and the current heads of the stable and default branches (it also ran all the tests of py-heptapod) |
Integration Tests Plan
This page to host a report about Python testing experiments I've conducted, helpers for Python tests and the potential they have.
Will be completed within the 5.6Sprint
Author: GeorgesRacinet
Summary
For my own testing needs outside of the core, I've started to write integration tests, using a small testhelper Python module and the pytest framework.
The resulting tests have been fast to run, easy to write and to plug in CI.
I'm hereby proposing to upstream that helper module, so that extensions and other external code can leverage it without wondering about compatibility, and perhaps start to introduce integration tests in the core.
Why integration tests?
- only if user CLI experience or full end-to-end is not the point of the test - they're fast - easy to debug (pdb inline or post-mortem) - lots of tooling out of the box
the testhelpers module
(to be completed)
It is a good time to upstream it, because it is currently part of Heptapod, which is dropping Python 2 support
- speed rather than a complete lib - ease some pains (bytes) without preventing explicit assertions about them
Demo: revsets
(chosen because it is quite long and arguably not of much interest for CLI stability)
In preparation for the 5.6 sprint, I've translated roughly a half of test-revset.t.
On my workstation, that half of test-revset.t takes between 25 and 30 seconds. The translated version runs in 0.3 seconds.
The code can be seen here (currently alongside testhelpers.py, i.e. with the Heptapod Python code).
Here is a CI pipeline that ran it for Mercurial 5.5, and the current heads of the stable and default branches (it also ran all the tests of py-heptapod)