Type checking is an important part of making Mercurial compatible with Python 3. This page describes the workflow, recommendations and gotchas.

== Setup ==

Mercurial source code files have to remain compatible with Python 2 for the foreseeable future. This is why we are storing type annotations in external .pyi files (called "stubs"). We then ''reapply'' them using [[https://github.com/ambv/retype|retype]] for the purpose of type checking using [[https://pypi.python.org/pypi/mypy|mypy]].

Types are stored in a root directory called, well, `types`. The output of the `retype` command lands in an .hgignore'd directory called `typed-src`.

== Type hints ==

Never used type hinting with Python before? Start here: http://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html

You will need Python 3.6. Download `mypy`. If you're a flake8 user, upgrade to the latest version and install the [[https://pypi.python.org/pypi/flake8-pyi|flake8-pyi]] plugin.

== Creating a new .pyi file ==

The simplest way to do it is to start with the `stubgen` script available in the [[https://github.com/python/mypy/blob/master/scripts/stubgen|mypy GitHub repo]]. It is crude but gets the job done. It recreates the basic structure of the given source file in the form of a .pyi file. Note that this .pyi file is missing most annotations, and the ones that it **does** add are likely wrong (in most cases defaulting to the `Any` type). Your job is to fill the blanks in with actual typing information.

Our .pyi files describe Python 3.5's type system and standard library. Not Python 2.7 because the purpose of this exercise is to find all the places in which the code currently doesn't play well with Python 3. Not Python 3.6 because the initial version of Python 3 that Mercurial is targetting is 3.5. That being said, syntax-wise, you are recommended to use variable annotations in .pyi files since they are cleaner and type checkers and `retype` use them anyway.

Speaking of .pyi files, you might have noticed that they are syntactically valid Python, with the single difference that they natively support forward references (in other words, you can specify definitions out of order without consequence). Stylistically, .pyi files are not meant to follow PEP 8. Instead, they conform to the [[https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#stub-file-coding-style|typeshed coding style]].

Mercurial ships with `mercurial.pycompat`, a library providing a number of compatibility bridges between Python 2 and Python 3. While this library is imported implicitly at runtime (via a magic import hook), we don't do that for the purposes of type checking. So, if needed, add the following import in your .pyi files:

{{{
from mercurial.pycompat import delattr, getattr, hasattr, setattr, xrange, open
}}}

== retype ==

NOTE: This tool is very new. If you hit any issues, open an issue on [[https://github.com/ambv/retype|GitHub]] or contact LukaszLanga directly.

From the root of the hg repo run:

{{{
$ retype --hg --quiet mercurial
}}}

A `retype` run does the following:

 * for each source file ...
 * find the corresponding .pyi  file in `./types` and ...
 * reapply typing information from .pyi to the source, writing the result to `typed-src/`;
 * if `retype` is executed with `--hg`, also translate all "native" string literals into bytes

Re-application of types requires copying typing imports and alias definitions from the .pyi file, which is why line numbers will no longer match original source code. It retains its original formatting though.

Some design principles:

 * it's okay for a given .pyi file to be incomplete (gradual typing, baby!), this will only generate warnings
 * it's okay for functions and classes to be out of order in .pyi files and the source
 * it's an '''error''' for a function or class to be missing in the source
 * it's an '''error''' for a function's signature to be incompatible between the .pyi file and the source
 * it's an '''error''' for an annotation in the source to be incompatible with the .pyi file

== Type checking ==

This is what you came here for, right? Once sources are translated in `typed-src/`, run:
{{{
$ cd typed-src/
$ mypy .
}}}
----
CategoryDeveloper