Secure Connections

1. Overview

Mercurial performs various checks to verify that connections to servers are secure. These checks vary depending on the Mercurial version and the version/capabilities of Python/OpenSSL being used.

2. Behavior of Various Configurations

The version of Python (and the OpenSSL it was compiled against) significantly impact behavior.

Modern versions of Python 2.7 contain a number of improvement to the ssl module which allow Python to have more control over and access more advanced security primitives. Generally speaking, these enhancements are only available on Python 2.7.9 or newer. However, some distributions (such as RHEL 7) have backported the improvements to e.g. Python 2.7.5. The following table lists the capabilities of Python with and without the modern ssl module.

Mercurial 3.9 contained a major refactor of the connection security code and user configuration. The main purpose of the changes was to make Mercurial more secure by default. A secondary goal was to give users control to opt in to better and more flexible security configuration.

The following table breaks down changes in behavior in Mercurial 3.9 related to connection security.

3. Common Errors and Warnings

3.1. warning: connecting to %s using legacy security technology (TLS 1.0)

The intent of this warning is to inform you that Mercurial is using old cryptography for communicating with a server. The technology in question (TLS 1.0) has known vulnerabilities and data sent to or from the server could be read by an attacker. While there are workarounds for these known vulnerabilities, the Mercurial project deems TLS 1.0 to have enough security concerns as to warn users when it is being used.

The most likely reason TLS 1.0 is being used is because the Python installation that Mercurial is using does not support anything newer. As documented above, versions of Python before 2.7.9 (including all versions of Python 2.6) don't have access to modern security technologies. Therefore, users seeing this warning are strongly encouraged to run Mercurial with as new a version of Python 2.7 as possible. In addition to better security, performance should also be better.

Installing or upgrading a modern Python is not always simple. Users on Linux distributions should likely not attempt to upgrade the system Python version (e.g. /usr/bin/python) manually, as doing so is dangerous and will likely break pieces of your operating system. While it is catered towards Python developers, pyenv provides a simple mechanism to install Python in your user directory (no root/sudo privileges needed).

If running Mercurial from a modern Python version is not possible, this warning can be disabled by setting hostsecurity.disabletls10warning=true in your Mercurial config file. e.g.

$ hg config -e
[hostsecurity]
disabletls10warning = true

3.2. could not negotiate a common security protocol (tls1.1+) with %s; the likely cause is Mercurial is configured to be more secure than the server can support

This warning precedes an error message - typically one containing protocol unsupported. As the warning says, the likely problem is that Mercurial is insisting on using more secure connection settings than what the server can support.

As of August 2016, the likely reason for this warning is that the server only supports TLS 1.0 (and not a newer version like TLS 1.1 or TLS 1.2). To verify this is the underlying reason and assuming the server is accessible on the internet, you can use an SSL analyzer such as Qualy's SSL Server Test or the Comodo SSL Analyzer. Alternatively, you can use OpenSSL to connect to the server and see what protocol it negotiates:

$ openssl s_client -connect <host>:<port>

e.g.

$ openssl s_client -connect www.mozilla.org:443
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256

If it reports the Protocol as TLSv1, then the server likely only supports TLS 1.0.

Security professionals recommend using a version of TLS newer than 1.0 because of security concerns.So if a server you are trying to connect to supports TLS 1.0 and nothing newer, you should consider contacting the server operator and ask them to support modern security.

While Mercurial insists on using TLS 1.1+ by default, it is still possible to communicate with servers using TLS 1.0. The preferred mechanism to do this is to set a per-host configuration option to allow TLS 1.0. e.g.

[hostsecurity]
hg.insecurehost.com:minimumprotocol = tls1.0

This creates a low-security exception for an individual host while still preserving the TLS 1.1+ requirement for every other server that Mercurial communicates to.

3.3. abort: unable to verify security of localhost (no loaded CA certificates); refusing to connect

This error occurs when Mercurial is unable to load CA certificates to verify the server's certificate.

This error message indicates your Mercurial installation/configuration is incomplete. Typically, Mercurial will load your system's CA certificates. However, it can't always do this. Reasons why it can't do this include:

• Running Python <2.7.9 (older versions of Python don't know how to locate the system CA store)

• Python is unable to find the system CA store (this may indicate the Python installation is mis-configured)

The mitigation for this error is to tell Mercurial where trusted CA certificates are located. This can be done by setting the web.cacerts configuration option to the path to a file containing PEM certificates. Read below for how to obtain a CA certificate bundle.

3.4. (unable to load CA certificates; see https://mercurial-scm.org/wiki/SecureConnections for how to configure Mercurial to avoid this message) OR (an attempt was made to load CA certificates but none were loaded; see https://mercurial-scm.org/wiki/SecureConnections for how to configure Mercurial to avoid this error)

This warning is printed when Mercurial is unable to load CA certificates to verify server connections. This warning is almost always followed by an abort due to failure to validate a server connection.

This warning likely indicates at least one of the following:

• The Python Mercurial is running with is unable to load CA certificates (the Python is likely old and/or misconfigured)
• The Mercurial packager did not properly configure Mercurial to use installed CA certificates

To make this warning go away:

• Run Mercurial with a modern Python (2.7.9 or newer)

• Configure Mercurial to use a CA certificate file by setting the web.cacerts config option

• export SSL_CERT_FILE=/path/to/ca-certificates-in-pem-format (if this works then you know it can work, and then review (again) using web.cacerts config option

3.5. (unable to load Windows CA certificates; see https://mercurial-scm.org/wiki/SecureConnections for how to configure Mercurial to avoid this message)

This warning only occurs on Windows. It occurs when Mercurial is not configured with the location of trusted CA certificates and is unable to load CA certificates from Windows.

This warning should not be seen very often. This is because Mercurial distributions on Windows should ensure that the Python Mercurial ships with is able to load CA certificates from Windows.

If you are installing Mercurial from source or from a wheel (such as through pip install Mercurial and you see this warning, you will need to take action.

To make this warning go away:

• Run Mercurial with a modern Python (2.7.9 or newer)
  • Python 2.7.12 has bug fixes related to interfacing with the Windows CA store.

• Obtain a CA certificate bundle and configure web.cacerts to point to it

3.6. (using CA certificates from %s; if you see this message, your Mercurial install is not properly configured)

This warning will be printed on non-Windows non-OS X machines when Mercurial automatically finds and uses CA certificates from a semi-trusted location.

If you see this warning, your Mercurial installation wasn't properly configured. The CA certificates should be explicitly configured in web.cacerts.

If you installed Mercurial from a package, you should file a bug against the package maintainer and tell them to read this page.

If you installed Mercurial yourself, you can make this warning disappear by setting web.cacerts in your hgrc to the path of a file contained trusted CA certificates.

4. Finding or Obtaining CA Certificates Bundles

Either Python or Mercurial needs to know how to load CA certificates to validate server connections. Modern versions of Python (2.7.9 or older versions with backports of the modern ssl module) that are properly configured/packaged/installed should know how to do this out of the box. If Python cannot load CA certificates, you'll need to explicitly point Mercurial at a CA certificate bundle file via the web.cacerts config option.

If you are on a *Nix operating system, chances are a CA certificate bundle file already exists on your system. Common locations include:

• /etc/pki/tls/certs/ca-bundle.trust.crt (RedHat, CentOS, Fedora)

• /etc/ssl/certs/ca-certificates.crt (Debian, Ubuntu, Gentoo)
• /usr/local/etc/openssl/certs.pem (Homebrew)

If you don't have a CA certificate bundle or would like to obtain a fresh one, Mozilla's curated list of CA certificates is used by many open source projects. The homepage of the certifi project provides more details and a link to download the CA certificate bundle. You can simply download the CA certificate bundle and configure web.cacerts to point to it.

5. Expectations of Mercurial Packages and Packagers

Proper Mercurial packages should guarantee one of the following:

1. The underlying Python install is able to load CA certificates and detect it has loaded CA certificates

2. The web.cacerts config option is defined to an appropriate location

3. The certifi Python package is installed

If one of these isn't true, Mercurial will fail to establish secure connections to servers and/or will print a warning saying that Mercurial is improperly configured.

To test whether the underlying Python install is able to load CA certificates from default locations, run the following Python code:

import socket, ssl
context = ssl.create_default_context()
context.verify_mode = ssl.CERT_REQUIRED
context.load_default_certs()
conn = context.wrap_socket(socket.socket(socket.AF_INET), server_hostname='www.mozilla.org')
conn.connect(('www.mozilla.org', 443))

This should work on Python 2.7.9+ or Python versions with a backported ssl module (such as RHEL7's Python 2.7.5 package). If this doesn't work on Python 2.7.9+, that Python package is arguably broken, as it doesn't provide a mechanism to verify CA certificates in the standard library.

If the above code works, the Python install is able to load CA certificates (using the location configured by the OpenSSL that Python is using). Mercurial should work out of the box. However, behavior may not be optimal (keep reading below).

Simply being able to load CA certificates using the standard library is not necessarily sufficient. On Python installs using OpenSSL (read: probably not Windows), Python will call into OpenSSL's SSL_CTX_load_verify_paths(). This will either loaded CA certificates from a standalone file or a directory containing files with the MD5 hashes of certificates. While both work, a problem with the directory-based approach is that OpenSSL (and Python by extension) don't load CA certificates until they are needed. This means that OpenSSL (and Python and Mercurial) don't know that CA certificates are actually loaded. So Mercurial is unable to differentiate between no CAs being present (the CA store being empty - which likely indicates a user/configuration error) and the CA being missing (which would indicate lack of trust and a danger sign as far as security goes). For this reason, Mercurial recommends loading self-contained CA files (as opposed to directories) because Mercurial can tell that CAs are present and a missing CA means lack of trust instead of a bad configuration.

To verify that Mercurial can tell that CAs are loaded, run the following code:

import ssl
context = ssl.create_default_context()
context.load_default_certs()
context.cert_store_stats()

If this prints {'x509': 0, 'x509_ca': 0, 'crl': 0} (note the 0s) it means that no detectable CAs were loaded. This is not ideal. (Note: Debian distros use the sub-optimal directory-based approach and RedHat distros use the optimal file-based approach.)

If either the original code fails (Python can't load CAs at all) or Python can't tell that it has loaded CAs, the Mercurial install should set the web.cacerts config option in the global/system hgrc to a file containing PEM encoded certificates of trusted CAs (preferred) or ensure the certifi Python package is installed. Configuring web.cacerts is preferred over installing certifi because your system likely already has a file containing trusted CA certificates on it and installing certifi will create N+1 CA files, which results in a more complicated system configuration and potential for things to get out of sync. You probably want a single CA certificate list.