Diff for "SecureConnections"

Overview

Mercurial performs various checks to verify that connections to servers are secure. These checks vary depending on the Mercurial and Python version being used.

Behavior of Various Configurations

Mercurial 3.9 contained a major refactor of the connection security code and user configuration. One of the changes is that Mercurial 3.9+ is more strict about connection security and will abort if a connection cannot be verified (prior versions would issue warnings and continue connecting).

Common Errors and Warnings

1. abort: unable to verify security of localhost (no loaded CA certificates); refusing to connect

This error occurs when Mercurial is unable to load CA certificates to verify the server's certificate.

This error message indicates your Mercurial installation/configuration is incomplete. Typically, Mercurial will load your system's CA certificates. However, it can't always do this. Reasons why it can't do this include:

• Running Python <2.7.9 (older versions of Python don't know how to locate the system CA store)

• Python is unable to find the system CA store (this may indicate the Python installation is mis-configured)

The mitigation for this error is to tell Mercurial where trusted CA certificates are located. This can be done by setting the web.cacerts configuration option to the path to a file containing PEM certificates. Read below for how to obtain a CA certificate bundle.

2. (unable to load CA certificates; see https://mercurial-scm.org/wiki/SecureConnections for how to configure Mercurial to avoid this message)

This warning is printed when Mercurial is unable to load CA certificates to verify server connections. This warning is almost always followed by an abort due to failure to validate a server connection.

This warning likely indicates at least one of the following:

• The Python Mercurial is running with is unable to load CA certificates (the Python is likely old and/or misconfigured)
• The Mercurial packager did not properly configure Mercurial to use installed CA certificates

To make this warning go away:

• Run Mercurial with a modern Python (2.7.9 or newer)

• Configure Mercurial to use a CA certificate file by setting the web.cacerts config option

3. (unable to load Windows CA certificates; see https://mercurial-scm.org/wiki/SecureConnections for how to configure Mercurial to avoid this message)

This warning only occurs on Windows. It occurs when Mercurial is not configured with the location of trusted CA certificates and is unable to load CA certificates from Windows.

This warning should not be seen very often. This is because Mercurial distributions on Windows should ensure that the Python Mercurial ships with is able to load CA certificates from Windows.

If you are installing Mercurial from source or from a wheel (such as through pip install Mercurial and you see this warning, you will need to take action.

To make this warning go away:

• Run Mercurial with a modern Python (2.7.9 or newer)
  • Python 2.7.12 has bug fixes related to interfacing with the Windows CA store.

• Obtain a CA certificate bundle and configure web.cacerts to point to it

Finding or Obtaining CA Certificates Bundles

Either Python or Mercurial needs to know how to load CA certificates to validate server connections. Modern versions of Python (2.7.9 or older versions with backports of the modern ssl module) that are properly configured/packaged/installed should know how to do this out of the box. If Python cannot load CA certificates, you'll need to explicitly point Mercurial at a CA certificate bundle file via the web.cacerts config option.

If you are on a *Nix operating system, chances are a CA certificate bundle file already exists on your system. Common locations include:

• /etc/pki/tls/certs/ca-bundle.trust.crt (RedHat, CentOS, Fedora)

• /etc/ssl/certs/ca-certificates.crt (Debian, Ubuntu, Gentoo)
• /usr/local/etc/openssl/certs.pem (Homebrew)

If you don't have a CA certificate bundle or would like to obtain a fresh one, Mozilla's curated list of CA certificates is used by many open source projects. The homepage of the certifi project provides more details and a link to download the CA certificate bundle. You can simply download the CA certificate bundle and configure web.cacerts to point to it.

Expectations of Mercurial Packages and Packagers

Proper Mercurial packages should guarantee one of the following:

1. The underlying Python install is able to load CA certificates and detect it has loaded CA certificates

2. The web.cacerts config option is defined to an appropriate location

3. The certifi Python package is installed

If one of these isn't true, Mercurial will fail to establish secure connections to servers and/or will print a warning saying that Mercurial is improperly configured.

To test whether the underlying Python install is able to load CA certificates from default locations, run the following Python code:

import socket, ssl
context = ssl.create_default_context()
context.verify_mode = ssl.CERT_REQUIRED
context.load_default_certs()
conn = context.wrap_socket(socket.socket(socket.AF_INET), server_hostname='www.mozilla.org')
conn.connect(('www.mozilla.org', 443))

This should work on Python 2.7.9+ or Python versions with a backported ssl module (such as RHEL7's Python 2.7.5 package). If this doesn't work on Python 2.7.9+, that Python package is arguably broken, as it doesn't provide a mechanism to verify CA certificates in the standard library.

If the above code works, the Python install is able to load CA certificates (using the location configured by the OpenSSL that Python is using). Mercurial should work out of the box. However, behavior may not be optimal (keep reading below).

Simply being able to load CA certificates using the standard library is not necessarily sufficient. On Python installs using OpenSSL (read: probably not Windows), Python will call into OpenSSL's SSL_CTX_load_verify_paths(). This will either loaded CA certificates from a standalone file or a directory containing files with the MD5 hashes of certificates. While both work, a problem with the directory-based approach is that OpenSSL (and Python by extension) don't load CA certificates until they are needed. This means that OpenSSL (and Python and Mercurial) don't know that CA certificates are actually loaded. So Mercurial is unable to differentiate between no CAs being present (the CA store being empty - which likely indicates a user/configuration error) and the CA being missing (which would indicate lack of trust and a danger sign as far as security goes). For this reason, Mercurial recommends loading self-contained CA files (as opposed to directories) because Mercurial can tell that CAs are present and a missing CA means lack of trust instead of a bad configuration.

To verify that Mercurial can tell that CAs are loaded, run the following code:

import ssl
context = ssl.create_default_context()
context.load_default_certs()
context.cert_store_stats()

If this prints {'x509': 0, 'x509_ca': 0, 'crl': 0} (note the 0s) it means that no detectable CAs were loaded. This is not ideal. (Note: Debian distros use the sub-optimal directory-based approach and RedHat distros use the optimal file-based approach.)

If either the original code fails (Python can't load CAs at all) or Python can't tell that it has loaded CAs, the Mercurial install should set the web.cacerts config option in the global/system hgrc to a file containing PEM encoded certificates of trusted CAs (preferred) or ensure the certifi Python package is installed. Configuring web.cacerts is preferred over installing certifi because your system likely already has a file containing trusted CA certificates on it and installing certifi will create N+1 CA files, which results in a more complicated system configuration and potential for things to get out of sync. You probably want a single CA certificate list.