Publishing Mercurial repositories

This document focuses on public publication of repositories over the Internet, although private/internal sharing of repositories is mentioned below.

1. Choosing a publishing method

There are a variety of different ways to publish your Mercurial repositories. Some are more powerful than others but may require more effort to set up and administer. See below for some general recommendations.

Note that starting with version 1.6 of Mercurial, the hgwebdir.cgi script no longer exists, and its functionality has been merged into the hgweb.cgi script. If you want to publish multiple repositories, follow the appropriate instructions, substituting hgweb.cgi for hgwebdir.cgi.

See also this comprehensive guide to acquiring Mercurial and configuring hgwebdir.

If using a recent Apache, use mod_wsgi instead of mod_python or CGI as it has better performance.

See also various other guides about configuring Mercurial's Web interface using various technologies.

Recommended solutions for public sharing of repositories are described below.

1.1. Quick recommendations

The easiest way to share changes with other people using Mercurial is to publish them on the Web. The recommended method for publishing repositories over HTTP is to use the hgweb scripts. This is a simple way to quickly publish a single repository or publish multiple repositories easily.

Less desirable are the following:

• Use the hg serve command. This is Mercurial's built-in Web server. It is not really recommended except for temporary situations where you need to publish a repository for a few minutes, for example to pull changes from a laptop.

• Make the plain repository available. This uses a much slower 'serverless' protocol called static-http. We won't cover it here, see StaticHTTP instead.

For private or restricted-access repositories, aside from the solutions explicitly marked as "private/internal" in the table above, authentication measures (certificates, logins) can be applied to many of the "public" solutions in order to restrict access.

2. Introduction and prerequisites

In this document we assume that repositories reside in the /home/user/hg directory. For example, a repository called myproject would reside in /home/user/hg/myproject.

To implement the mechanisms described in this document, you will need the following:

• Some control over the behaviour of the Web server you use.
• (Optional) control over the DNS domain you use.

With control over DNS, such as that provided with various Web hosting service control panels, you should be able to set up a subdomain; this makes the URL of your repositories a little tidier, so that http://hg.example.com/myproject can be used instead of http://www.example.com/hgwebdir.cgi/myproject, for example.

Such an approach, known as virtual hosting, is entirely optional. To implement it for the fictional example.com, a CNAME record for hg.example.com would be defined for the same address as that already used by the Web server.

3. Setting up the hgweb scripts

Note that in Mercurial 1.5.x and earlier, the hgwebdir.cgi script also exists specifically for the publishing of multiple repositories (whilst hgweb.cgi is only able to serve single repositories).

To setup the hgweb.cgi script to publish a single repository or multiple repositories, perform the following steps:

1. Find the hgweb.cgi script in the root of your Mercurial source tree.

2. Copy it to a directory where your Web server can access it. This will be illustrated below using /home/user/webdir as this directory.

3. Rename it to index.cgi if you want, then edit its contents.

4. Make sure the Web server (such as Apache) is configured and can execute the script.

If Mercurial is not installed system-wide, uncomment and edit the Python path in hgweb.cgi (or index.cgi) as indicated:

# adjust python path if not a system-wide install:
import sys
sys.path.insert(0, "/home/user/lib")

It is recommended to use the hgweb.cgi script that is the same version as the version of Mercurial that is installed on your system. If you are using Debian and installed the Mercurial package, you can find the hgweb.cgi script in /usr/share/doc/mercurial-common/examples/ or /usr/share/doc/mercurial/examples/ (depending on your version).

If you choose to create a symbolic link to the hgweb.cgi script in your Mercurial source tree instead of copying it, you might run into Python library problems as it might try to use the libraries from the source tree instead of the ones already installed on your system.

4. Publishing a single repository

You will need to edit the call to hgweb.hgweb as indicated in the following example:

h = hgweb.hgweb("/home/user/hg/myproject", "My Project")

While you could use this mechanism to publish multiple repositories, it requires a little work to configure each copy of the script to have slightly different paths.

5. Publishing multiple repositories

To publish multiple repositories with the hgweb.cgi script takes some work to initially set up, but once it's working, it lets you publish new repositories easily and cheaply. Its advantage is that to publish a repository, you simply place a clone in a particular directory, then add one line to a config file to tell the CGI script that it is allowed to publish that repository.

5.1. Setting up the hgweb.config file

In the /home/user/webdir directory, create a file called hgweb.config. Here are the contents of an hgweb.config file:

[paths]
myproject = /home/user/hg/myproject
otherproject = /home/user/hg/otherproject

The paths setting in this file tells the CGI script which repositories it is allowed to publish and how their published locations map to their actual locations in the filesystem.

• The keys (on the left) are URL paths which can incorporate / characters - these paths appear as part of the URL used to access a repository

• The values (on the right) are filesystem paths which can be relative to the CGI directory - note that it is typically preferable to keep the repositories outside the CGI directory

If you are publishing a fixed set of repositories, using the paths setting should be sufficient, and you should not need to consider the collections setting described below.

5.1.1. Working with collections

Where many repositories are being served, it can be preferable to refer to each directory holding such collections of repositories instead of listing each and every repository as is done above. The above hgweb.config file could be rewritten as follows:

[collections]
/home/user/hg = /home/user/hg

The collections setting in this file tells the CGI script where to look for repositories.

• The keys (on the left) and the values (on the right) are both filesystem paths

• The keys should be prefixes of the values and are "subtracted" from the values in order to generate the URL paths to each repository

Consider two repository collections given in the hgweb.config file:

[collections]
/home/user/private = /home/user/private
/home/user/official = /home/user/official

Where both /home/user/private and /home/user/official contain repositories using the same names (myproject and otherproject), a combined list will then be shown by the Web interface containing two entries for each of someproject and otherproject: one from the private collection and one from the official collection. This may be confusing to the end-user, so we may modify the configuration file as follows:

[collections]
/home/user = /home/user

This will now produce entries for private/someproject, private/otherproject, official/someproject and official/otherproject. Unfortunately, it will also find other repositories outside the private and official directories. It is therefore recommended that repositories are located in suitably organised directory hierarchies if exported in this way.

6. Configuring Apache

There are many ways of configuring Apache to run CGI scripts, and a few of the possibilities are provided below. Where the main configuration files are mentioned, you should use the appropriate conventions for your system in defining such files in the conf.d and/or sites-available directories.

In each example, hgwebdir.cgi is mentioned, but the same principles apply to directories hosting the hgweb.cgi script.

To ensure that a script is executable by the Web server, the following command is typically used:

chmod u+x hgwebdir.cgi

The preferred mechanism for persuading Apache to use updated configuration information can vary from platform to platform and from distribution to distribution. Please consult your distribution's documentation, if appropriate, or the more general Apache documentation (for example, the apachectl documentation) for details.

6.1. Publishing the CGI script directly

This example requires access to the main configuration files.

The easiest way to serve the hgwebdir.cgi script is to use a ScriptAlias directive:

ScriptAlias /hg "/home/user/webdir/hgwebdir.cgi"

This actually exports the repository browser at the URL path /hg (for example, http://www.example.com/hg) and doesn't expose the name of the script at all.

See the Apache httpd documentation for ScriptAlias.

6.2. Using a simple CGI directory

This example requires access to the main configuration files.

If the directory containing the script is supposed to hold this and other CGI programs, such a CGI directory can be configured as follows:

ScriptAlias /hg "/home/user/webdir"

This should permit URLs like http://www.example.com/hg/hgwebdir.cgi to show the repository browser. To hide the hgwebdir.cgi script name in URLs, more work is required (and is mentioned below).

See the Apache httpd documentation for ScriptAlias, especially for information about the pitfalls of putting CGI directories inside existing Web-accessible directories.

6.3. Using an .htaccess file

This example can be used with pre-configured CGI directories.

If you may not change the main Apache configuration files, you may still be able to use .htaccess file to make URLs nicer, as suggested in the previous section. Here is an .htaccess file which sits in the published webdir directory on the Web server and redirects http://www.example.com/hg/* URLs to the hgwebdir.cgi script inside that folder. As a result it would no longer be not necessary to mention the CGI script name in URLs: one could use http://www.example.com/hg/myproject instead of http://www.example.com/hg/hgwebdir.cgi/myproject.

# Taken from http://www.pmwiki.org/wiki/Cookbook/CleanUrls#samedir
# Used at http://ggap.sf.net/hg/
Options +ExecCGI
RewriteEngine On
#write base depending on where the base url lives
RewriteBase /hg
RewriteRule ^$ hgwebdir.cgi  [L]
# Send requests for files that exist to those files.
RewriteCond %{REQUEST_FILENAME} !-f
# Send requests for directories that exist to those directories.
RewriteCond %{REQUEST_FILENAME} !-d
# Send requests to hgwebdir.cgi, appending the rest of url.
RewriteRule (.*) hgwebdir.cgi/$1  [QSA,L]

A corresponding change in hgweb.config can be made to make sure that the nicer urls are used in the HTML produced by the CGI scripts:

[web]
baseurl = /hg

Where the CGI scripts are made to appear at the server root (for example, http://hg.example.com/), leave the baseurl setting blank:

[web]
baseurl =

Generally, the value specified should not end with a / character.

6.4. Adding authentication

The following configurations requires access to the main configuration files. They can be combined with the script or directory declarations to impose authentication and access restrictions on repositories.

To use these configurations with the pre-configured CGI directories, the Location directive start and end tags can be omitted, leaving the bare authentication-related directives.

6.4.1. Restrict to known users

This configuration restricts access to a known set of users as defined in the /home/user/hg/hgusers password file:

<Location /hg>
    AuthType Basic
    AuthName "Mercurial repositories"
    AuthUserFile /home/user/hg/hgusers
    Require valid-user
</Location>

Since the AuthType directive is set to Basic, passwords are communicated as plain text, and it is therefore recommended that this only be used with a server configured for HTTPS. See the Apache SSL documentation for more information.

To set up the password file, use the htpasswd tool as described in the relevant Apache documentation.

This alternative configuration employs digest authentication and thus offers an alternative to basic authentication and HTTPS:

<Location /hg>
    AuthType Digest
    AuthName "Mercurial repositories"
    AuthDigestProvider file
    AuthUserFile /home/user/hg/hgusers
    Require valid-user
</Location>

To set up the password file, use the htdigest tool as described in the relevant Apache documentation.

See the Apache mod_auth_digest documentation for more information on digest authentication and its limitations.

6.4.2. Restrict pushing to known users

To exercise finer control and to provide global read-only access to the repositories, but require authentication for pushing, a LimitExcept directive can be added. Here are the previous examples with such a directive in use. First with basic authentication:

<Location /hg>
    AuthType Basic
    AuthName "Mercurial repositories"
    AuthUserFile /home/user/hg/hgusers
    <LimitExcept GET>
        Require valid-user
    </LimitExcept>
</Location>

And with digest authentication:

<Location /hg>
    AuthType Digest
    AuthName "Mercurial repositories"
    AuthDigestProvider file
    AuthUserFile /home/user/hg/hgusers
    <LimitExcept GET>
        Require valid-user
    </LimitExcept>
</Location>

See the Apache documentation for LimitExcept for more information.

Now consult the instructions on allowing the push operation in Mercurial to complete this configuration task.

6.4.3. Using groups

Apache also provides support for user groups through the AuthGroupFile directive. Here it is in context:

    AuthUserFile /home/user/hg/hgusers
    AuthGroupFile /home/user/hg/hggroups

Instead of a Require directive involving users, the following directive can be used in its place. Here it is in context:

    <LimitExcept GET>
        Require group hobbits
    </LimitExcept>

Here, the hobbits group is defined in the nominated file as described in the relevant Apache documentation for AuthGroupFile, connecting users who will authenticate themselves with groups such as hobbits.

6.5. Using virtual hosts

This example requires access to the main configuration files.

Here is an example Apache configuration for publishing repositories at http://hg.example.com/:

<VirtualHost *>
  ServerName hg.example.com

  ServerAdmin webmaster@example.com
  CustomLog logs/access_log.example combined
  ErrorLog logs/error_log.example

  ScriptAlias / "/home/user/webdir/hgwebdir.cgi"
</VirtualHost>

The ScriptAlias directive is taken from the script example; all other directives support the virtual host hg.example.com.

Note that the CustomLog and ErrorLog directives may need to be changed to refer to files in standard locations such as /var/log/apache2 or /var/log/httpd, depending on how Apache is configured.

Here is a more complicated example using rewrite rules and explicit Directory directives. A description follows the example.

<VirtualHost *:80>
  ServerName hg.example.com

  ServerAdmin webmaster@example.com
  CustomLog logs/access_log.example combined
  ErrorLog logs/error_log.example

  RewriteEngine on
  RewriteRule (.*) /home/user/webdir/hgwebdir.cgi/$1

  # Or we can use mod_alias for starting CGI script and making URLs "nice":
  # ScriptAliasMatch ^(.*) /home/user/webdir/hgwebdir.cgi/$1

  <Directory "/home/user/webdir/">
    Order allow,deny
    Allow from all
    AllowOverride All
    Options ExecCGI
    AddHandler cgi-script .cgi
  </Directory>
</VirtualHost>

The directives in the above have the following purposes:

• The ServerName directive matches the hostname configured for the domain.

• The next section (ServerAdmin and so on) is just administrative cruft.

• The rewrite-related directives (RewriteEngine and RewriteRule) tell Apache to turn URIs ending in /myproject into /home/user/webdir/hgwebdir.cgi/myproject. This causes Apache to fire up the CGI script, giving it the remainder of the URI as an argument.

• Finally, the Directory section lets Apache know that we have a CGI script to look at.

See the Apache virtual hosts documentation for more information.

7. Allowing push

Make sure that your repository is writeable by the user running the Apache server (such as www-data), and that the repository's .hg/hgrc file (or your Web server user's .hgrc file, such as /home/www-data/.hgrc, or a system-wide hgrc file like /etc/mercurial/hgrc) contains the allowed users:

[web]
allow_push = frodo, sam

This would allow pushing for frodo and sam. You can allow pushing for everyone with the following:

[web]
allow_push = *

By default, pushing is only allowed via HTTPS. To permit HTTP pushing you have to add this to your repository's .hg/hgrc file (or your Web server user's .hgrc file, such as /home/www-data/.hgrc, or a system-wide hgrc file like /etc/mercurial/hgrc):

[web]
push_ssl = false

Now consult the instructions on configuring Apache to restrict pushing in order to set up the authentication/authorisation infrastructure.

7.0.1. Defining user credentials

To define credentials for the allowed users, use the htpasswd tool. For example:

htpasswd -c /home/user/hg/hgusers frodo

You will need to enter the desired password for the username frodo. Later, you can add more usernames without the -c option:

htpasswd /home/user/hg/hgusers sam

See the relevant Apache documentation for htpasswd.

8. Troubleshooting

Mercurial is executed on the server by Apache and therefore runs as the Apache user and group. If experiencing flaky behavior, it may be because the CGI script is failing because it does not have enough rights. In that case, you should check the log files, but you can also make some common-sense permissions checks.

8.1. Permissions

There are two ways that permissions problems primarily manifest themselves on the server: either you won't see any repositories at all (indicating missing read or execute permissions) or you won't be able to push to the server (indicating missing write permissions), which gives you the error message:

abort: ‘http://foo/bar’ does not appear to be an hg repository!

Another common error often related to permissions:

abort: HTTP Error 500: Internal Server Error

The best way to solve permissions problems is to grant the required permissions to the Apache group (the www-data group on Debian). You should have some familiarity with assigning permissions under Linux/Unix before attempting the following.

Suppose your main user is john, your Web server runs as www-data and your repositories are in /home/john/repositories. Then, execute the following commands to change the group for all files in your repositories on the server and make the files writable to the server process as well as make the home directory readable.

chown -R john:www-data /home/john/repositories
chmod -R g+rw /home/john/repositories
chmod g+x /home/john/repositories

For each repository, you will have to make both the repository folder and the .hg folder executable as well:

chmod g+x /home/john/repositories/rep1
chmod g+x /home/john/repositories/rep1/.hg

If each of your repositories are subdirectories from some main folder which only contains repositories (such as /var/www/html/hg/repos, with underlying repositories /var/www/html/hg/repos/repo1, /var/www/html/hg/repos/repo2, and so on), you may find it easier to remember to script the setting of these permissions. Write the following at the prompt to create a new executable shell script:

cat <<EOM >permission.sh
chown -R john:www-data repos
chmod -R g+rw repos
chmod g+x repos
chmod g+x repos/*
chmod g+x repos/*/.hg
EOM
sudo chown root:root permission.sh
sudo chmod u+x permission.sh

These assume your username is john, the Apache server's user's group is www-data, and the folder containing your repositories is called repos. Now you can update permissions for your entire repository by navigating to this containing directory and issuing a single command:

sudo ./permission.sh

Before you start crawling through logs to find out why your Mercurial server isn't letting you pull, push, or authenticate, run this command and see if it solves your issue.

It is important to note that the entire repository tree must be accessible by the Web server user (www-data in the above examples). For example, the tree /home/john/source/repos/hg/repo1 requires john, source, repos, and hg to be executable by the Web server user.

8.2. HTTP Error 400: Bad Request

A similar problem with pulling through http proxies is discussed here.

This error may occur when you try to push/pull changes to/from the web server. Mercurial will send out a long (~7k bytes) request line that the server refuses to process.

In Apache 2.2, the LimitRequestLine defaults to 8190 (bytes). To circumvent the problem, try increasing the request line limit e.g to 16380.

8.3. Mercurial in verbose mode

On certain occasions, observed when attempting to clone a repository via hgwebdir.cgi, the Web server will produce a 500 (Internal Server Error) status code, aborting the operation. This is due to extra output being sent by Mercurial to the Web server as part of an HTTP response.

If your repository's .hg/hgrc file (or your Web server user's .hgrc file, such as /home/www-data/.hgrc, or a system-wide hgrc file like /etc/mercurial/hgrc) includes the following setting, you should consider removing or disabling it (or moving it to your normal user's .hgrc file, if appropriate):

verbose = true

8.4. Using an unsupported version of python

When the Web server produces a 500 (Internal Server Error) status code, the error log may contain messages resembling the following:

/var/lib/python-support/python2.6/mercurial/hgweb/common.py:24: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6

Such messages are likely to upset the Web server if issued by CGI scripts. The solution is either to use an appropriate version of Python for the version of Mercurial in use or to suppress deprecation warnings.

In the above example, Python 2.6 is being used to run code which uses deprecated features and has not been reviewed for use with that version of Python. If an earlier version of Python is available, such as Python 2.5, and if Mercurial has been set up for use by that Python version, you can change the first line of hgwebdir.cgi (or hgweb.cgi) as follows (assuming a system-wide installation of Python 2.5):

#!/usr/bin/python2.5

To suppress warnings, it may be sufficient to modify hgwebdir.cgi (or hgweb.cgi), adding the following code immediately after any sys.path adjustments:

import warnings
warnings.simplefilter("ignore", DeprecationWarning)

Note that this latter solution is not really recommended as it can hide issues with the code that should ultimately be addressed.

8.5. Problems with trust settings

Trust-related messages of the form Not trusting file... may appear in the Web server logs. Although it is unlikely that such messages would cause a 500 (Internal Server Error) status code, it can be useful to eliminate them in order to troubleshoot other problems.

9. Putting useful information in the index page

If you get everything working properly, pointing a browser at the CGI script directly should give a list of the repositories you've published. This will be a table containing four columns. Let's say you have published a repository named lord/rings. To fill out the first three columns of the index entry for that repository, you will need to edit its .hg/hgrc file, and add a new section:

[web]
contact = Bilbo Baggins
description = My precious!
name = lord/rings

10. Allowing archive downloads

Make sure that your repository's .hg/hgrc file (or your Web server user's .hgrc file, such as /home/www-data/.hgrc, or a system-wide hgrc file like /etc/mercurial/hgrc) contains the allow_archive setting:

[web]
allow_archive = gz, zip, bz2

This example illustrates how gzip, zip and bzip2 archive formats can be supported. As a result, links should appear in the Web interface corresponding to these archive types.

11. What can go wrong?

If the version of hgwebdir.cgi is newer than the version of Mercurial you have installed, you may experience strange results. This could happen if you use a binary installer for Mercurial, and manually fetch hgwebcir.cgi from a source repository. Newer versions of Mercurial support older versions of the CGI scripts, so you usually do not have to upgrade all your CGI installations, though it might be useful.

If you are trying to publish repositories, and you haven't configured Apache to force all accesses to go through the hgwebdir.cgi (or hgweb.cgi) script, you will not be able to access any of the repositories you have published. You may then be tempted to copy repositories into a directory published by your Web server, perhaps setting up a hgweb.cgi script in each published repository. Do not do this! Only the hgwebdir.cgi (or hgweb.cgi) script is meant to be published; when correctly functioning, the script will itself serve up the repository content, not the Web server.

Whatever mechanism you are trying to use, the important thing is to ensure that all accesses go through hgwebdir.cgi, so that Apache can pass the rest of the path to it using the PATH_INFO environment variable.

12. Theming

The hgweb interface is completely themable. See the Theming page for additional instructions on customizing the look of your site.

13. See also

• HgWebDirStepByStep for more details on publishing multiple Mercurial repositories through CGI

• modwsgi to do the same using WSGI and Apache

• Quick Tip: Getting Started with Mercurial describes Mercurial installation and repository serving using mod_python

• Vampire Basic Blog has a very good walkthrough on how to set up hgwebdir on IIS

• Setting up a Mercurial server under IIS7 on Windows Server 2008 R2 covers IIS and includes pictures to illustrate the process

CategoryWeb CategoryHowTo CategoryTipsAndTricks