Differences between revisions 4 and 5
Revision 4 as of 2016-04-28 15:16:21
Size: 1262
Comment: hgwatchman is now in core as fsmonitor
Revision 5 as of 2016-04-28 15:26:47
Size: 3197
Comment: Copy in and format extension documentation.
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
= hgwatchman = = fsmonitor =
Line 6: Line 6:
Integrates the file-watching program watchman with Mercurial to produce faster status results.
On a particular Linux system, for a real-world repository with over 200,000 files hosted on ext4, vanilla hg status takes over 3 seconds. On the same system, with hgwatchman it takes under 0.6 seconds.
'''This extension is currently being distributed along with Mercurial.'''

''Author: '' ''Facebook, Inc.''
Line 11: Line 12:
== Status == == Overview ==
Line 13: Line 14:
'''This extension is not distributed with Mercurial.''' Integrates the file-watching program Watchman with Mercurial to produce faster
status results.
Line 15: Line 17:
This extension is slated to be integrated into core Mercurial. On a particular Linux system, for a real-world repository with over 400,000
files hosted on ext4, vanilla `hg status` takes 1.3 seconds. On the same
system, with fsmonitor it takes about 0.3 seconds.
Line 17: Line 21:
''Author: '' ''Facebook, Inc.'' == Configuration ==
Line 19: Line 23:
Web page: ''https://bitbucket.org/facebook/hgwatchman'' Configure your ''`.hgrc`'' to enable the extension by adding following lines:
Line 21: Line 25:
== Upstreaming == {{{
[extensions]
fsmonitor =
}}}
Line 23: Line 30:
 * Only one open pull request in current repo, plus one issue, perhaps need resolving before merge.
 * Windows and Solaris support is in Alpha state.
fsmonitor requires no configuration -- it will tell Watchman about your
repository as necessary. You'll need to install Watchman from
https://facebook.github.io/watchman/ and make sure it is in your PATH.
Line 26: Line 34:
The code should be mergeable as-is (just clean up for style guide and compatibility compliance). The test suite largely relies on the default mercurial suite, we'll need to look into how this can be integrated in a way that doesn't require manual enabling of the extension, and then re-run. The following configuration options exist:
Line 28: Line 36:
Facebook impact: high; it is in wide use within Facebook. {{{
[fsmonitor]
mode = {off, on, paranoid}
}}}

When `mode = off`, fsmonitor will disable itself (similar to not loading the
extension at all). When `mode = on`, fsmonitor will be enabled (the default).
When `mode = paranoid`, fsmonitor will query both Watchman and the filesystem,
and ensure that the results are consistent.

{{{
[fsmonitor]
timeout = (float)
}}}

A value, in seconds, that determines how long fsmonitor will wait for Watchman
to return results. Defaults to `2.0`.

{{{
[fsmonitor]
blacklistusers = (list of userids)
}}}

A list of usernames for which fsmonitor will disable itself altogether.

{{{
[fsmonitor]
walk_on_invalidate = (boolean)
}}}

Whether or not to walk the whole repo ourselves when our cached state has been
invalidated, for example when Watchman has been restarted or .hgignore rules
have been changed. Walking the repo in that case can result in competing for
I/O with Watchman. For large repos it is recommended to set this value to
false. You may wish to set this to true if you have a very fast filesystem
that can outpace the IPC overhead of getting the result data for the full repo
from Watchman. Defaults to false.

== Platforms Supported ==

 * '''Linux:''' ''Stable''. Watchman and fsmonitor are both known to work reliably, even under severe loads.

 * '''Mac OS X:''' ''Stable''. The Mercurial test suite passes with fsmonitor turned on, on case-insensitive HFS+. There has been a reasonable amount of user testing under normal loads.

 * '''Solaris, BSD:''' ''Alpha''. watchman and fsmonitor are believed to work, but very little testing has been done.

 * '''Windows:''' 'Alpha'. Not in a release version of watchman or fsmonitor yet.

== Known Issues ==

 * fsmonitor will disable itself if any of the following extensions are enabled: largefiles, inotify, eol; or if the repository has subrepos.
 * fsmonitor will produce incorrect results if nested repos that are not subrepos exist. *Workaround*: add nested repo paths to your `.hgignore`.

The issues related to nested repos and subrepos are probably not fundamental
ones. Patches to fix them are welcome.

fsmonitor

This extension is currently being distributed along with Mercurial.

Author: Facebook, Inc.

1. Overview

Integrates the file-watching program Watchman with Mercurial to produce faster status results.

On a particular Linux system, for a real-world repository with over 400,000 files hosted on ext4, vanilla hg status takes 1.3 seconds. On the same system, with fsmonitor it takes about 0.3 seconds.

2. Configuration

Configure your .hgrc to enable the extension by adding following lines:

[extensions]
fsmonitor =

fsmonitor requires no configuration -- it will tell Watchman about your repository as necessary. You'll need to install Watchman from https://facebook.github.io/watchman/ and make sure it is in your PATH.

The following configuration options exist:

[fsmonitor]
mode = {off, on, paranoid}

When mode = off, fsmonitor will disable itself (similar to not loading the extension at all). When mode = on, fsmonitor will be enabled (the default). When mode = paranoid, fsmonitor will query both Watchman and the filesystem, and ensure that the results are consistent.

[fsmonitor]
timeout = (float)

A value, in seconds, that determines how long fsmonitor will wait for Watchman to return results. Defaults to 2.0.

[fsmonitor]
blacklistusers = (list of userids)

A list of usernames for which fsmonitor will disable itself altogether.

[fsmonitor]
walk_on_invalidate = (boolean)

Whether or not to walk the whole repo ourselves when our cached state has been invalidated, for example when Watchman has been restarted or .hgignore rules have been changed. Walking the repo in that case can result in competing for I/O with Watchman. For large repos it is recommended to set this value to false. You may wish to set this to true if you have a very fast filesystem that can outpace the IPC overhead of getting the result data for the full repo from Watchman. Defaults to false.

3. Platforms Supported

  • Linux: Stable. Watchman and fsmonitor are both known to work reliably, even under severe loads.

  • Mac OS X: Stable. The Mercurial test suite passes with fsmonitor turned on, on case-insensitive HFS+. There has been a reasonable amount of user testing under normal loads.

  • Solaris, BSD: Alpha. watchman and fsmonitor are believed to work, but very little testing has been done.

  • Windows: 'Alpha'. Not in a release version of watchman or fsmonitor yet.

4. Known Issues

  • fsmonitor will disable itself if any of the following extensions are enabled: largefiles, inotify, eol; or if the repository has subrepos.
  • fsmonitor will produce incorrect results if nested repos that are not subrepos exist. *Workaround*: add nested repo paths to your .hgignore.

The issues related to nested repos and subrepos are probably not fundamental ones. Patches to fix them are welcome.


CategoryExtensionsByOthers

FsMonitorExtension (last edited 2017-01-26 21:01:58 by NathanGoldbaum)