Dirstate

Internals of the directory state cache.

Contents

Introduction

1. Introduction

Mercurial tracks various information about the working directory (the dirstate):

what revision(s) are currently checked out
what files have been copied or renamed
what files are controlled by Mercurial

For each file that Mercurial controls, we record the following information:

its size
its mode
its modification time
its "state"

The states that are tracked are:

n - normal
a - added
r - removed
m - 3-way merged

With this information, we can quickly determine what files in the working directory have changed.

Here's a real example of a dirstate of a clone of the Mercurial repository itself:

-  ⇤ ← Revision 17 as of 2012-02-24 16:33:05 → 
  Size: 14397
  Editor: WagnerBruna
  Comment: use interwiki links
+   ← Revision 20 as of 2012-11-06 09:04:21 → ⇥
  Size: 8194
  Editor: mpm
  Comment: moinfs edit
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
+#pragma section-numbers 2

<<Include(A:dev)>>
-Line 4:
+Line 8:
-/!\ This page is intended for developers.
-Line 163:
+Line 165:
-n   0         -1 unset               templates/header-gitweb.tmpl
n   0         -1 unset               templates/header-raw.tmpl
n   0         -1 unset               templates/header-rss.tmpl
n   0         -1 unset               templates/header.tmpl
n   0         -1 unset               templates/index.tmpl
n   0         -1 unset               templates/manifest-gitweb.tmpl
n   0         -1 unset               templates/manifest.tmpl
n   0         -1 unset               templates/map
n   0         -1 unset               templates/map-cmdline.changelog
n   0         -1 unset               templates/map-cmdline.compact
n   0         -1 unset               templates/map-cmdline.default
n   0         -1 unset               templates/map-gitweb
n   0         -1 unset               templates/map-raw
n   0         -1 unset               templates/map-rss
n   0         -1 unset               templates/notfound.tmpl
n   0         -1 unset               templates/search-gitweb.tmpl
n   0         -1 unset               templates/search.tmpl
n   0         -1 unset               templates/shortlog-gitweb.tmpl
n   0         -1 unset               templates/static/hgicon.png
n   0         -1 unset               templates/static/style-gitweb.css
n   0         -1 unset               templates/static/style.css
n   0         -1 unset               templates/summary-gitweb.tmpl
n   0         -1 unset               templates/tagentry-rss.tmpl
n   0         -1 unset               templates/tags-gitweb.tmpl
n   0         -1 unset               templates/tags-rss.tmpl
n   0         -1 unset               templates/tags.tmpl
n   0         -1 unset               templates/template-vars.txt
n   0         -1 unset               tests/README
n   0         -1 unset               tests/fish-merge
n   0         -1 unset               tests/md5sum.py
n   0         -1 unset               tests/run-tests
n   0         -1 unset               tests/test-addremove
n   0         -1 unset               tests/test-addremove.out
n   0         -1 unset               tests/test-archive
n   0         -1 unset               tests/test-archive.out
...
}}}

For files having state "n" in the dirstate, Mercurial compares the file modification time and the size in the dirstate with the modification time and the size of the file in the working directory. If both the modification time ''and'' the size are the same, Mercurial will assume it has not changed and will thus not include it in the next [[Commit|commit]].

Having size "-1" and date "unset" means that Mercurial assumes nothing about the contents of that file and will have to look into the file to determine whether it has changed or not. See also [[Cset:af3f26b6bba4|dirstate: ignore stat data for files that were updated too recently]], changeset {{{af3f26b6bba4}}} by Alexis S. L. Carvalho (introduced in Mercurial release 1.0); and an explanation given by Matt Mackall in http://selenic.com/pipermail/mercurial/2008-August/020984.html for this.

== File format ==

{{{
.hg/dirstate:
<p1 binhash><p2 binhash>
<list of dirstate entries>
}}}

a dirstate entry is composed of:
{{{
8bit: status
32bit: mode
32bit: size
32bit: mtime
32bit: length
variable length entry (length given by the previous length field) with:
"<filename>" followed if it's a copy by: "\0<source if copy>"
}}}

status can be either:
 * 'n': normal
 * 'm': merged
 * 'a': added
 * 'r': removed

=== Details of the semantics ===

mode stores the st.st_mod of the file as it was clean, but only the user
x-bit is ever checked

size is usually the size of the file, as it was stored (after any
potential filters).
If size is -1 or -2, it has a different semantic. First -1, in
conjunction with mtime can be used to force a lookup.
Secondly they are although use used when the dirstate is in a merge
state (p1 != nullid):
-2 will *always* return dirty, it is used to mark a file that was
cleanly picked from p2
With a status of 'r', -2 means that the previous state was -2 (always
dirty, picked from p2), -1 means the previous status was 'm' (merged),
those allows revert to pick the right status back during a merge.

mtime is usually the mtime of the file when it was last clean. If the
size is < 0, setting -1 as mtime will force a lookup (and allows us to
correctly deal with changes done less than one second after we updated
the dirstate).

== Summary ==

In summary, we have the additional "meta" status:
 * 'nl' : normallookup (status == 'n', size == -1, mtime == -1 (or sometimes 0))
 * 'np2': merged from other parent (status == 'n', size == -2)
 * 'rm' : removed and previous state was 'm' (status == 'r', size == -1)
 * 'rp2': removed and previous state was 'np2' (status == 'r', size == -2)

And we can notice that no bits from mode are used, except 0x40 (user
x-bit). Assuming the bits from stat.st_mode are portable across
plateform and OSs, the upper bits are set in the following way (in binary)

{{{
S_IFIFO  0001 /* FIFO.  */
S_IFCHR  0010 /* Character device.  */
S_IFLNK  1010 /* Symbolic link.  */
S_IFBLK  0110 /* Block device.  */
S_IFDIR  0100 /* Directory.  */
S_IFREG  1000 /* Regular file.  */
S_IFSOCK 1100 /* Socket.  */ 
}}}

Since hg should only add regular files or symlinks to the dirstate, it means we
can signal the presence of the extended dirstate entry by setting either
0100, or 0001. Then we can use the remaining bits (30 free bits!) to
encode whatever we want.

== Proposed extensions ==

 * 'l' flag (is the entry a symlink)
 * 'fallback-x': should the on-disk file be considered as having the x-bit set,
  useful if the FS doesn't support exec bit, the bit can still be
  changed with a git patch).
 * 'fallback-l': should the on disk-file be considered a symlink (useful
  if the FS doesn't support symlinks, they can still be added to the
  repo, with hg import and a git patch for example)
 * correctly mark 'np2', for merges we can use a bit to indicate if the
  file is clean from p1 or from p2.
 * anything else?

== See also ==

 * [[RepositoryCorruption]] for info about dirstate corruption

----
CategoryInternals
+n   0

Diff for "DirState"

Dirstate

1. Introduction