Differences between revisions 4 and 23 (spanning 19 versions)
Revision 4 as of 2006-12-10 19:53:49
Size: 1360
Editor: 10
Comment:
Revision 23 as of 2015-03-08 04:19:32
Size: 2332
Editor: KevinBullock
Comment: clean up formatting
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
RevlogNG was introduced with Mercurial 0.9. #pragma section-numbers 2
Line 3: Line 3:
Deficiencies in original revlog format: <<Include(A:dev)>>

= RevlogNG =

RevlogNG was introduced with Mercurial 0.9 (see also [[Revlog]]).

<<TableOfContents>>

== Deficiencies in original revlog format ==
Line 17: Line 25:
 * 20 bytes: nodeid
Line 20: Line 27:
 * '''72 bytes total'''  * 20 bytes: nodeid
* '''76 bytes total'''
Line 22: Line 30:
RevlogNG format: == RevlogNG format ==
Line 24: Line 32:
 * 6 bytes: offset -- This is how far into the data file we need to go to find the appropriate delta
Line 25: Line 34:
 * 6 bytes: offset (allows for 256TB of compressed history per file)
 * 4 bytes: compressed length
 * 4 bytes: uncompressed length
 * 4 bytes: base revision
 * 4 bytes: link revision
 * 4 bytes: parent 1 revision
 * 4 bytes: parent 2 revision
 * 32 bytes: nodeid
 * 4 bytes: compressed length -- Once we are offset into the data file, this is how much we read to get the delta
 * 4 bytes: uncompressed length -- This is just an optimization. It's the size of the file at this revision
 * 4 bytes: base revision -- The last revision where the entire file is stored.
 * 4 bytes: link revision -- Another optimization. Which revision is this? Which commit is this?
 * 4 bytes: parent 1 revision -- Revision of parent 1 (e.g., 12, 122)
 * 4 bytes: parent 2 revision -- Revision of parent 2
 * 32 bytes: nodeid -- A unique identifier, also used in verification (hash of content + parent IDs)
Line 35: Line 43:
RevlogNG header: === RevlogNG header ===
Line 37: Line 45:
As the offset of the first data chunk is always zero, the first offset is used to indicate revlog version number and flags. As the offset of the first data chunk is always zero, the first 4 bytes (part of the offset) are used to indicate revlog version number and flags. all values are in big endian format.
Line 39: Line 47:
RevlogNG also supports interleaving of index and data. This can greatly reduce storage overhead for smaller revlogs. In this format, the data chunk immediately follows its index entry. The position of the next index entry is calculated by adding the compressed length to the offset.  RevlogNG also supports interleaving of index and data. This can greatly reduce storage overhead for smaller revlogs. In this format, the data chunk immediately follows its index entry. The position of the next index entry is calculated by adding the compressed length to the offset.

For how renames are stored see "[[http://selenic.com/pipermail/mercurial/2008-February/017139.html|Problems extracting renames]]", a reply by [[mpm]] posted on Feb 12, 2008, on the [[http://selenic.com/pipermail/mercurial/|mercurial mailing list]].

== See Also ==

 * ParentDeltaPlan
Line 42: Line 56:
CategoryNewFeatures CategoryInternals

[[FrenchRevlogNG|Français]]

Note:

This page is primarily intended for developers of Mercurial.

RevlogNG

RevlogNG was introduced with Mercurial 0.9 (see also Revlog).

1. Deficiencies in original revlog format

  • no uncompressed revision size stored
  • SHA1 hash is potentially too weak
  • compression context for deltas is often too small
  • offset range is limited to 4MB
  • some metadata is indicated by escaping in the data

The original index format was:

  • 4 bytes: offset
  • 4 bytes: compressed length
  • 4 bytes: base revision
  • 4 bytes: link revision
  • 20 bytes: parent 1 nodeid
  • 20 bytes: parent 2 nodeid
  • 20 bytes: nodeid
  • 76 bytes total

2. RevlogNG format

  • 6 bytes: offset -- This is how far into the data file we need to go to find the appropriate delta
  • 2 bytes: flags
  • 4 bytes: compressed length -- Once we are offset into the data file, this is how much we read to get the delta
  • 4 bytes: uncompressed length -- This is just an optimization. It's the size of the file at this revision
  • 4 bytes: base revision -- The last revision where the entire file is stored.
  • 4 bytes: link revision -- Another optimization. Which revision is this? Which commit is this?
  • 4 bytes: parent 1 revision -- Revision of parent 1 (e.g., 12, 122)
  • 4 bytes: parent 2 revision -- Revision of parent 2
  • 32 bytes: nodeid -- A unique identifier, also used in verification (hash of content + parent IDs)
  • 64 bytes total

2.1. RevlogNG header

As the offset of the first data chunk is always zero, the first 4 bytes (part of the offset) are used to indicate revlog version number and flags. all values are in big endian format.

RevlogNG also supports interleaving of index and data. This can greatly reduce storage overhead for smaller revlogs. In this format, the data chunk immediately follows its index entry. The position of the next index entry is calculated by adding the compressed length to the offset.

For how renames are stored see "Problems extracting renames", a reply by mpm posted on Feb 12, 2008, on the mercurial mailing list.

3. See Also


CategoryInternals

Français

RevlogNG (last edited 2020-01-13 04:19:53 by aayjaychan)