Diff for "RevlogNG"

Some thoughts on possible improvements to Revlog.

Things that could use fixing:

• no uncompressed revision size stored
• SHA1 hash is potentially too weak
• compression context for deltas is often too small
• offset range is limited to 4MB
• some metadata is indicated by escaping in the data

The current index layout is:

• 4 bytes: offset
• 4 bytes: compressed length
• 4 bytes: base revision
• 4 bytes: link revision
• 20 bytes: nodeid
• 20 bytes: parent 1 nodeid
• 20 bytes: parent 2 nodeid

• 72 bytes total

Possible future layout:

• 2 bytes: flags
• 6 bytes: offset (allows for 256TB of compressed history per file)
• 4 bytes: compressed length
• 4 bytes: uncompressed length
• 4 bytes: base revision
• 4 bytes: link revision
• 4 bytes: parent 1 revision
• 4 bytes: parent 2 revision
• 32 bytes: nodeid

• 64 bytes total

Also, it may be interesting to compress entire spans with a single gzip context, with flushes between revisions. This will possibly result in much better compression.

CategoryNewFeatures