Diff for "BundleFormat2"

Differences between revisions 1 and 14 (spanning 13 versions)

Note:

This page is primarily intended for developers of Mercurial.

This page describes the current plan to get a more modern and complete bundle format. (for old content of this page check BundleFormatHG19)

Contents

New bundle format
Changes in Push Orchestraction
1. Current situation
2. Aimed orchestration
Changesets exchange
1. New header

(current content is copy pasted from 2.9 sprint note)

New bundle format

lightweight
new manifest
general delta
bookmarks
phase boundaries
obsolete markers
>sha1 support
pushkey
extensible for new features (required and optional)
progress information
resumable?
transaction commit markers?

It's possible to envision a format that sends a change, its manifest, and filenodes in each chunk rather than sending all changesets, then all manifests, etc. capabilities

Changes in Push Orchestraction

Current situation

push:
- changesets:
  - discovery
  - validation
  - actual push
- phase:
  - discovery
  - pull
  - push
- obsolescence
  - discovery
  - push
- bookmark
  - discovery
  - push

Aimed orchestration

* push:

discovery:
- changesets
- phase
- obs
- bookmark
post-discovery action:
- current usecase move phase for common changeset seen as public.
local-validation:
- (much easier will everything in hands)
- complains about:
  - multiple heads
  - new branch
  - troubles changeset
  - divergent bookmark
  - Rent in Manhattan
  - etc…
push:
- (using multipart-bundle when possible)
  - The one and single remote side transaction happen here
pull:
- (mostly for phase)
  - and any other data the server send as a reply to the multipart-bundle The server would be able to reply a multi-bundle. To inform the client of potential phase//bookmark//changeset rewrites etc…

Changesets exchange

New header

type Header struct {
    length       uint32
    lNode        byte
    node         [lNode]byte

    // if empty (lP1 ==0) then default to previous node in the stream
    lP1          byte
    p1           [lP1]byte

    // if empty, nullrev
    lP2          byte
    p2           [lP2]byte

    // if empty, self (for changelogs)
    lLinknode    byte
    linknode     [lLinknode]byte

    // if empty, p1
    lDeltaParent byte
    deltaParent  [lDeltaParent]byte 
}

We'll modify the existing changegroup type so it can pretend to be a new changegroup that just has a variety of empty fields. Progress information fields might be optional.

CategoryNewFeatures

-  ⇤ ← Revision 1 as of 2011-05-11 12:24:36 → 
  Size: 2108
  Editor: cyanite
  Comment: initial version
+   ← Revision 14 as of 2014-03-10 19:57:35 → ⇥
  Size: 2643
  Editor: Pierre-YvesDavid
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-This page describes the second iteration of the bundle format, tentatively called HG19 (since we hope to include it with Mercurial 1.9).
+<<Include(A:dev)>>
 Line 3:
-Bundles consist of the following sections:
 * A bundle header, describing the version and features present in the data.
 * Changegroup sections for the changelog, the manifest and each relevant filelog.
 * Optionally, a footer containing an index for more efficient random access?
+This page describes the current plan to get a more modern and complete bundle format. (for old content of this page check [[BundleFormatHG19]])
-Line 10:
+Line 7:
-== Sections ==
=== Header ===
The bundle header has the following format:
+(current content is copy pasted from 2.9 sprint note)
-Line 14:
+Line 9:
-|| '''Offset''' || '''Size'''  || '''Type''' || '''Description''' ||
||<)> 0         ||<)> 4        || string     || Bundle format version. Always contains "HG19". ||
||<)> 4         ||<)> 2        || string     || Compression type. Either "BZ", "GZ" or "UN". ||
||<)> 6         ||<)> a        || string     || Bundle features (or requirements). A list of comma separated words (lowercase ASCII) describing features present in the bundle. The string is terminated by a newline character. ||
+=== New bundle format ===
-Line 19:
+Line 11:
-=== Changegroup sections ===
The changegroup sections has the following format:
+ * lightweight
 * new manifest
 * general delta
 * bookmarks
 * phase boundaries
 * obsolete markers
 * >sha1 support
 * pushkey
 * extensible for new features (required and optional)
 * progress information
 * resumable?
 * transaction commit markers?
    
It's possible to envision a format that sends a change, its manifest, and filenodes in each chunk rather than sending all changesets, then all manifests, etc.
capabilities
-Line 22:
+Line 27:
-|| '''Offset''' || '''Size'''  || '''Type''' || '''Description''' ||
||<)> 0         ||<)> 4        || uint       || Number of changelog entries. ||
||<)> 4         ||<)> b        || group       || Changegroup containing changelog entries. ||
||<)> b + 4     ||<)> 4        || uint       || Number of manifest entries. ||
||<)> b + 8     ||<)> c        || group       || Changegroup containing manifest entries. ||
||<)> b + c + 8 ||<)> 4        || uint       || Number of filelog changegroups (note: not the number of entries). ||
+=== Changes in Push Orchestraction ===
 Line 29:
-Then, for each filelog, the following:
+==== Current situation ====
 Line 31:
-|| '''Offset''' || '''Size'''  || '''Type''' || '''Description''' ||
||<)> 0         ||<)> 4        || uint       || Number of filelog entries. ||
||<)> 4         ||<)> 4        || uint       || Length of filename. ||
||<)> 8         ||<)> d        || string     || Filename (unterminated). ||
||<)> d + 8     ||<)> e        || group       || Changroup containing filelog entries. ||
+ * push:
   * changesets:
     * discovery
     * validation
     * actual push
   * phase:
     * discovery
     * pull
     * push
   * obsolescence
     * discovery
     * push
   * bookmark
     * discovery
     * push
-Line 37:
+Line 47:
-== Changegroups ==
...
+==== Aimed orchestration ====

* push:
  * discovery:
    * changesets
    * phase
    * obs
    * bookmark
  * post-discovery action:
    * current usecase move phase for common changeset seen as public.
  * local-validation:
    * (much easier will everything in hands)
    * complains about:
      * multiple heads
      * new branch
      * troubles changeset
      * divergent bookmark
      * Rent in Manhattan 
      * etc…
  * push:
      * (using multipart-bundle when possible)
        The one and single remote side transaction happen here
  * pull:
      * (mostly for phase)
        and any other data the server send as a reply to the multipart-bundle
        
        The server would be able to reply a multi-bundle. To inform the client of potential phase//bookmark//changeset rewrites etc…

=== Changesets exchange ===

==== New header ====

{{{#!C
type Header struct {
    length       uint32
    lNode        byte
    node         [lNode]byte

    // if empty (lP1 ==0) then default to previous node in the stream
    lP1          byte
    p1           [lP1]byte

    // if empty, nullrev
    lP2          byte
    p2           [lP2]byte

    // if empty, self (for changelogs)
    lLinknode    byte
    linknode     [lLinknode]byte

    // if empty, p1
    lDeltaParent byte
    deltaParent  [lDeltaParent]byte 
}
}}}
We'll modify the existing changegroup type so it can pretend to be a new changegroup that just has a variety of empty fields. Progress information fields might be optional.



----
CategoryNewFeatures