''The new bundle format design is described on the BundleFormat2 page. ''''' '''


Bundle format is the format in which ''changegroups'' are exchanged. It is used in the WireProtocol, as well as from the command line.

On the command line, bundle files are generated with the `hg bundle` command. They consist of a header, followed by a block of binary data, which may be compressed. The header is 6 bytes long and indicates the compression type:

 * '''HG10BZ''' - Compressed with the Python 'bz2' module
 * '''HG10GZ''' - Compressed with the Python 'zlib' module
 * '''HG10UN''' - Not compressed.

<<TableOfContents>>

== Decompressor in Python ==
The following is a Python program to convert an ''HG10BZ'' or ''HG10GZ'' file into an ''HG10UN'' file:

{{{#!PYTHON
#!/usr/bin/python

# Program to decompress Mercurial bundles
# This program contains extracts from the Mercurial
# source, and is therefore subject to the GNU General
# Public License.

import bz2, zlib, sys

def decompress(filename):
        infile = open(filename, "rb")
        outfile = open(filename+'.uncompressed', "wb")
        outfile.write('HG10UN')
        for chunk in unbundle(infile):
                outfile.write(chunk)

def filechunkiter(f, size=65536, limit=None):
    """Create a generator that produces the data in the file size
    (default 65536) bytes at a time, up to optional limit (default is
    to read all data).  Chunks may be less than size bytes if the
    chunk is the last chunk in the file, or the file is a socket or
    some other type of file that sometimes reads less data than is
    requested."""
    assert size >= 0
    assert limit is None or limit >= 0
    while True:
        if limit is None: nbytes = size
        else: nbytes = min(limit, size)
        s = nbytes and f.read(nbytes)
        if not s: break
        if limit: limit -= len(s)
        yield s


def unbundle(fh):
    header = fh.read(6)
    if header == 'HG10UN':
        return fh
    elif not header.startswith('HG'):
        # old-style uncompressed bundle with no header - we've read into actual data
        fh.seek(0)
        def generator(f):
            yield header
            for chunk in f:
                yield chunk
    elif header == 'HG10GZ':
        def generator(f):
            zd = zlib.decompressobj()
            for chunk in f:
                yield zd.decompress(chunk)
    elif header == 'HG10BZ':
        def generator(f):
            zd = bz2.BZ2Decompressor()
            zd.decompress("BZ")
            for chunk in filechunkiter(f, 4096):
                yield zd.decompress(chunk)
    return generator(fh)

if len(sys.argv) != 2:
   print "Usage: expandbundle <file>"
   exit()

decompress(sys.argv[1])
}}}
== Overview of v1 Bundles ==
 . To understand how the bundle format works, it is helpful to understand  how changesets are committed to the repository. You might want to take a  look at [[http://mercurial.selenic.com/wiki/ChangeSet#Committing_a_new_changeset|ChangeSet#Committing_a_new_changeset]].

A bundles contains all the information necessary to add one or more changesets to a repository. A changest includes:

 * a particular version of the ChangeLog
 * a particular version of the [[Manifest]]
 * a particular version of content for each tracked file

As such, a bundle contains the following for each changeset it includes:

 * changes made to the ChangeLog
 * changes made to the Manifest
 * changes made to a set of files

The bundle is divided into three corresponding sections, with a common structure called a Group used in each section (with a slight variation for the files section). Each Group is composed of one or more structures called Chunks, each of which is simply a 4-byte len field followed by data. The len field is interpretted as a big-endian integer and specifies the number of bytes in the ''entire'' Chunk, i.e., it includes its own 4 bytes.
||||<tablewidth="365px" tableheight="59px"style="text-align:center">'''Chunk ''' ||
||'''4 bytes - big-endian''' ||'''(''len'' - 4) bytes''' ||
||''len'' ||data ||




The group is terminated by a ''NullChunk'', which is simply a Chunk whose ''len'' is no more than 4 and therefore has no data (but a NullChunk always has all 4 bytes for the len field).
||||||||<tablewidth="365px" tableheight="59px"style="text-align:center">'''Group''' ||
||Chunk 0 ||... ||Null Chunk ||




The changelog section and the manifest section of the bundle are both simply one group each. The final section is called the Filelist, and it is sequence of two-tuples, one two-tuple for each file that was modified by any one of the changesets in the bundle. Each two-tuple in the Filelist contains the file's path, in the form of a chunk, and a Group. the filelist is termianted by a NullChunk (in place of the next filepath chunk):
||||||||||||||<tablewidth="889px" tableheight="108px"style="text-align:center">'''Filelist''' ||
||||<style="text-align:center">'''FileEntry 0''' ||||<style="text-align:center">... ||||<style="text-align:center">'''FileEntry F-1''' ||'''Terminator''' ||
||filepath (Chunk) ||filedata (Group) ||||<style="text-align:center"> ||filepath (Chunk) ||filedata (Group) ||NullChunk ||




Putting it all together a bundle looks like this:
||||||||||||||||||||||||<tablewidth="1014px" tableheight="88px"style="text-align:center">'''Bundle''' ||
||||||||<style="text-align:center">'''Changelog (Group)<<BR>>''' ||||||||<style="text-align:center">'''Manifest (Group)<<BR>>''' ||||||||<style="text-align:center">'''Filelist<<BR>>''' ||
||Chunk 0 ||... ||Chunk C-1 ||NullChunk ||Chunk 0 ||... ||Chunk M-1 ||NullChunk ||FileEntry 0 ||... ||FileEntry F-1 ||NullChunk ||




=== Inside the Groups ===
Inside each Group (1 Group for the changelog, 1 Group for the manifest, 1 Group for each of the modified files), there is 1 Chunk for each changeset which is included in the Bundle. These Chunks are a special species of Chunk called a RevChunk, which have the Chunk data further divided into the following fields:
||'''4 bytes - big-endian''' ||'''20 bytes - big-endian''' ||'''20 bytes - big-endian''' ||'''20 bytes - big-endian''' ||'''20 bytes - big-endian''' ||'''(''len'' - 84) bytes''' ||
||''len'' ||''node'' ||''p1 (parent 1)'' ||''p2 (parent 2)'' ||''cs (changeset link)'' ||''revdata'' ||




The ''len'' field is the same as for all other Chunks, it specifies to the total number of bytes in the chunk. The next four fields are each 20 byte nodeids, stored in big-endian binary form (as opposed to the ASCII hexidecimal form commonly seen by the user). Each RevChunk contains the data needed to create a new entry in the corresponding revlog (revlog for the changelog, manifest, or tracked file), and the required nodeids are stored in the four fields. The ''node'' field is the identifier for the new entry that the RevChunk creates, while p1 and p2 are the nodeids for the new entry's parents. The ''cs'' fields is the [[ChangeSetID|ChangeSetId]] for the changeset that this RevChunk belongs to.

Lastly, the ''revdata'' field contains a sequence of structures called RevDiffs, each with the following format:
|| '''4 bytes - big-endian''' || '''4 bytes - big-endian''' || '''4 bytes - big-endian''' || ''blocklen'' bytes ||
|| ''start'' || ''end'' || ''blocklen'' || ''textdata'' ||

Note that the sequence of RevDiffs does not need to be terminated, because the total length of the ''revdata'' is known (len - 84).

Each RevDiff item specifies a simple single-hunk patch, in the form of a replacement. The RefDiff item says to replace the bytes from offset ''start'' up to (but not including) offset ''end'' with the specified ''textdata''. To delete text, the ''textdata'' would be empty (''blocklen'' would be 0), and to insert text, the ''start'' and ''end'' would be the same.

For instance, the following python code applies a patch specified by a RevDiff:

{{{#!PYTHON
#!/usr/bin/python

def applypatch(original, start, end, textdata):
    pre = original[:start]
    post = original[end:]
    return pre + textdata + post

}}}

Taken all together, the sequence of RevDiffs in a RevChunk's ''revdata'' field indidate how to transform the parent version into the new version. For instance, for a RevChunk in the Changelog, it specifies how to change the parent changelog into the new changelog. For a RevChunk in on of the FileEntry's, it specifies how to change the parent version of the file into the new version of the file.

Note that the RevDiffs in a given RevChunk must be in order from the beginning of the file to the end of the file, and the ''start'' and ''end'' offsets always refer to offsets in the original file, ''not'' the results of applying the previous RevDiffs in the sequence. This allows individual RevDiffs to be applied selectively, without applying any others. However, it means that in order to apply more than one RevDiff, the must be applied in ''reverse'' order: patches that have a greater ''start'' offset must be applied first so that they don't change the offsets for other patches.

For the first revision of a file (or of the changelog or manifest), the RevDiffs are applies against an empty string.