Changesets Evolution - development page.

This page is intended for developer

For a user perspective have a look at the ChangesetEvolution page.

1. Contributing

The simplest way to help is to grab one of: list of easy bug

There is also a multiple well defined topic that where idea exist but needs an implementation

• Rebase could make more use of obsolescence marker:
  • detection that part of the rebase set is already in the destination
  • warning about divergence creation.

• Obsolescence markers exchange (no really we have idea waiting to be implemented)

• A hg split command

• Prototype for a command bringing changeset back to life.

There is more complicated part that requires attention too.

• Better storage//loading//cache for markers (depends on marker exchange)

• Solving the problem of Topic Branch

• Handling moving changesets around with the "plan" concept,

• in memory merge (helps a lot of troubles resolution throught evolve)

• Wide transaction capability
  • Continue/stop/abort for all command including evolve
  • In transaction content non exchangeable.
• Making evolve capable of solving all troubles that user can encounter
• Computing UI message about troubles from event who happened during the transaction.

2. Roadmap

Current status:

• Alpha Stage,

• Beta Stage,

• Release Stage,

2.1. Alpha Stage

Changeset Evolution is currently at Alpha Stage. If won't eat people data, but only handful of people knowns how to get out some situation..

2.2. Beta Stage

At beta state, the UI and experience will not be easy/pleasant enough for normal user, but advance user of Mercurial should find their mark. We may consider shipping it with Core Mercurial with an experimental flag.

Blocker to beta release:

• Obsmarkers exchange should be good enough to define a final storage format for markers.
• On disk format should be stable in the foreseeable future and fit performance//exchange need.
• Evolve should be able to solve (or provide a way to) all troubles that a user may encounter, especially divergence,
• Evolve should have predictable result (--rev options and co),
• Evolve should be abortable (wide transaction),

• Performance impact should be reasonable,

2.3. Release Stage

A which point we can merge changesets evolution into core.

• UI offering a Solution to the N² markers creation when editing history (TopicPlan),

• Commands set defined enough to be freezed for backward compatibility

• No race condition when exchanging with server (bundle2 + repo layout allowing atomic transaction),

• No impactful Performance Regression (including efficient exchange),
• Concrete plan to handle high volume of markers (archiving or something),

3. In progress Features

3.1. Using Obsolescence Marker during Rebase

There is two big issues with rebasing a set containing obsolescence changeset:

• It is easy to create divergence
• You get a lot of conflict when rebasing an obsolete stack on it's successors version.

To enable the current implementation set the config experimental.rebaseskipobsolete to true.

Current progress:

• rebase can skip obsolete changeset when rebased on successors,

• same logic as above, handling prune,

• same logic as above, handling split,

• ability to detect divergence creation and bail out,

• ability to rebase set with obsolescence inside the set (rebasing both precursors and successors) without creating divergence,

• official config to control these two behavior (either in one or two config)

• config on by default.

4. Archived Topic

4.1. Obsstore Format

Markers are stored in an append-only file stored in '.hg/store/obsstore'.

4.1.1. V1 (current) Format

(see in line document for latest data)

4.1.1.1. quick summary

• <number-of-successors(=N)><metadata-lenght(=M)><bits-field><precursor>(<successor>*N)<metadata>

• B, I, B, 20s, (20s*N), s*M

4.1.1.2. longer explanation

The file starts with a version header:

• 1 unsigned byte: version number, starting at zero.

The header is followed by the markers. Each marker is made of:

• 1 unsigned byte: number of new changesets "N", can be zero.
• 1 unsigned 32-bits integer: metadata size "M" in bytes.
• 1 byte: a bit field. It is reserved for flags used in common
  • obsolete marker operations, to avoid repeated decoding of metadata entries.
• 20 bytes: obsoleted changeset identifier.
• N*20 bytes: new changesets identifiers.
• M bytes: metadata as a sequence of nul-terminated strings. Each
  • string contains a key and a value, separated by a colon ':', without additional encoding. Keys cannot contain '\0' or ':' and values cannot contain '\0'.

4.1.2. V2 (current) Format

4.1.2.1. motivation

There is two extra information we would like to see in a second version of the format:

• date: There is currently *always* a date in the meta data. So storing it explicitly would be more space efficient. It would also open the way to quickly access the date for sorting purpose (no use case yet but not crazy to think about it)
• parents: When a changesets is pruned (obsoleted, no successors) we needs to records its parents. This is necessary to link the markers chain to the push/pull operation it is relevant to.
• We may want to extend the bit field to 2 bytes. We currently use 1 and can see use case for 3-5 others (tracking the type of changes introduce by the rewriting (desc, patches, content, etc) so we are running short
• We may also want to explicitly store the username of the marker's creator are they will always be ones. however there is no need for quick access of such information so it could stay in the metadata field.

4.1.2.2. possible change

Date:

• The date will be a 64bits float (for seconds since epoch) followed by a 16 bits integer (time zone)
• It will make sense to put the date in front of the markers. that would give markers sorting some semantic.

Parents:

We have multiple option for storing parents:

1. Having an explicite field similar to successors (one byte to know how many parents, then parents)
2. Having an explicite field but store the number of parent in the bit fields (since we never have more than 2 parents)
3. Using the successors field. Having negative number of successors mean it is a prune.

Option (3) is the most space saving but prevent use to store parent information for more changesets if needed in the future (We do not have a final exchange plan yet).

Option (1) and (2) takes 2 to 8 bits more than (3) but are more flexible.

bit field

If we extend the bit field to 2 Bytes, it makes sense to use option (2) for storing parent.

4.1.2.3. Proposed Format

• <date><timezone><number-of-successors(=N)><metadata-lenght(=M)><bits-field+nb-parents(=P)><precursor>(<successor>*N)(<parents>*P)<metadata>

• d, h, B, I, H, 20s, (20s*N), (20s*P), s*M

The P number would be hidden in the bit field. We need to store 4 possible values here: 0 parents, 1 parent, 2 parents, ø parents information stored. Possible assignement is 00, 01, 10, 11. this let both 0 parent and ø parent info to be 0 module 3.