Differences between revisions 50 and 51
⇤ ← Revision 50 as of 2017-01-04 02:57:53 →
Size: 9314
Editor: AntonShestakov
Comment: update mailing list link, some typos
← Revision 51 as of 2017-01-04 18:57:53 → ⇥
Size: 9365
Editor: Erik van Zijst
Comment: Fixed spelling and grammar
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
 * It offers a '''safe''' and '''simple''' way to refine changesets locally and propagate that changes to other repository.
 * It can ''automatically detect'' and ''handles the complex issues'' that can arise from exchanging draft changeset.
 * It makes is even possible for ''multiple developer'' to rewrite the same part of the history in a ''distributed'' way.
 * It fully respect the [[Phases]] concept to unsure user will only rewrite part of the history that are meant to be. Phases have been part of Mercurial since early 2012.		  * It offers a '''safe''' and '''simple''' way to refine changesets locally and propagate those changes to other repositories.
 * It can ''automatically detect'' and ''handle the complex issues'' that can arise from exchanging draft changesets.
 * It makes it even possible for ''multiple developers'' to rewrite the same part of the history in a ''distributed'' way.
 * It fully respects the [[Phases]] concept so unsure users will only rewrite parts of the history that are safe to be changed. Phases have been part of Mercurial since early 2012.
Line 12: Line 12:
You can [[ https://www.mercurial-scm.org/doc/evolution/|read the documentation]] or [[https://air.mozilla.org/changesets-evolution-with-mercurial/|watch the talk at FOSDEM 2013]] to learn more.
([[https://www.logilab.org/120046|Slides]]).		 You can [[https://www.mercurial-scm.org/doc/evolution/|read the documentation]] or [[https://air.mozilla.org/changesets-evolution-with-mercurial/|watch the talk at FOSDEM 2013]] to learn more. ([[https://www.logilab.org/120046|Slides]]).
Line 16: Line 15:
{i} While well on the way, the full implementation of the changeset evolution concept is still in progress. Core Mercurial already supports many of the associated features, but for now they are still '''disabled by default'''. The current implementation has been usable for multiple years already, and some parts of it are used in production in multiple projects and companies (including the Mercurial project itself, Facebook, Google, etc…).
Line 17: Line 17:
{i} While well on the way, the full implementation of the changeset evolution concept is still in progress.
Core Mercurial already support many of the associated feature, but for now they are still '''disabled by default'''.
The current implementation have been usable for multiple years already, and some part of it are used in production in multiple project and company (including the Mercurial project itself, Facebook, Google, etc…).

However, there is still some area were the current implementation have gaps. This means some use case or performance are not handled as well as they current are without evolution. Mercurial have been around for a long time and is strongly committed to backward compatibility, turning evolution on by default nowaday could regress the experience of some of our current users. The feature will be enabled by default at the point where user who do not use or care about the new feature added by evolution won't be impacted by it.		 However, there are still some areas were the current implementation has gaps. This means some use cases or performance issues are not handled as well as they currently are without evolution. Mercurial has been around for a long time and is strongly committed to backward compatibility, and turning evolution on by default nowadays could regress the experience of some of our current users. The feature will be enabled by default at the point where users who do not use or care about the new features added by evolution won't be impacted by it.
Line 24: Line 20:
Using evolution is safe and no data loss/corruption is to be expected. Once you turn evolution on, all commands from core Mercurial will use it. In addition, one can enable evolution locally and still use an "old" server. You won't "poison" the server (but won't be able to use the new feature with that server).
Line 25: Line 22:
Using evolution is safe and no data loss/corruption is to be expected. Once you turn evolution on, all commands from core Mercurial will use it. In addition, one can enable evolution locally and still use an "old" server, you won't "poison" the server (but won't be able to use the new feature with that server). Testing the concept early is useful for us, core developers. It provides us with data and use cases that we need to iron things out before enabling it by default. This is especially true if you can test evolve in a context that involves client/server/distributed workflows.
Line 27: Line 24:
Testing the concept earlier is useful for us, core developer. It provides us with data and use case that we need to iron out before enabling it by default. This is especially True if you can test evolve in a context that involve client/server/distributed workflow. Here are the recommended steps to try evolution:
Line 29: Line 26:
Here are the recommended step to try evolution:
  1. Subscribe to the [[https://www.mercurial-scm.org/mailman/listinfo/evolve-testers|evolve beta tester]] mailing list,
  1. Install and use the EvolveExtension
  1. Enjoy changeset evolution		  1. Subscribe to the [[https://www.mercurial-scm.org/mailman/listinfo/evolve-testers|evolve beta tester]] mailing list,
 1. Install and use the EvolveExtension
 1. Enjoy changeset evolution
Line 34: Line 30:
The [[EvolveExtension|evolve extensions]] will take care of enabling all the appropriate features in core and add a small layer with the latest state of the new commands and algorithm. The extension is developed by core Mercurial developers and its code collaborate  closely with Mercurial core internal. It leave outside of core to support older versions of Mercurial (extending the tester base) and gain flexibility when experimenting new algorithm. The [[EvolveExtension|evolve extensions]] will take care of enabling all the appropriate features in core and add a small layer with the latest state of the new commands and algorithms. The extension is developed by core Mercurial developers and its code collaborates closely with Mercurial's core internals. It is left outside of core to support older versions of Mercurial (extending the tester base) and gain flexibility when experimenting with new algorithms.
Line 36: Line 32:
As the implementation is still in progress, some command behavior might change. Subscribe to the [[http://selenic.com/mailman/listinfo/evolve-testers|evolve beta tester]] mailing list to make sure you stay aware of new release and the change they might introduce. As the implementation is still in progress, some command behavior might change. Subscribe to the [[http://selenic.com/mailman/listinfo/evolve-testers|evolve beta tester]] mailing list to make sure you stay aware of new releases and the changes they might introduce.
Line 39: Line 35:
Line 43: Line 38:
Line 48: Line 42:
 * `hg histedit`: can perform rewrite operation on some of your changesets (requires the HisteditExtension)  * `hg histedit`: can perform rewrite operations on some of your changesets (requires the HisteditExtension)
Line 50: Line 44:
The experimental EvolveExtension adds more commands, which will eventually moved into core: The experimental EvolveExtension adds more commands, which will eventually be moved into core:
Line 57: Line 51:
All these operations are '''very safe to use''', even for Mercurial rookies.
Mercurial will actively prevent you from rewriting parts of history which are not safe to rewrite. Read about the [[Phases]] concept for details.		 All these operations are '''very safe to use''', even for Mercurial rookies. Mercurial will actively prevent you from rewriting parts of history which are not safe to rewrite. Read about the [[Phases]] concept for details.
Line 65: Line 58:
Obsolescence is tracked using "obsolescence markers", a piece of metadata that tracks which changesets have been made obsolete, potential successors for a given changeset, the moment the changeset was marked as obsolete, and the user who performed the rewriting operation. The markers are stored separately from standard changeset data and can be exchanged without any of the precursor changesets, preventing unnecessary exchange of obsolescence data. Obsolescence is tracked using "obsolescence markers", a piece of metadata that tracks which changesets have been made obsolete, potential successors for a given changeset, the moment the changeset was marked as obsolete, and the user who performed the rewriting operation. The markers are stored separately from standard changeset data and can be exchanged without any of the precursor changesets, preventing unnecessary exchange of obsolete data.
Line 70: Line 63:
Exchanging mutable changesets has inherent ''troubles'' that we must be prepare to deal with. Most people will never run into them but Mercurial is able to detect and solve them automatically. Exchanging mutable changesets has inherent issues that we must be prepare to deal with. Most people will never run into them but Mercurial is able to detect and solve them automatically.
Line 72: Line 65:
There are three kinds of '''troubled changesets''' There are three kinds of '''troubled changesets:'''
Line 74: Line 67:
In some situations you may have non-obsolete changesets descending from obsolete changesets. Such changesets are said to be "unstable".  1. In some situations you may have non-obsolete changesets descending from obsolete changesets. Such changesets are said to be "unstable".
 1. In some other situations you may have successors for changesets which are now [[immutable]]. In such case the obsolescence marker does not apply and the unlucky successors are said to be "bumped".
 1. Finally when multiple changesets claim to be the successors of changesets they are said to be "divergent".
Line 76: Line 71:
In some other situations you may have successors for changesets which are now [[immutable]]. In such case the obsolescence marker does not apply and the unlucky successors are said to be "bumped".

Finally when multiple changesets claim to be the successors of changesets they are said to be "divergent".

When Mercurial detect such ''troubles'' it will warn the user and prevent push by default. You can use the `hg evolve` command to automatically resolve them.		 When Mercurial detect such troubled changesets, it will warn the user and prevent push by default. You can use the `hg evolve` command to automatically resolve them.
Line 85: Line 76:
As of January 2017, the following areas are covered:
Line 86: Line 78:
As of January 2017, the following area are covered:  * All commands know how to create, store and read obsolescence information,
 * obsolete changesets are properly excluded from the user's view and not exchanged between repositories,
 * the obsolescence markers are properly exchanged between repositories, propagating relevant obsolescence information alongside the changesets,
 * hg log and summary will display some information related to evolution,
 * commands like hg update, hg pull and hg rebase can use obsolescence information to make smarter decisions,
 * all evolution issues are properly detected and diagnosed,
 * the `evolve` command can automatically resolve the most common cases of such issues.
Line 88: Line 86:
 * All commands know how to creates, store and read obsolescence information,
 * obsolete changesets are properly excluded the user view and not exchanged between repository,
 * the obsolescence markers are properly exchanged between repository, propagating relevant obsolescence information alongside changesets,
 * hg log and summary will display some information related to evolution,
 * command like ``hg update``, ``hg pull`` and ``hg rebase`` can use obsolescence information to take smarter decision,
 * all evolution troubles are properly detected and diagnosed,
 * the `evolve` command can automatically resolves the most common cases of such troubles.		 And the following areas need improvement: (get on the tester mailing list if you get stuck on one of them)
Line 96: Line 88:
And the following area need improvement:
(Get on the tester mailing list if get stuck into one of them)

 * the most rare/advance case of evolution troubles are not automatically handled yet,
 * we need more advance barrier to prevent the user to shoot himself in the foot in some corner cases
 * raw obsolescence data are available to the user, but we lack good tool to visualise the obsolescence history,
 * bringing an obsolete changesets to life is harder that it should be,
 * support for other advance maintenance operation (like bundling or stripping obsmarkers) is missing,		  * the most rare/advance case of evolution issues are not automatically handled yet,
 * we need more barriers to prevent the user from shooting themselves in the foot in some corner cases
 * raw obsolescence data is available to the user, but we lack good tools to visualise the obsolescence history,
 * bringing an obsolete changeset to life is harder that it should be,
 * support for other advance maintenance operations (like bundling or stripping obsmarkers) is missing,
Line 105: Line 94:
 * we need more scaling work for large/old installation,
 for update and bookmarks.		  * we need more scaling work for large/old installations, for update and bookmarks.
Line 114: Line 102:

* [[http://bz.selenic.com/buglist.cgi?keywords=easy%2C%20&keywords_type=anywords&order=Bug%20Number&resolution=---&query_format=advanced&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=NEED_EXAMPLE&bug_status=IN_PROGRESS&component=evolution&list_id=5014|list of easy bug]]		  * [[http://bz.selenic.com/buglist.cgi?keywords=easy,%20&keywords_type=anywords&order=Bug%20Number&resolution=---&query_format=advanced&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=NEED_EXAMPLE&bug_status=IN_PROGRESS&component=evolution&list_id=5014|list of easy bug]]

Changeset Evolution

Contents

  1. Status
  2. Using Evolution
  3. Overview
    1. Rewriting history
    2. Tracking and sharing rewriting
    3. Automatic detection and resolution arising troubles
  4. Current implementation state
  5. Older materials
  6. Related Developer pages

Changeset Evolution is a set of features to gracefully handle history rework operations.

  • It offers a safe and simple way to refine changesets locally and propagate those changes to other repositories.

  • It can automatically detect and handle the complex issues that can arise from exchanging draft changesets.

  • It makes it even possible for multiple developers to rewrite the same part of the history in a distributed way.

  • It fully respects the Phases concept so unsure users will only rewrite parts of the history that are safe to be changed. Phases have been part of Mercurial since early 2012.

You can read the documentation or watch the talk at FOSDEM 2013 to learn more. (Slides).

1. Status

{i} While well on the way, the full implementation of the changeset evolution concept is still in progress. Core Mercurial already supports many of the associated features, but for now they are still disabled by default. The current implementation has been usable for multiple years already, and some parts of it are used in production in multiple projects and companies (including the Mercurial project itself, Facebook, Google, etc…).

However, there are still some areas were the current implementation has gaps. This means some use cases or performance issues are not handled as well as they currently are without evolution. Mercurial has been around for a long time and is strongly committed to backward compatibility, and turning evolution on by default nowadays could regress the experience of some of our current users. The feature will be enabled by default at the point where users who do not use or care about the new features added by evolution won't be impacted by it.

2. Using Evolution

Using evolution is safe and no data loss/corruption is to be expected. Once you turn evolution on, all commands from core Mercurial will use it. In addition, one can enable evolution locally and still use an "old" server. You won't "poison" the server (but won't be able to use the new feature with that server).

Testing the concept early is useful for us, core developers. It provides us with data and use cases that we need to iron things out before enabling it by default. This is especially true if you can test evolve in a context that involves client/server/distributed workflows.

Here are the recommended steps to try evolution:

  1. Subscribe to the evolve beta tester mailing list,

  2. Install and use the EvolveExtension

  3. Enjoy changeset evolution

The evolve extensions will take care of enabling all the appropriate features in core and add a small layer with the latest state of the new commands and algorithms. The extension is developed by core Mercurial developers and its code collaborates closely with Mercurial's core internals. It is left outside of core to support older versions of Mercurial (extending the tester base) and gain flexibility when experimenting with new algorithms.

As the implementation is still in progress, some command behavior might change. Subscribe to the evolve beta tester mailing list to make sure you stay aware of new releases and the changes they might introduce.

3. Overview

{i} You can also check the evolve documentation.

3.1. Rewriting history

Mercurial offers multiple commands to rewrite history:

  • hg commit --amend: can add more changes into a commit

  • hg rebase: can move changesets around in your graph (requires the RebaseExtension)

  • hg histedit: can perform rewrite operations on some of your changesets (requires the HisteditExtension)

The experimental EvolveExtension adds more commands, which will eventually be moved into core:

  • hg uncommit: can remove changes from a commit and put them back in your working directory

  • hg fold: can squash multiple changesets together as a single new commit

  • hg prune: can remove changesets from your history

  • hg split: can remove changesets from your history

All these operations are very safe to use, even for Mercurial rookies. Mercurial will actively prevent you from rewriting parts of history which are not safe to rewrite. Read about the Phases concept for details.

3.2. Tracking and sharing rewriting

Obsolescence markers make it possible to mark changesets that have been deleted or superseded by a new version of the changeset.

Unlike the previous way of handling such changes (which stripped the old changesets from the repository), obsolescence markers can be propagated between repositories. This allows for a safe and simple way of exchanging mutable history and altering it after the fact. Changeset phases are respected, such that only draft and secret changesets can be altered (see hg phases for details).

Obsolescence is tracked using "obsolescence markers", a piece of metadata that tracks which changesets have been made obsolete, potential successors for a given changeset, the moment the changeset was marked as obsolete, and the user who performed the rewriting operation. The markers are stored separately from standard changeset data and can be exchanged without any of the precursor changesets, preventing unnecessary exchange of obsolete data.

The complete set of obsolescence markers describes a history of changeset modifications that is orthogonal to the repository history of file modifications. This changeset history allows for detection and automatic resolution of edge cases arising from multiple users rewriting the same part of history concurrently.

3.3. Automatic detection and resolution arising troubles

Exchanging mutable changesets has inherent issues that we must be prepare to deal with. Most people will never run into them but Mercurial is able to detect and solve them automatically.

There are three kinds of troubled changesets:

  1. In some situations you may have non-obsolete changesets descending from obsolete changesets. Such changesets are said to be "unstable".

  2. In some other situations you may have successors for changesets which are now immutable. In such case the obsolescence marker does not apply and the unlucky successors are said to be "bumped".

  3. Finally when multiple changesets claim to be the successors of changesets they are said to be "divergent".

When Mercurial detect such troubled changesets, it will warn the user and prevent push by default. You can use the hg evolve command to automatically resolve them.

This command is partially implemented in the EvolveExtension.

4. Current implementation state

As of January 2017, the following areas are covered:

  • All commands know how to create, store and read obsolescence information,
  • obsolete changesets are properly excluded from the user's view and not exchanged between repositories,
  • the obsolescence markers are properly exchanged between repositories, propagating relevant obsolescence information alongside the changesets,
  • hg log and summary will display some information related to evolution,
  • commands like hg update, hg pull and hg rebase can use obsolescence information to make smarter decisions,
  • all evolution issues are properly detected and diagnosed,

  • the evolve command can automatically resolve the most common cases of such issues.

And the following areas need improvement: (get on the tester mailing list if you get stuck on one of them)

  • the most rare/advance case of evolution issues are not automatically handled yet,
  • we need more barriers to prevent the user from shooting themselves in the foot in some corner cases
  • raw obsolescence data is available to the user, but we lack good tools to visualise the obsolescence history,
  • bringing an obsolete changeset to life is harder that it should be,
  • support for other advance maintenance operations (like bundling or stripping obsmarkers) is missing,
  • during exchange, the discovery protocol for markers can be very slow in some case (we have a second iteration to be implemented),
  • we need more scaling work for large/old installations, for update and bookmarks.

5. Older materials

ChangesetEvolution (last edited 2022-08-14 21:06:00 by StephenRasku)