Indexes extension

This extension does not exist yet. Also, this page is not maintained by the Mercurial developers. These are one person's (PeterHosey's) ideas for an extension.

In Mercurial and other VCSs, you build your next commit in your working copy. Git works differently: it provides a separate staging area called “the index”, and you assemble your next commit explicitly there.

The index has one key advantage: Explicitly assembling your next commit separately from the working copy makes it improbable that you'll contaminate the commit with unrelated changes, or publish it without changes that should be part of it.

It's possible to use interactive commit to achieve the same result, but Mercurial's RecordExtension does not provide a way to review the commit after assembling it in the interactive process. It also does not allow you to build the commit a little at a time; you must strain out the commit all at once at the end of your working period.

Besides, the index and interactive commit are not mutually exclusive. Git has the record command, too, in the form of the -i (interactive) switch to its add command (the command that adds changes to the index). Plus, Git's interactive add command does allow you to review the state of the index—not just when you're done, but at any time.

The problem with Git

Git is infamous for a non-obvious interface. This may or may not be as true as it used to be, but it is still true in some ways. For example, git rm is not the inverse of git add, as it is in some VCSs. This is only natural when you know how Git works (git add adds new files and file changes to the index; git rm adds a file deletion to the index), but it is surprising to the new Git user.

Also, Git does not allow you to build multiple commits in parallel. This is not unique to Git: most VCSs force you to assemble commits one at a time. Mercurial solves this problem with the MqExtension. (Git users may be likely to use Quilt or one of the Git-oriented patch managers.)

But mq has its own share of complexity.

This extension aims to bring these two things together and solve their problems.

The proposed solution

The main goal is to bring Git's concept of the Index into Mercurial.

At the same time, the extension will extend the concept by providing multiple indexes. Similar to Mercurial's named branches system, each index has a name, and there is always a (possibly-empty) index named “default”. Unlike the named branches system, there is no concept of a “current index” (since indexes are not part of history, like commits with their branch names are); if you want to use an index other than default, you must explicitly name it using the --index option.

Commands:

• index-add (iadd): Add changes to an index.

• index-unadd (iua): Remove changes from an index (like git reset). If you unadd everything from an index, the result is like index-abandon.

• index-remove-file (index-rm, irm): Add file deletions to an index (like git rm).

• index-restore (irest): Apply changes from an index to the working copy.
• index-rename: Change the name of an index.
• index-transfer (ixf): Move changes from one index to another. (Open question: Is there a point to allowing the user to copy changes? Maybe for branches.) When moving all changes, the result is like index-rename.
• index-commit (ici): Commit the contents of an index.
• index-abandon: Forget the existence of an index without committing it. When used on the default index, drops the contents, but does not forget the default index (which always exists).
• index-list (ils, indexes): List all indexes.
• index-property (iprop): Get or set a property of an index.
• index-import (iload): Create a new index from an hg patch, git-style diff, or unified diff file.
• index-export (idump): Save the contents of an index to an hg patch, git-style diff, or unified diff file.

You create a new index by adding changes to it for the first time. (The extension should perhaps have some sort of Levenshtein-distance check to prevent accidentally adding to the wrong index, esp. because of typos.) Committing the contents of an index will empty the index. If it is not the default index, it will also delete that index.

The index-add, index-unadd, and index-restore commands accept an --interactive (-i) option, which tells them to enter an interactive editor, like the one git add -i provides.

One intriguing idea is an option to index-restore to apply the inverse of a change. For example, if you add a line to a file, then add that change to the index, index-restore --inverse would remove the line.

The extension will not attempt to defeat built-in Mercurial commands. In particular, it will not prevent the commit command from working normally; you will be able to, at any time, make an end run around the index. I have not yet decided whether directly committing should drop the contents of affected indexes (with due warnings, of course) or not; if not, each index would simply always be relative to the working copy parent. The latter solution presents other challenges, as committing or upping could render indexed changes obsolete, or even introduce conflicts.

The index-property command is for one idea I had: the ability to set a permanent parent revision for an index, rather than have the index use the same parent revision as the working copy. This property would default to “.”, which is the WC's parent, but you could set it to an explicit revision. You could, of course, set it back to “.”. You could use this feature to bind different indexes to different branches.

index-property index #List properties with their values
index-property index foo #Get property foo
index-property index foo bar #Set property foo to value bar

Implementation details (EXTREMELY SUBJECT TO CHANGE)

.hg
        indexes
                default
                        result
                                versioned-file.txt
                                some-directory
                                        versioned-file-2.txt
                        diff
                                versioned-file.txt
                                some-directory
                                        versioned-file-2.txt
                        file-removals
                                some-directory
                                        versioned-file-3.txt

The results directory contains the result of applying the diff to the working copy's (or the index's) parent revision.

If necessary, the index-commit command will work the same way as the RecordExtension's record command: by moving (or hard-linking/copying) the result files into the working copy, then running the regular commit command.

Here are the effects of different commands:

index-add

Raise an error if any named file is in the file-removals directory.

Add the changes/files to the result. When not working interactively, do this by copying any named files (recursively copying any named directories). When working interactively, build each result file a little at a time, adding the user's choice of each differing hunk as the user decides upon it.

The extension probably will not generate the diff at this point.

index-remove-file

Raise an error if any named file is in the result or diff subdirectory.

Touch the file's relative path in the file-removals sub directory. (No need to copy the contents.)

index-unadd

Remove the file from the result, diff, and file-removals subdirectories.

If all three subdirectories are now empty, and the index is not the default index, delete the index.

update

If the index's parent is not bound to the working copy parent (it is explicitly set to a specific revision), do nothing. Otherwise:

Generate the diff, if necessary, as the diff between the result and the old working copy parent. Generate the result by applying the diff to the new working copy parent. Then throw away the diff, since it applies to the old parent.

commit (the built-in one)

Same as update.

rollback

If the previous commit was direct (didn't come from an index), works the same way as commit/update, simply updating the parent of the index.

If the previous commit was born from an index, re-creates the index from that commit and its parent.

Possible failure point: What if the user creates a new index with the same name after rolling back?

index-commit

This is assuming that there's not a better way to do this. This is the behavior of the RecordExtension, and is frustrating if you switch to your editor while the commit message editor is hanging open.

Move the affected files in the working copy to a safe place. (The extension should always restore them from that place whenever Mercurial is run.) Copy the result files into their places in the working copy. Run the built-in commit command.

After committing, delete the result and diff files for the affected files from the index. Delete any empty directories until there are none.

If the index's result and diff subdirectories are now empty, and the index is not the default index, delete the index.

index-abandon

Deletes the index from the indexes directory.

This goes for the default index as well, even though the interface presents that index as always existing. Like any index, attempting to add changes to the default index will create it implicitly, so this little white lie is not as false as it first appears: The index does exist, just not yet on the file-system.

Other commands

Many of these commands are hard to specify because the user can run them in an interactive mode, and it would be easier to implement them on a file-by-file basis than a hunk-by-hunk basis.

The name

The worst UI problem with Git's index is the name.

The concept is a staging area where you construct your next commit. What of that says “index” to you?

Therefore, the concept that this extension adds to Mercurial needs a better name.

The correct name:

• is reasonably short
• clearly describes the concept (“index” fails here)
• can be abbreviated to one or two characters without making a mash of any abbreviated commands (“staging area” fails here, as “irm” becomes “sarm”)

Suggestions welcome.

Ideas so far

• “Schedules extension” (PeterHosey)

  • Reasonably short (one word)
  • Describes the concept (you are scheduling changes for a future commit)
  • Indexes → Commit schedules
  • Abbreviation example: index-remove-file/irm → schedule-remove-file/schrm

• "Stage extension" (JayAdkisson)

  • also reasonably short
  • is actually what you're doing (git docs refer to the index as a "staging area")

  • potentially awesome commands like hg stage, hg unstage, etc.