<> <> == Patch handling unification IRC logs and ideas == This is a wiki page to try to summarize ongoing IRC chatlogs to find a way to unify patch handling extensions and tools (mainly mq, attic, shelve and record). <> === Attempt at capturing the situation today === Today, we have a number of overlapping extensions for managing patches on top of Mercurial repositories: mq, shelve, attic, and pbranch. It would be desirable to prune this list, optimally down to just one. However, we likely don't want to lose any of the main functionality offered by the extensions today: ==== mq ==== Goals: * UI close to quilt (http://savannah.nongnu.org/projects/quilt/) * Maintain multiple, usually related patches. * Typically for later inclusion upstream. * Or for different local configurations (guards). * Import patches from downstream developers. Strong points: * Patches, while applied, are normal revisions and can be emailed etc. as such. * Can go back and forth between patches. * Can reorder patches, as long as they don't interdepend. * Can fold multiple consecutive patches into one. * Has guards to conveniently enable/disable patches. * Can easily promote patches to regular changesets. * Can convert regular changesets into malleable patches. * Can be used directly in your normal work repo. * Directly supported on bitbucket.org. * Patches are stored as files, so they can be viewed and edited as such. * `hg strip` to strip unwanted history (dangerous, too). * Is an established extension shipped with Mercurial. * Is documented in the Mercurial book. So-so points: * Can collaborate on patches by versioning patch files. Weak points: * Collaboration is not straightforward (merging patches - yuck). * Base revision is not clearly defined. * Rebase is complicated (or used to be?). * Updating patches is destructive, so it's easy to shoot yourself in your foot. Especially when distributing pending changes to their proper place. * Updating a patch is a special operation. So needs special tool support (qrecord, qct). * Actual intended patch dependencies are not explicit. Always assumes incremental patches. Ideas to fix it * records the parent/base revision of a patch. * qsplit Splitting patch (or/and interactive options for qrefresh). * integrated interdiff (diff of diff stool) for patches repository versioning. ==== pbranch ==== Don't call it p(atch)branch call it f(eature)branch Goals: * Maintain multiple, usually related patches. * Typically for later inclusion upstream. * Be better than mq for * collaboration and long-term patch maintenance, * traceability of changes to patches, and * distribution of changes to patches. Strong points: * Can go back and forth between patches. * Can fold multiple consecutive patches into one. * Maintains an explicit DAG of patch dependencies. * Each patch has a clearly recorded base it officially applies to. So you always know which patches are already rebased, and which are not. * Rebasing on upstream and on changes in base patches are normal Mercurial merges. * Can collaborate on patches using normal Mercurial commands. * Is non-destructive. * Updating a patch is a normal commit. Tools like record, crecord, TortoiseHg etc. can be used. * Can email patches directly. * Has extensive documentation (Mercurial book style). * May reuse merge conflict resolution (could be included in resolved). So-so points: * Can export patches as mq patch series. * Needs sound understanding of multiple branches/heads and merge flow. Weak points: * Leads to intractable history in the pbranch clone (all these automated merges). * "Applied" patches are not regular changesets (patches are slightly filtered diffs between branches). * Patches will likely need to be linearized before submission to avoid potential fuzz when committers try to apply them. Currently, such linearization is permanent. * Reordering patches can be non-trivial (backout or rebase needed sometimes). Better to start out with proper dependencies in the first place. * Must be used in a throw-away clone of the main repo, unless you intend to eventually strip the patch branches. * Not included with Mercurial. Tests and documentation currently not according to Mercurial's standard. Ideas to fix it * Adds a pfinish command to handy create the final changeset (using lbranch or garbage collected bookmark). * Prevents pbranch managed branches to be pulled/pushed. * Allows cherrypicking a single patch(branch). ==== shelve ==== Goals: * Put aside a single piece of unfinished work. Strong points: * Simple to use. * Some tool integration. * Can be used directly in your normal work repo. Weak points: * Hunk selection always interactive. * Not included with Mercurial. * It's a subset of attic ==== attic ==== Goals: * Maintain multiple unrelated, unfinished strands of work. * Allow passing unfinished work on to colleagues. * Easily and quickly store unfinished work away so you can come back to it later. * Import singular, unrelated patches from downstream developers. Strong points: * Easy to use. * Can untangle multiple unrelated strands (`hg shelve --interactive`). * Can be used directly in your normal work repo. * Patches are stored as files, so they can be viewed and edited as such. * When working on a patch, it is right in the working copy (no refreshing changesets and such). * Works well alongside mq/pbranch. * Is non-destructive (doesn't touch history at all). * Finishing a patch is a normal commit. * Stronger unshelve as the base revision of a shelve is stored and used during the unshelve process. So-so points: * Can move patches back and forth to mq. * Can share .hg/attic dir as a repo on its own, thus sharing patches Weak points: * Not included with Mercurial. * Can get rejects when picking up a patch when base repo has evolved. * Only deals with one patch at a time. ==== lbranch ==== Goals: * Get multiple clone feature with a single working directory frontend. Strong points: * Hidden commit Weak points: * Can not share branches So-so points: * XXX Ideas to fix it: * Integrated collapse stuff. ==== bundle overlay ==== Goals: * looks like lbranch but without actually cloning the repository. Strong points: * Hidden commit. Weak points: * XXX So-so points: * Nested overlay. Ideas to fix it: * XXX === List of use-cases === To aid in making a good decision about the way that the functionality of these tools can be merged, several use-cases are listed here (please contribute new ones if you have one in mind that is not yet listed): * Barack is working on a single-branch repository, and realizes that he wants to try to experiment with coding feature A, without sharing any of this work yet. He signals his intent to hg, and proceeds to work on feature A, making several commits. After some time, he realizes that he won't get far until feature B is first implemented. He issues a command that starts a new branch parallel to the 'feature A' branch, and makes a few commits. When feature B is finished, Barack rebases the 'A' branch onto the tip of the 'B' branch, and continues his work on feature A. When he's happy with feature A, he cleans up the history of the B+A branch, and either makes it public, or merges it into a public branch. * Jan has a repository on his machine with various non-public feature branches in it, and wants to locally (i.e. on his machine) clone the repository so that the clone contains all of the non-public branches. He also wants to make another clone which does not contain any non-public branches. * Sarah has been working on several branches, both public and non-public, and decides to strip off a portion of one of the non-public branches. She continues working on this truncated branch, and then realizes that she stripped away some work which she really needs. She runs an hg command which launches a history browser, allowing her to see all the work she's done for the last 30 days, on both public and non-public (permanent and non-permanent) branches. She chooses a changeset, and a diff is computed against the working directory. She is given the option to apply any of the hunks/lines in the diff to her working directory. * Eduardo has a repository he uses on his work computer with various public and non-public branches. He makes this repository available via hgweb, and restricts the access so that only he and his friends Renaldo and Maria have push/pull access to the repository. Renaldo is only given access to pull/push public branches, whereas Eduardo and Maria are (via an hgrc setting) given the option to add a flag when pushing/pulling so that they can push/pull both public and private branches. When Eduardo or Maria eliminate/restructure a private branch, and push with the "both public and private" flag, the private branch on the remote repository is also eliminated/restructured. * Marty is using an OSS tool and finds a bug. He goes on the tool mailing list and learns that the list community submits patches inline in emails. After some encouragement, Marty grabs the source and writes a patch. After posting the patch somebody suggests that he split the patch up. So, Marty resubmits a new set of patches. At this point another person suggests a better way to solve Marty's problem and asks Marty to share his repository so that they can collaborate on the patches. A short while later the two submit a much improved patchset and an upstream developer says he is applying the patches. * Jim is a full time developer for an application with an active mailing list based developer community. Somebody on the list found a bug and has written a patch for it, but it isn't up to the standards of the current codebase yet. Jim replies with a suggestion for improving the patch and says that in its current form, the patch will not be accepted. At the same time Jim runs his mbox-patch-importer and declines the new awaiting patch. Shortly later a better patch set comes along and Jim says he will apply these new patches. Around the same time, somebody else provides another patch set that Jim says isn't ready yet. At this time Jim reruns his importer and accepts the approved patches and declines the others. === Tentative proposal === ==== Hidden branches/heads ==== Add a special kind of commits heads/branches that don't get transfered during pulling/pushing/cloning. Let's call it ''''hidden'''' branches. They'd get transfered just when requested explicitly or when converted to normal ''public'' heads/branches. A possible way to mark those heads/branches is using an additional extra field, similar to how closed branches are done. Removing that extra field would "unhide" the head and the branch it tips. An alternative implementation would avoid keeping the 'hidden' property in extra and use instead some tagging for bookmarks. In some way, this kind of branches would be similar to git topic branches, or ''a way to do an hg clone/pull/push -r exclusion''. ==== Applying hidden branches to unify mq/pbranches/attic/shelve ==== This new kind of branches allow for a safe ''playground zone'' without resorting to full separate clones. As they can be used without propagating its changesets to other clones they're suited for history altering changes like {{{strip}}}, {{{rebase}}}, {{{qfold}}} or just in-progress work that's not ready to be published. With that, the {{{mercurial queues}}}, {{{pbranches}}} or {{{attic/shelve}}} stores could live in the main repo instead of outside repos and this would also ease '''rebasing''' or full 3-way '''merging'''. {{{mq}}}, {{{pbranch}}} or {{{attic}}} would be just different ways to interface with evolving changesets. An outline of how mq, pbranches, attic or shelve could be implemented is given: ===== attic ===== attic's shelve could move the working dir changes to a commit in a 'hidden' branch and unshelve would just merge back those changes to the working dir without (optionally) commiting them. shelves could be still named and versioned (bookmarks and/or making it a named branch would do the trick). unshelve --delete would be just a strip of the hidden branch that keeps the patch ===== pbranches ===== pbranches could be also used without the need to clone the main repo to keep tentative patches, as the pbranches could be of the hidden kind. ===== mercurial queues ===== mq could use a hidden branch to store its patches info that gets rooted at the point (changeset) where the queue is needed. When time passes and new changes are added it could be possible to rebase the mq branch to a more recent changeset and use the full rebase machinery. ==== Alternative approach: Overlay repositories ==== Instead of using hidden branches, another possibility would be using overlay repositories, with similar behaviour. ==== Alternative approach: Tasks ==== Use [[http://bitbucket.org/alu/hgtasks/wiki/Home|tasks]] instead of hidden branches. ==== Alternative approach: Plans to make attic a superset of mq ==== The use cases for the existing patch handling extensions could be reduced to two different feature sets / models. One that allows the '''handling and management of patches''' and its conversion into and from commits, and another one that implements a way to '''collaboratively develop patches'''. The first feature set would merge the capabilities of mq, attic and shelve, while the second one would correspond to pbranch. Naturally, the two extensions implementing these feature sets should be highly integrated with each other. This proposal tackles the first part, as of right now attic is already a superset of shelve and I (BillBarry) have plans to extend the functionality of attic to become a superset of mq. ===== General interaction with patches ===== Interaction with patches can be viewed from different angles: * from how they interact with the repo history and the working copy contents: they can be '''working copy changes''' or '''mutable changesets'''. * As '''working copy changes''', patches don't affect the repository history and are just a way to store changes using a name so they can be applied or unapplied later. They're used like that in the existing shelve and attic extensions. * As '''mutable changesets''', the existing changes are solidified as changesets in the repository history, so they can be rebased, merged, pushed or pulled... This is how Mercurial Queues view patches. * from how they are created: they can '''consolidate existing changes''' in the working copy or used to '''stash away changes''' those changes. * Using patches to consolidate changes in the working copy is useful when trying to evolve those changes or when they correspond to a changing base content which needs some adaptation in the form of additional patches, and giving these changes a name is very convenient. This is how Mercurial Queues generate its patches, taking them from existing content to later manage in a convenient way. * Stashing away changes allows a more convenient workflow, avoiding having to mess with unrelated changes, so they can be temporarily put appart to concentrate on a different aspect and later can be reapplied. The existing shelve extension uses patches like this. * patches can be a '''single set of changes''' or '''stacked''', either in series, with a given order, or as unrelated changes. These series allow reordering of patches or partial application, allowing guards. Once patches are created, there's also the need to refresh its changes, apply or unapply them to the working copy or repository history, and delete them. ===== Conceptual user interface ===== Patch creation: * Create named patch and keep related changes to the working copy * Create named patch and stash away related changes to the working copy Patch management: * Apply changes from named patch (or series of patches) to the working copy * Unapply changes from named patch (or series of patches) to the working copy * Apply changes from named patch (or series of patches) to the repo history * Unapply changes from named patch (or series of patches) from the repo history * Set guards for a patch Patch transformation: * Refresh changes belonging to the named patch * Transfer content between patches (rename, fold, split, join, rearrange...) Patch removal: * Delete this patch name or named series of patches Patch status: * Status information (changes, guards,...) * Diff ===== Simplified proposed user interface for attic ===== (none of these commands necessarily have their final names yet) * Patch creation * {{{store}}} - Create named patch and keep related changes in the working copy: {{{store --rev imports existing repo changesets (multi-rev-spec, rebase around merges?) --import imports a patch from elsewhere}}} * {{{shelve}}} - Create named patch and stash away related changes to the working copy {{{shelve --rev imports an existing revision --import imports from elsewhere}}} * Patch editing * {{{unshelve}}} - Apply changes from named patch to the working copy * {{{shelve}}} - Unapply changes from named patch to the working copy (patch creation is just a special case) * {{{refresh}}} - Refresh changes belonging to a named patch that is applied to repo history * {{{push}}} - Apply changes from named patch (or series of patches) to the repo history * {{{pop}}} - Unapply changes from named patch (or series of patches) from the repo history * {{{goto}}} - pop/push to get a different applied patchset in repo history * Transfer content between patches * {{{fold}}} - rename, join patches, delete patches * {{{split}}} - split queues * {{{join}}} - join queues * {{{rearrange}}} - open up the attic rearrange-editor * Status information * {{{diff}}} - get changes that would be used in a following shelve/refresh/store/... command * {{{display}}} - list patches/queues in the attic (knowing the storage model, this would be something like a attic-aware {{{ls .hg/attic/}}} printing a bit of status info) * {{{status}}} - list info about the current queue (something like {{{ls .hg/attic/[queuename]/}}} while printing status info) * {{{finish}}} - Delete this patch name or named series of patches * ''Is this needed?''. It should delete a patch (series of patches) in the attic and would convert mutable changesets to normal ones (without qbase/qtip tags), without changing the working copy state (applied content would remain applied and same for unapplied content, which would disappear). advanced usage: * {{{split --interactive}}} - split a patch * {{{shelve --interactive}}} - shelve part of a patch (leave the rest applied) * {{{guard}}} - adds a guard to a patch (meaning afterwords the patch will be skipped over when pushing/popping/..., unless you have the guard selected) * {{{select}}} - selects a guard ===== Storage data model for attic ===== attic stores shelves as patch files in a private directory, and queues are represented as subdirectories with a series file and patches in it. Additional status information, as current patch and stack, applied patches or current guards would be stored in plain text files, preferably starting with a period (conventions dictating not to start patch names with a . are already in place). ===== attic command set ===== My current thoughts for the additions to attic with respect to mq are: {{{ mq: strip strip a revision and all its descendants from the repository attic: probably will not include (this will be a separate extension you need to turn on by itself) mq: qclone clone main and patch repository at same time attic: might be considered for ease of users using a host like bitbucket.org mq: qcommit commit changes in the queue repository qinit init a new queue repository qsave save current queue state qrestore restore the queue state saved by a rev attic: will not be included by default mq: qfinish move applied patches into repository history attic: commit (commit will commit all applied patches followed by the current working directory) mq: qfold fold the named patches into the current patch qdelete remove patches from queue qrename rename a patch attic: fold fold all named patches into stdout --output name fold to output file instead (implies --remove) --remove erase named patches --applied fold all applied patches and the current working copy --limit # fold # topmost applied patches and the current working copy commit ops (-U, -D, -m, -e, -l, ...) examples: fold 3 patches into one: hg fold -o finalpatch queue1/patcha queue1/patchb someshelf delete a patch (output to /dev/null unnecessary, but would eliminate console output): hg fold --remove queue1/patcha >/dev/null rename a patch: hg fold -o queue1/patchb queue1/patcha mq: qimport import a patch qnew create a new patch qpop pop the current patch off the stack qpush push the next patch onto the stack attic: will mostly stay the same (name changes maybe?) mq: qgoto push or pop patches until named patch is at top of stack attic: qgoto will now be able to switch between queues as well: hg qgoto mypatch1 - stay in the current queue, goto mypatch1 hg qgoto otherqueue/ - pop all patches in the current queue and apply patches in the other queue until you hit the one you were on before in that queue hg qgoto otherqueue/somepatch - pop all patches in the current queue and go to 'somepatch' in the other queue mq: qguard set or print guards for a patch qselect set or print guarded patches to push attic: haven't thought about these yet mq: qdiff diff of the current patch and subsequent modifications qrefresh update the current patch attic: these will probably stay as shortcuts, both can be translated into fold: hg qdiff == hg fold -l 1 hg qrefresh == hg fold -l 1 -o tmp && hg qpop && mv [tmp] [current patch] && hg qpush mq: qprev print the name of the previous patch qapplied print the patches already applied qnext print the name of the next patch qseries print the entire series file qtop print the name of the current patch qunapplied print the patches not yet applied qheader Print the header of the topmost or specified patch attic: qstatus print the status of the current patch queue --header [name] print the header of the topmost or specified patch --verbose print all patches in the current queue (including patches not in series) --quiet print prev, current, next patches --queue name use queue instead of current patch queue So, the initial commands for attic-queues will be the following to hit mq functionality: fold fold all given named patches into stdout qimport import a patch qgoto push or pop patches until named patch is at top of stack qnew create a new patch qpop pop the current patch off the stack qpush push the next patch onto the stack qguard set or print guards for a patch qselect set or print guarded patches to push qdiff diff of the current patch and subsequent modifications qrefresh update the current patch qstatus print the status of the current patch queue Additionally: split split a named patch into two patches interactively rearrange bring up an editor where you can swap lines to reorder queues, split them and pull patches out qsplit split a queue into two along some given arrangement qjoin append one queue to another There will also be some special names: 'hg qgoto somethingspecial/floor' - will goto the floor of the somethingspecial queue (change the current active queue to 'somethingspecial' and apply no patches) 'hg qgoto somethingspecial/ciel' - will goto the somethingspecial queue and apply all patches (according to guards) 'hg qfold somethingspecial/*' - will fold the entire somethingspecial queue and print the new patch to stdout 'hg qgoto queue/floor' - will go to the floor of the default queue (queue will be the name used if none is provided) 'hg qgoto someshelf/' - will create a queue from a shelf and apply it: move .hg/attic/someshelf to .hg/attic/someshelf/someshelf create .hg/attic/someshelf/series 'hg qgoto nonexistingqueue/' - will create a new queue with the provided name some example commands: 'hg qpush --shelf shelfname' - will push the shelf onto the queue as the topmost applied patch (will move the shelf) 'hg qpop --shelf' - will pop the topmost applied patch into a shelf - will turn a queue into a shelf if there is only one patch 'hg qnew --shelf shelfname' - will turn shelfname into a queue (just like 'hg qgoto shelfname/') 'hg qnew --queue newqueue patchname' - will create a new patch in a new queue (newqueue/patchname will be the new patch) 'hg qjoin queue1 queue2' - will move queue1/* into queue2 and append queue1/series to the end of queue2/series 'hg qsplit --guard "+stable" newqueue' - will split the current queue into: current containing all patches that would be applied if that guard was set newqueue containing all patches that would not be applied if that guard was set 'hg qsplit newqueue' - current contains all patches currently applied, newqueue contains the rest 'hg qsplit --patches p1 p2 p3 newqueue' - current contains all but p1, p2, p3; newqueue contains p1, p2, p3 in that order }}} === IRC logs === ==== Edited #mercurial IRC Log 28-01-2009 ==== {{{ * mpm would prefer if there were one non-buggy, easy-to-use union of mq/pbranch/record/shelve/attic. after_fallout: I have the new status and default patch working too mpm: I don't think that will quite happen; mq and attic are entirely different directions and not very similar when you actually use them; they are actually decent compliments of each other shelve/record/attic maybe could be one pbranch appears to be a superset of mq, possibly of attic as well. If there's some reason why we need to have more than one way to handle "storing work as patches", then I'm open to it. But not thrilled about it. mpm: I guess we at least need to keep a "quilt-like" mode Yes, but there's no reason that we can generalize it to a DAG or whatever pbranch is doing. Or teach it about chunk granularity like record. yeah it seems like attic and pbranch could be combined I've not really looked closely at any of them aside from mq, that's just my view from a distance. attic is one-level pbranch with nice interactive mode I think attic is quite a bit simpler than pbranch too it isn't meant for long term patch management I appreciate that, however.. big difference: pbranch work can be pulled/pushed around It kind of sucks to have one tool for beginners and a second incompatible tool for experts. (And to fix bugs in them both) I agree, I don't think we will have that though No? What happens when a user that starts with attic outgrows it? it sure would be nice to have just one extension that does everything, with one or two commands for beginners and the others for advanced usage if the reason for having a patch around changes from "I am working on it but it is gonna sit around here for a bit while I do something else" to "I have to manage it for an extended period of time" then it is there for a different reason if attic could become pbranch when working in "versioned" patches mode... yeah something like that tonfa: especially now that the three extensions have borrows a lot of code from each other a considerable amount of the shared work can be joined together attic uses the record code straight out of record after_fallout: yeah, I'know its not that bad -- it just suggest that things should be merged like mpm says or that there is some refactoring to do for example see record.py line 68 class header(object): """patch header XXX shoudn't we move this to mercurial/patch.py ? """ in order to support collaboration like pbranch does, the extension would need some way to extend the wire protocol ideally we should have a hg import --interactive when that happens it means the record stuff is sufficiently refactored :) }}} ==== Edited IRC log, conversation between parren and pachi ==== {{{ after_fallout is working on attic. I just did some minor contributions, but like it very much I can imagine. It sounds attractive. See, I think where attic/shelve shine is for quick juggling of several strands of work. though it looks like having something as easy to work with as attic, but which gets into pbranch when it gets versioned looks like an ideal solution yes So you want something like a pbranch import from attic? but, from reading the pbranch docs I thought that the only "problem" with it is that it's 'in-history' And that is exactly why pbranch is so much more heavyweight than both mq and attic and shelve. Why are we discussing this in private? one idea (probably stupid, but that's why I'm asking more competent people like you) is if it would be feasible to tag branches that are used in pbranch like the closed branches using an extra tag that avoids to push/pull them unless tagged to work otherwise Well, pulling/pushing these is one of the reasons I created pbranch. For when you want to collaborate on patches. that would get us some sort of "local-only" branches that don't propagate and one could strip them easily so, if one wants to share them, one just untags the branch and they work as normal but they could be kept as a local only thing meanwhile Hmm. so shelve/unshelve could work like a pbranch without worry of history polution as it could be possible to reset that branch history (folding) conceptually it's like having a full clone that you can publish when you want Are the new closed branches already exempt from push/pull? I doubt it somehow. no I had a look to what pull/push does, and it uses findincoming to find the tips that lack in local that are in remote, so excluding tagged heads could work (or even walking a branch and having a cache) Yep, I can imagine. does it make sense? Could work. Feels somewhat like git branches. but they're like the inverse concept, you don't add remotes but tag "local-only" branches, as, by default, hg shares all of the branches Yes, of course. I think after_fallout is willing to work on something better, and even on record/mq refactoring, but your pbranches look brilliant too I know you're really busy, but do you think a common plan could be layed out? What this does not yet address is mq's guards. pbranch cannot do them. And mq is better at juggling patches freely. Swapping the order of two patches with pbranch is, for example, rather hard with pbranch (which is why you should be starting them as parallel patches in the first place). but it looks more like an UI problem, doesn't it (the patchgraph thing IIRC)? or is it a problem of how to get the desired order manipulating the DAG? No. If you have A->B->C, then C contains everything in B. Now try swapping to A->C->B. You have to first rebase C on A, then B on C. And preferably in a non-destructive way. I see And guards are even harder. Every change in guard setup would effectively cause rebases recorded in history. Not likely what people want. well, maybe mq still has its uses, but for special cases like those. At least a versioned version of attic/shelve would be good, even without stacked patches maybe extracting those bits makes mq cleaner and stacks are implemented as an addition to the stacked patches something like pbranch, but with an additional series file ? pbranch has the .hg/pgraph file, which is like .hg/patches/series. so the problem is how the new patches are created, using overlays instead of disjoint patches? I like the idea of making attic more akin to starting a series of parallel pbranch patches. You've lost me there. What do you mean by "overlay"? yes, that's the idea (mpm suggested that pbranch is like a superset of mq) Like, yes. But as I said, mq still has its strong points. as I understand it, the problem with pbranch is that you start a patch an you later don't really want to create a new ppatch on top of it if they need to be reordered later ala mq s/an/and/ Right. You should start it independently. Which is a kind of bad premise because you're prone to forget. It's like doing fixes properly at the root cause so you don't have to backport, only forward-merge. Everybody knows this, but it just does not always happen this way. but if mq manipulations had a separate pbranch that gets done and undone transparently but take each of the pieces of the stack from isolated pbranches, then it could perhaps work s/done and undone/applied and unapplied/ Only as long as all the patches are truly independent. Usually, some are, some are not. the problem now (as I see it) is that creating new branches that get destroyed are a bad idea because they get shared and then are there forever but if they were not independent wouldn't we gave the same problems than with mq? Yes. When you work with pbranch today, you have to work in a separate clone which you plan to throw away eventually (when the patches have been merged upstream). *have Yes, with mq you then get failed applies. if that clone were an 'in-repo but only local' branch, it wouldn't be needed Correct. So I kind of like this idea. the point is... if we had throw away branches that don't get published... would it allow to do things differently? Yes. Maybe we could even say that while you're only using this by yourself, we don't have to use hg branches at all and use bookmarks instead. And flag some bookmarks as non-push/pullable. I had the intutition that it would be so, as hg can do the same as git, but we need clones for strategically hiding branches and that has a cost when trying to work from a given repo, as some operations are problematic without resorting to an external clone yes all existing syntactic sugar could be applied to it When you want to share, you migrate them to proper branches. This could be a pbranch command, committing all pbranch bookmarks to pbranch branches. yes, that's it Using bookmarks you also won't run into trouble with colliding branch names (say remote has a new branch that collides with one of your private attic branches). But I think we should define a branch namespace for pbranch et al anyway. Both bookmarks and pbranch still need the wire protocol change so they can transmit their metadata (bookmark definitions and .hg/pgraph). indeed, it would have it's own space (really, like what mq has) to do the bookkeeping Well, maybe not. In this case it probably suffices if local fs->fs clones copied the stuff. but while that doesn't happen we'd only miss how patches relate to each other, wouldn't we? I mean... we'd have all the patches (if they were shared) but not how they get merged in a series pbranch actually recreates a good idea of the relations from the recorded dependencies. See http://arrenbrecht.ch/mercurial/pbranch/collab.htm#pagetoc__1_2 that would be still very useful, but more similar to a versioned attic I see And now that we have branch closing, it could recreate an even better idea. maybe tonfa knows if that 'local only' or 'locked' branches is feasible. IIRC he worked on findincoming... I've also dug into it for the shallow clones. I think it can work. After all, it really amounts to the same as `hg push -r x -r y -r z` where xyz are the heads of all non-blocked branches, no? yes, and I even thing (from a very superficial understanding) that it could be just removing the 'local' heads So for push it sounds like even an extension could do it, but wrapping `hg push`. For pull it's likely harder, since remote has to do the filtering. but pulling from it should work the same I can imagine that that's where the fun begins (how to get the subgraph) subgraph meaning? pgraph? no, how to 'obliterate' the non shared parts the non-shared branches could be useful for many other things I suppose besides pbranch Sure. But I'd really focus on non-shared heads, not branches. More general, and works with bookmarks. fine I was thinking in the same semanthics as branch "closing" So: attic becomes a bunch of independent pbranch patches, but based on bookmarks instead of named branches. And we block them from getting out of the repo. And we make pbranch work with both bookmarks and named branches, with an option to promote bookmarks to branches for sharing. Is that the plan? but 'hiding' here instead parren that's it :) Maybe this could be tied into bookmarks directly. Flag a bookmark as hidden. well, those would be details that depend on the wire protocol allowing the transfering of some bookkeeping even if a named branch had to be used to have a name for patches while that's not available would do IMHO Actually, attic would become just a bunch of hidden heads, with no pbranch meta-info needed. But you can seamlessly upgrade from a patch in the attic to making it a full-blown pbranch patch. handling that as metadata or inmutable information is equivalent in some way if we can throw away branches having the easiest UI to start from something like attic and go to a full pbranch is key, IMHO Things is, we're talking about hiding these things. So if the things don't get propagated, then we don't need to propagate meta-info about them either, do we? no only if they get shared, that's why how storing the patch names isn't important until they get shared, and probably the bookmark wire extension is needed but the problem of sharing metadata is not a problem of this particular idea Well, as soon as we share, we don't them hide anymore, do we? Which is why I say we teach pbranch to work with both bookmarks (non-shared) and named branches (shared). So once we share, we don't hide, so there is no hiding metadata to share anymore I'd use only bookmarks, as mpm probably wants to solve the bookmarks sharing problem anyway but, that's not important ATM, IMHO at the very least we'd have what pbranch has now ;) Right. And since pbranch already does named branches and needs to be taught to use bookmarks, we lose nothing. And once bookmarks can be shared we can maybe drop the named branch support in pbranch. So this might be the time to rename pbranch. :) How about sending a summary as an RFC to the devel list? With this transcript and excerpts from the one tonfa sent appended? but otherwise I can try. I can edit text better than coding ;) I'll see if I can find some time tonight. Got to go now. Kids and dinner time. I enjoyed this discussion. Thanks. ok, thanks a lot }}} ==== Edited IRC log, conversation between after_fallout and pachi ==== {{{ hi I've been talking to parren, the pbranch extension author about an idea to unify pbranch/attic/shelve, as mpm suggested I had the same thoughts as he does prettymuch with pbranch looking at pbranch we seem to have a couple of mostly minor issues 1. you have to know in advance when you want parallel branches 2. it does things to the history of your repository that get propagated when you push and pull mq doesn't really have those problems 1 and 2 can be solved with the 'local only' branches IMHO I think you can avoid them in the first place well, they can be avoided if some branches can be hidden from sharing that's at the end what mq does and attic, even if unversioned it just hides information from propagating true so you can alter contents without polluting history it just adds an outer repo to do its job, but it could be done in a branch (a specialized branch) to avoid the problems of managing it (qclone, qpush -R and so on) same with versioned attic ahh, bookmarks instead of named branches :) that would solve some of the problems right, in these secondary repos, you don't actually care about the history, only that you have the right version you need the history to be able to direct a merge, but you don't need it as they won't pollute anything, and could even be stripped of folded or anything without disturbing other repos to see what happened at any given time yes pbranch has a sort of pseries in the form of pgraph yeah sorta as I see it it's like an out of band DAG so the relationships between changesets for ordering are done with that instead of using hg itself, which is used to get the real thing, but not to store their relations initially more like a set of dags I think a set of dags and a way to relate them using another metadag that does the stacking well, at least that's what I understand... the way I understood it, you could start multiple pbranches on different nodes on the repository and never have them relating to each other yes, I think it's like that, though they can relate too, but it's not required a set of connected pbranches is a DAG, but the pgraph contains all the sets of pbranches which may or may not be all connected that's the problem parren saw to replicate mq behaviour, but it can be done using a new 'local' branch for stacking essentially, that's what mq does with the main repo it versions I guess use another repo to do the compositing, but we could do that in the same repo using another local branch (that can be stripped and reapplied at will) and still keep patches standalone }}} ---- CategoryOldFeatures