Review Workflows

- enumeration of key concepts related to development process and tooling on top of VCS.

WIP, notes based on discussion at the 3.8 sprint.

Purpose

- what do we mean when we talk about review workflows and what do we hope to get out of it:

• Get (early) feedback to changes - leverage other developer's experience
• Knowledge sharing - inform others about changes
• Catch bugs and issues in changes before landing them in mainline (rare but valuable)
• Let area owners maintain ownership while the area is changed by others
• Enforce and support a formal process to make sure best practice actually is followed - collect and check human and automatic approvals
• Coordinate and automate the actual landing of changes to the main line
• Ensure consistent and possibly clean version control history while landing - prepare for actual use of history with bisect and annotate

Nomenclature

- we identify some general areas that can be observed in most review workflows and we investigate what's in it:

Submit

- how the changes are submitted:

• Push - deep integration with repo hosting
  • Repository - where to push
    • Push to the main repo (on branches?)
    • Push to some shared "try"/"review"/"draft" fork
    • Push to personal (temporary) fork
  • Branches - how to tell incoming changes apart
    • Follows from using separate fork
    • Using temporary branch (named or bookmark/git) (rebase and/or git/bookmark merge will discard it)
    • Using permanent branch (named branch, merged not rebased)
  • Create a "Pull request" on data in the repo (usually misleading name - better named as "Merge request", "Integration request", "Review request", "Change request" ... or just as a "Series" or "Changeset set" that can be RFC, reviewed, approved, grafted, merged, rebased, etc)
  • Diff and annotate info can come from live repo data
  • Iterations can be submitted by pushing additional changes on top or with history modification (possibly using obsolescence to chain iterations)
• Patch submission - review pretty much works on a plain diff
  • Transport:
    • Upload diff to server using something like a web interface
    • Mail - probably patchbomb to mailing list
    • Push gate can make anything look like the repo push model above
    • Deep VCS integration can make anything look like anything
  • Will inherently lean towards rebasing and not using branches
• Series - submitting more than one change at once
  • Dirty history
    • Probably append only history, no history revisionism
    • Only the end result matters - don't look so much at individual changes but take all or nothing
    • Intent of either merging dirty history or collapsing (squashing) history
  • Clean history
    • History has been cleaned and is a product too
    • Each change is self contained and leaves a perfectly fine and releasable state but might depend on other changes before it
    • Changes are ordered by maturity/risk so they can be taken one by one or as far as everybody agrees and the later ones
• Local future of submitted changesets
  • Submitted changes might be considered "published" and permanent and will be merged
  • Submitted changes might be considered "draft" and will not be applied upstream as is - the submitted changeset was temporary and should disappear

Show

- how the changes are shown to reviewers:

• Unified diffs or side-by-side
• Plain text or with syntax highlighting
  • diff/word-diff highlighting, language highlighting
• Semantic information/markup/hyperlinking
• Individual changesets or aggregate diff
• Conflict warnings/merge diffs
• Aggregate diff since ancestor or diff between heads
• Trailing whitespace/code style rules
• Sensible diffs for non-plaintext file types (images, GPG keyrings, etc.)
• Web/e-mail/native client
  • define a standard for review system client plugins?
  • Be able to review within your favorite editor/GUI SCM client?

Comment / communicate

- how reviewers can communicate with author:

• forum style async communication centered around a line of change, a changeset or the whole pull request
  • web click'n'comment
  • mail inline replies
  • local native client comments
• post comments 1 by 1 or batch them (like mail might have several inline replies)
• chat style real time discussion - inline in system or link to external chat and archive
• constructive contribution, show don't tell, make changes instead of asking for them (using existing process of modifying history or adding changesets, thus effectively iterating)

Iterate

- how changes might change during the process:

• show meta diff between iterations
• static or moving target
• extra commits with changes
• history modification
• but how to show comments from old iteration in new iteration where the original lines might have moved, be hard to find, or have gone completely
  • heuristics are tough to make accurate
    • fuzzy match diff context around comment?
  • use some sort of annotation mechanism to track new location?
  • tracking built in to vcs and moved around as source changes?
• new iteration might invalidate reviews / approvals

Approve

- how reviewers can control the propagation of the changes:

• some approval workflow can be specified and be more or less mandatory/enforced
• possibly formalized ownership of touched files, suggested or added/informed automatically
• different kind of reviews and approvals - architectural, ownership, catch bugs, full or partial, FYI/ACK, stylistic review, product management, release management, CI
• may or may not be invalidated by iteration

Land

- actual landing of the changes:

• automatic / "merge button" manual / external manual
• collapse / rebase/linearize / merge
• partially landing a series while continuing developing

Tools and implementations

How different tools and implementations relate to this nomenclature / frame of reference:

Mercurial project

• Submit: mainly patchbombs, automated with push gateway
• Show: patch emails to mailing list, also snarfed by patchwork
• Comment: inline replies on mailing list
• Approve: trusted people collect patches from mailing list, read informal responses, reject bad parts, might tweak, approve good parts by pushing to pre-landing repo
• Land: pre-landed changesets are (currently) re-reviewed, rebased, tweaked and pulled upstream by mpm
• Iterate: rejected parts must be "history modified" and re-sent to the mailing list tagged v(n+1)

Kallithea

• Submit: repo hosting, fork or shared, PR on live diff
• Show: web with unified diff (partly side-by-side), mail listing commit descriptions, notifications on iterations
• Comment: web inline comments, notifications
• Approve: reviewers are assigned manually - no automatic landing and thus no enforcement
• Land: manual offline, shows hint about what to pull and merge
• Iterate: static view until iterating (closes old PR and creates new one with additional changes - no good support for modifications)

Unity Technologies

Tweaked implementation of Kallithea

• Submit: shared repo with named branches
• Approve: automatic recommentation/review of assigned revieweres based on touched files plus release management plus "auto land request", CI status shows up too
• Land: "trunk guardians" are gate keepers and make final meta review, prepare batch merging all possible PRs to verification branch, runs all tests, bisect failures, merge good parts to mainline, bounce PRs causing conflicts or failures
• Iterate: append-only history - not clean, only bisectable on mainline, new iteration pretty much invalidates previous approvals ... but negotiable with guardians

CategoryDeveloper