Note:
This page is primarily intended for developers of Mercurial.
Generic Templating Plan
Adding advanced templating support to commands beyond log
Contents
1. Purpose
Ideally, we should be able to generate customizable output for all output-oriented commands. This will give users fine-grained control for numerous tasks without adding lots of new command line switches. Additionally, we should be able to generate output in common formats such as JSON, XML, and pickle for convenient parsing and automation without any additional code.
2. Interface
Here's what a simple use of templates might look like for a 'showfiles' command:
# create a templater from the command options fm = ui.formatter('showfiles', opts) wctx = repo[None] for f in wctx: # start a new template item fm.startitem() # pass data that's otherwise not shown to the templater fm.data(size=wctx[f].size()) # show some extra data with -v fm.condwrite(ui.verbose, 'flags', '%d %1s ', wctx[f].flags()) # show the path fm.write('path', '%s\n', f) # shut down the templater fm.end()
Internally, template data is modeled as a list of dictionaries, which is well-matched to most of our output. Here, we construct a list of (size, flags, path) elements.
3. Specifying a format
There are several types of format we might want to specify:
- built-in (json, xml, pickle...)
- style we ship (compact, svn...)
- path to a style (/home/bob/lib/myannotate)
- inline template spec ("{path}\t{size}\n")
- path to a file containing a template spec (not a style)
We've added a single new command-line option -T/--template that gives us convenient access to all of these modes:
- -T xml
- -T compact
- -T /some/path
- -T '{author}\n'
3.1. Per-command styles, legacy styles, and verbosity levels
The existing templating system for log doesn't envision support for more than just the log-like commands, but we'll need to continue to support template styles built for this scheme.
We'd also like to have single-file template maps that handle a variety of commands. This might be done by having one map key per command. Dealing with verbosity might be done either with template conditionals or with '.verbose' key suffixes.
4. JSON, XML, and encoding troubles
Given our simple schema, it's a relatively easy matter to construct outputs in standard marshalling formats. One trick here, however, is dealing with non-ASCII data. Given things like filenames may not be in UTF-8 and we often don't even know their encoding (see EncodingStrategy), we can't simply convert to Unicode. Instead, we'll use "UTF-8b", a scheme that can round-trip arbitrary binary data through UTF-8. This is used in various places which have encountered similar problems, for instance Python 3 uses this scheme for dealing with Unix filenames.
5. Steps
add a formatter object to track formatting state
add pass-through backend
add debugging backend
add UTF-8b encoder
add JSON backend
add pickle backend
add -T option handler
add template backend
unify variable names and structures (see below)
document keywords
fix nested use of changesetdisplayer in formatter (see journal.py)
- add XML backend (unlikely?)
- sanity check output (see below) (can be fixed later)
6. Sanity check output
A quick scan turns up the following problems:
The first/last output is often not according to -T (e.g. -Tjson):
hg incoming -- comparing with ...
hg resolve -- merging ... and ... to ...
- Command progress isn't wrapped:
hg rebase -- rebasing x:xxxxxxx ...
- Subprocess output isn't wrapped:
hg resolve --tool
- hints aren't formated:
(no more unresolved files)
continue: ...
Non-formatter stuff:
The first/last output is often not according to -T (e.g. -Tjson):
hg outgoing -- largefiles adds stuff largefiles to upload (...):
The graph interferes in hg log -G, the output isn't JSON -- perhaps we need to expose the marker color and parents or something...)
- There's no warning if you use -T / --template more than once
Consolidate output for the wdir() revision: null vs 0x7fffffff vs {p1rev}+
- "amend_source" + "histedit_source" + "intermediate-source"? + "rebase_source" + "source"
See https://www.irccloud.com/pastebin/TTwtyP5i for an example of this mess. My plan is "source" + "producer" (or something like that) which would result in "source: {hash}" + "producer: graft" or "source: {hash}" + "producer: amend" ...
7. Dictionary
Note that this isn't an official list of what we want, it's merely things that are currently present (some of which should be changed).
- Proposal for unification:
rename {abspath}, {file}, and {name} of file copies to {path} (leaving {file}/{name} as aliases)
remove (relative) {path} in favor of {path|relpath}
alias {user} to {author} in log templates
change {parents} of hg identity to a plain list of nodes
rename {newhashes}/{oldhashes} to {newnodes}/{oldnodes} respectively
remove {statemsg}, {conflictmsg}, and {helpmsg} from status, which are structurally odd
remove {repolookuperror} from journal, which looks wrong
rename {line_number} to {line} or {lineno}
rename {lines} to {texts} in annotate (the data structure is different than grep's, but they are similar in that they contain chunks of file contents)
rename {status} variants
unify {dirty} in identify and {change} in grep they are slightly different. leave them as they are.
- TBD:
- sparse and fastannotate will need some churn, but they are experimental so we can revisit them later
inspect new keywords added to absorb (dcda50856843)
Changeset-oriented commands:
topic\keyword |
bookmarks |
branch |
date |
desc |
node |
parents |
rev |
tags |
user |
(others) |
export |
|
branch |
date |
desc |
node |
parents |
|
|
user |
diff |
identity |
bookmarks |
branch |
|
|
node |
parents |
|
tags |
|
dirty, id |
File history commands:
topic\keyword |
(abs)path |
date |
node |
rev |
user |
(data) |
(others) |
annotate |
path |
date |
node |
rev |
user |
lines{line} |
lineno |
grep |
path |
|
node |
rev |
|
texts{text, matched} |
change, lineno |
{|splitlines} |
|
|
|
|
|
line |
|
Directory-oriented commands:
topic\keyword |
(abs)path |
status |
(data) |
(others) |
cat |
path |
|
data |
|
files |
path |
|
|
flags, size |
kwfiles |
|
|
|
kwstatus |
manifest |
path |
|
|
|
resolve |
path |
|
|
mergestatus |
status |
path |
status |
|
copy(source) |
{file_copies} |
path, name |
|
|
source |
{files} |
path, file |
|
|
|
Namespace commands:
topic\keyword |
(name) |
(active) |
node |
rev |
(others) |
bookmarks |
bookmark |
active |
node |
rev |
|
branches |
branch |
active |
node |
rev |
closed, current |
showbookmarks |
bookmark |
active |
node |
|
longestbookmarklen, nodelen |
tags |
tag |
|
node |
rev |
type |
{bookmarks} |
bookmark |
active |
|
|
current |
{branches} |
branch |
|
|
|
|
{tags} |
tag |
|
|
|
|
Miscellaneous commands:
topic\keyword |
(others) |
config |
source, name, value |
histedit |
nodechanges{oldnode, newnodes} |
journal |
command, date, newnodes, oldnodes, user |
narrow |
pat, (include/exclude)status |
paths |
name, pushurl, url |
rebase |
nodechanges{oldnode, newnodes} |
sparse |
profiles_added, include_rules_added, exclude_rules_added, files_added, files_dropped, files_conflicting |
version |
ver, extensions{name, bundled, ver} |
8. Status
As of 4.6, Mercurial can generate manual templates in addition to JSON and pickle formats for:
- annotate
- bookmarks
- branches
- cat
- config
- debugdeltachain / debugextensions / debuginstall / debugobsolete
- export
- files
- grep
- identify
- log / parents / tip / incoming / outgoing
- manifest
- paths
- status
- tags
- resolve
- summary
- version
$ hg log -r3.1 -Tjson [ { "rev": 23089, "node": "3178e49892020336491cdc6945885c4de26ffa8b", "branch": "stable", "phase": "public", "user": "Pierre-Yves David <pierre-yves.david@fb.com>", "date": [1406923295, 25200], "desc": "status: do not reverse deleted and unknown\n\nWhen reversing a status, trading \"added\" and \"removed\" make sense.\nReversing \"deleted\" and \"unknown\" does not. We stop doing it.\n\nThe reversing is documented in place for the poor soul not even able to remember\nthe index of all status elements by heart.", "bookmarks": [], "tags": ["3.1"], "parents": ["8864528874f77272742551223b8265ff4d125534"] } ]