Generic Templating Plan

Adding advanced templating support to commands beyond log

1. Purpose

Ideally, we should be able to generate customizable output for all output-oriented commands. This will give users fine-grained control for numerous tasks without adding lots of new command line switches. Additionally, we should be able to generate output in common formats such as JSON, XML, and pickle for convenient parsing and automation without any additional code.

2. Interface

Here's what a simple use of templates might look like for a 'showfiles' command:

# create a templater from the command options
fm = ui.formatter('showfiles', opts)
wctx = repo[None]
for f in wctx:
    # start a new template item
    fm.startitem()
    # pass data that's otherwise not shown to the templater
    fm.data(size=wctx[f].size())
    # show some extra data with -v
    fm.condwrite(ui.verbose, 'flags', '%d %1s ', wctx[f].flags())
    # show the path
    fm.write('path', '%s\n', f)
# shut down the templater
fm.end()

Internally, template data is modeled as a list of dictionaries, which is well-matched to most of our output. Here, we construct a list of (size, flags, path) elements.

3. Specifying a format

There are several types of format we might want to specify:

• built-in (json, xml, pickle...)
• style we ship (compact, svn...)
• path to a style (/home/bob/lib/myannotate)
• inline template spec ("{path}\t{size}\n")
• path to a file containing a template spec (not a style)

Ideally, we'd add a single new command-line option like -T/--template that would give us convenient access to all of these modes:

• -T xml
• -T compact
• -T /some/path (how do we know if this is a style or a template?)
• -T '{author}\n'

Specifying a precedence order and/or parsing rules may make this doable, but we need to avoid causing confusion if there happens to be a file 'xml' in the working directory or the user specifies a trivial template like '.'.

3.1. Per-item templates vs overall templates

One unresolved issue is how to deal with a per-item template description (ie what 'hg log --template' gives) vs an overall template that specifies start, end, and inter-item text as would be needed to build something like an HTML table. Most of the time, we'll want the former for convenience.

3.2. Per-command styles, legacy styles, and verbosity levels

The existing templating system for log doesn't envision support for more than just the log-like commands, but we'll need to continue to support template styles built for this scheme.

We'd also like to have single-file template maps that handle a variety of commands. This might be done by having one map key per command. Dealing with verbosity might be done either with template conditionals or with '.verbose' key suffixes.

4. JSON, XML, and encoding troubles

Given our simple schema, it's a relatively easy matter to construct outputs in standard marshalling formats. One trick here, however, is dealing with non-ASCII data. Given things like filenames may not be in UTF-8 and we often even know their encoding (see EncodingStrategy), we can't simply convert to Unicode. Instead, we'll use "UTF-8b", a scheme that can round-trip arbitrary binary data through UTF-8. This is used in various places which have encountered similar problems, for instance Python 3 uses this scheme for dealing with Unix filenames.

5. Steps

• add a formatter object to track formatting state

• add pass-through backend

• add debugging backend

• add UTF-8b encoder

• add JSON and XML backends
• add template backend
• add -T option handler
• convert core commands
• convert log-like commands
• convert remaining commands

6. Status

As of 2.4, Mercurial can generate some basic template debugging output for one command:

$ hg status --config ui.formatdebug=True
status = {
    {'status': 'M', 'path': 'mercurial/commands.py'},
    {'status': '?', 'path': 'changes.txt'},
    {'status': '?', 'path': 'clonecache.py'},
    {'status': '?', 'path': 'contrib/minimal.wsgi'},
    {'status': '?', 'path': 'mercurial-2.4-rc.tar.gz'},
    {'status': '?', 'path': 'mercurial-2.4.tar.gz'},
    {'status': '?', 'path': 'tests/test-symlink-replace-dir.t'},
}

CategoryNewFeatures