Templatz for string substitutions

tzbitsby
Jason Kantz
Sep 2018
 vol 1.

As part of getting the tzbits publication up and going, I needed some form of templating solution. I started with the Saait boring HTML page generator (Posthuma, 2016), and it worked pretty well. That could have been my solution, but I was reading the source code, and there was this one bit that bothered me:

/* item block */
for (j = 0; j < LEN(templates); j++) {
    /* TODO: page is a special case for now */
    if (!strcmp(templates[j].name, "page")) {
    ...

So I started thinking about templating languages and some of the options, including rolling my own. By the way, if you just want to try templatz, see the README at

https://github.com/jkantz/templatz/blob/master/README.md

The templates I'm looking for need to produce static files, so no PHP or lit-html. For the interpolation, saait uses the really common ${identifier} convention, and only supports strings. That works for this project, but the main question is this: can control structures be introduced into templates without making them too complicated?

For a typical static site, the index page is where the control structure is--a loop over page items. The index combines a few reusable templates (header, item, and footer). The header and footer are populated using site-wide bindings, but the item template within is repeatedly filled using new bindings for each page. The content pages are made from reusable page header and page footer templates with bindings for a page merged with bindings for the site. So individual pages don't require loops, but they do need some composability.

To keep things simple, the saait command assumes the "for each page, fill item" loop applies for all templates, but makes page templates a special case where there is no loop.

How do other templating implementations handle looping?

One of the older tools in this space is the M4 Macro Processor (Kernighan & Ritchie, 1977). The GNU implementation of m4 (Seindal, 1989) is widely available in Linux distributions due to its use in autoconf. It has a few simple but powerful rules for generating textual output. Looping is accomplished through recursion, so control structures like foreach are actually library routines. The resulting template code is interesting, but does not lend itself to simple maintenance. The GNU m4 manual has this to

say about m4:

Some people find m4 to be fairly addictive. They first use m4 for simple problems, then take bigger and bigger challenges, learning how to write complex sets of m4 macros along the way. Once really addicted, users pursue writing of sophisticated m4 applications even to solve simple problems, devoting more time debugging their m4 scripts than doing real work. Beware that m4 may be dangerous for the health of compulsive programmers.

Another templating system is XSLT 1.0. One free software implementation is xsltproc, which is part of the libxml library (Veillard, 1999). Going down this road for static sites forces a choice between xhtml and html5, since html5 is not valid XML. With html5, the page content would need to be wrapped in CDATA tags. To contrast the control structures in XSLT, here is an XSLT foreach example,

<xsl:for-each select="item">
  <td>
    <xsl:apply-templates/>
  </td>
</xsl:for-each>

and here is a conditional,

<xsl:template match="item">
  <tr>
    <xsl:if test="position() mod 2 = 0">
       <xsl:attribute name="bgcolor">pink</xsl:attribute>
    </xsl:if>
    <xsl:apply-templates/>
  </tr>
</xsl:template>

Other general template libraries include perl's HTML::Template and a variation on that for Common Lisp, HTML-TEMPLATE (Weitz, 2003). The control-structure example for HTML-TEMPLATE is:

<table border=1>
  <!-- TMPL_LOOP rows -->
    <tr>
      <!-- TMPL_LOOP cols -->
        <!-- TMPL_IF colorful-style -->
          <td align="right" bgcolor="pink"><!-- TMPL_VAR content --></td>
        <!-- TMPL_ELSE -->
          <td align="right" ><!-- TMPL_VAR content --></td>
        <!-- /TMPL_IF -->
      <!-- /TMPL_LOOP -->
    </tr>
  <!-- /TMPL_LOOP -->
</table>

In these approaches, the templates need to accept lists or arrays, so the control structures naturally lead to more complexity since they lead to more data types--booleans and arrays or lists. And if strings are now one of several data types, then we need operations on them right? So now we need functions, and then ... no I think I just want strings, tz.

Going down this path of building out full-blown template languages is what Google engineers did very well with closure templates in 2009. Here's one of their foreach examples,

{foreach $operand in $operands}
  {if not isFirst($operand)} + {/if}
  {$operand}
{ifempty}
  0
{/foreach}

and if you populate it with ['alpha', 'beta', 'gamma'], you get "alpha + beta + gamma". I've used closure templates in some circumstances, but tzbits is not one of them. I'll pass on maintaining JavaScript _and_ Java dependencies just to generate static files.

What I ended up doing for the tzbits project was writing templatz, a short Python 2.7 program very much in the spirit of saait. templatz requires a config file for each page to be generated. Configs use python dict syntax and are parsed using ast.literal_eval.

The config contains the bindings and a description of how the template files are assembled. So instead of adding control structure and more data structures in the templates, there is multiplicity in the config. The index page config has the following structure:

{
    '_blocks': ["index-header.htm",
                ["index-item.htm",
                 "pages/page1.cfg",
                 "pages/page2.cfg"],
                "index-footer.htm"],
    'filename': "index.htm",
    'title': "Site title",
    'author': "T.S. Author",
    ...
}

The output is written to 'index.htm', specified by the dict key, 'filename'. The key '_blocks' is a special entry specifying template files to fill and combine. By default, the template files are filled using the values of the current config, but it's possible to add an override with multiple configs. The second template file entry above does this with a list where the first element is the template file name and the remaining elements are the configs to apply to it repeatedly. With globbing the _blocks entry shortens to

    '_blocks': ["index-header.htm",
                ["index-item.htm', "pages/*.cfg"],
                "index-footer.htm"],

The page config has no iteration and looks like

{
    '_blocks': ["page-header.htm", 
                "page1-content.htm",
                "page-footer.htm"],
    'title': "Page One",
    'filename': "page1.htm",
}

The page-header/footer are reused across multiple page configurations that are all in the same directory. Config strings can be html escaped when inserted into templates by prefixing the variable reference with a &, for example, "${&title}" will be

escaped and "${title}" will not.

The configs are composable so you could use scripts or Makefiles to do crazy things like assemble template files that are themselves the output of other templates.

The python implementation was short and straightforward, so it should be easy to use and maintain.

http://github.com/jkantz/templatz

References

Clark, James, Ed. 1999. XSL Transformations (XSLT) Version 1.0: W3C Recommendation 16. http://www.w3.org/TR/1999/REC-xslt-19991116

Kernighan, Brian. Ritchie, Dennis. 1977. The M4 macro processor. Bell Laboratories, Murray Hill, NJ. https://wolfram.schneider.org/bsd/7thEdManVol2/m4/m4.pdf

Google LLC. 2009. Closure Tools: Closure Templates Commands. https://developers.google.com/closure/templates/docs/commands

Mackenzie, David. 1991. autoconf [computer software]. http://www.gnu.org/software/autoconf

Polymer Authors. (2018). lit-html: Next-generation HTML Templates in JavaScript. https://polymer.github.io/lit-html

Posthuma, Hiltjo. 2016. Saait: a boring HTML page generator. https://codemadness.org/saait.html

Seindal, René. Pinard, François. et al. 1989. GNU M4 [computer software]. https://www.gnu.org/software/m4/m4.html

Tregar, S. 2000. HTML::Template : A module for using HTML templates with Perl. http://html-template.sourceforge.net/html_template.html

Veillard, Daniel, et. al. 1999. libxslt [computer software]. https://gitlab.gnome.org/GNOME/libxslt

Weitz, Edmund. 2003. HTML-TEMPLATE - Use HTML templates from Common Lisp. https://edicl.github.io/html-template/

SensioLabs. 2018. Twig: the flexible, fast, and secure template engine for PHP. https://twig.symfony.com/doc/2.x/