The time is finally upon us - I've gotta touch up my web site. I had to do the obligatory styling maintenance and fix up a few outdated sentences here and there, but I also have been putting off starting an actual blog for years now. If you've seen this site before, it looks mostly the same as it did previously, but is now running a completely different setup under the hood. I figured that talking about how I accomplished this is as good of a way as any to kick off my internet ramblings!

I've been meaning to do this for quite some time - but naturally, things have gotten in the way. First, like many programmers, I'm stubbornly picky about my tools, to such an extent that I've built out my own web framework a few times - first in Hack, then in Go, and most recently in Rust. All of that was in service of a grand plan I'd cooked up with a buddy many years ago: We essentially wanted to build a headless CMS à la Prismic, with a few key differences that I might ramble about in another post some time. Regardless, a lot of life has happened since then and now, and those projects have all stalled in favor of other stuff.

Nowadays, I get to work with Real Frameworks as a part of my day job, developing Javascript-heavy web apps that get deployed to the cloud, all of that noise. I've been longing for a cozy static site, with zero runtime Javascript or other dependencies ^{(well, unless you count fonts)}running on my cozy VPS. I decided then that I wanted to rebuild all of this statically.

Righto, how do we do it?

There are a few well-known static site generators out there - the first one that comes to mind is Hugo. Hugo seems great, and I've heard great things about it! It's tailored towards sites that have long-form content, not unlike this blog post you're reading. It also was designed with a heavy emphasis on user friendliness, such that you don't need to be a full-on programmer to use it, which I respect. However, upon first glance, there seems to be a bit of a learning curve - there's a whole concept of "themes" that is very intuitive to use, but not so much to develop against. I'm sure if I had looked at it for longer than ten minutes it would be fine, but part of the problem I'm trying to solve here is that I already have layouts and stylesheets that work, and I'd like to directly use those without any additional fuss. This drove me back into the arms of my old friend Gulp. It's now considered out-of-fashion by some, as bundlers like Webpack have risen to prominence, but I still find it enjoyable to work with.

For the uninitiated, Gulp is a Javascript build system. It has a surprisingly simple architecture built entirely around Transform Streams, which makes everything nicely composable. Their plugin guidelines have a clear emphasis on modularity, which gives rise to a rich ecosystem of useful plugins. The result of any task can be piped into another - so I can compile my LESS into CSS, pipe the result into a minifier, concatenate it all into a single file, and wrap the whole thing in the gulp-sourcemaps adapter for LESS so that I have that information in the debugger. It has simple-yet-powerful Glob and Chokidar integrations to boot, so that makes it quick and painless to set up a build system that covers all of my needs, along with a development mode that automatically rebuilds the appropriate files when I make changes.

I became familiar with Gulp through work over the past few years, and I've had a great time working with it. I also have a bit of experience working with Transform Streams to create Gulp Plugins for similar use cases, so this is a great opportunity to formalize those setups into my first public NPM package.

Okay, what about templating?

Great question! Well, perhaps now's a good time to take a step back and find out what we actually want. After all, I'm far from the first person to have this problem, so there may very well be a good solution out there already.

• We need nestable, composable layouts, to keep our markup DRY, which means we need:
• A templating language, like you said!
• The templating language should be simple and ergonomic - we're just rendering documents, not full-on application UIs.
• We should be able to refresh any template when necessary, to facilitate development,
• And we should only be loading files from disk when necessary, so we don't end up with runaway performance issues.
• I'd also like to have a configuration loaded separately from the templates, so that repeated elements (such as the navigation links up there in the masthead) can be easily edited or rearranged without fiddling with markup. After all, one of the purposes of templating is to keep our presentation markup separate from the data that drives it, wherever it's practical to do so.
• We should also be able to pass in data programmatically from our build pipeline without specifying it in hand-written configuration, so that we can always tailor the system to suit our needs.
• I'd also like to leave the door open for internationalization. As a passion project, I once translated a previous version of this site into Spanish, Italian, and about-5th-grade-level Swedish. I haven't redone that since this content was updated, but it's definitely on my radar. The separate configuration described above will go a long way towards making that feasible, but I also would like to configure entirely separate language versions for long-form content such as blog posts.
• If we end up designing our own solution, it needs to comply with the Gulp Plugin Guidelines. This means modularity and configurability above all else, in that order.

There is quite a variety of template engines out there, for various languages and use cases, ranging from "simple side project like this one" to "production-ready powerhouse used by web apps like Twitter". Admittedly, I didn't do very much research into this topic, but I quickly settled on Mustache because I had some prior experience with it. It's super simple, which covers all of our needs, and is designed to be runtime-agnostic, hence the self-styled "logic-less" description. It has a solid Javascript implementation ready to go. The only thing that bothers me is the vagueness of its official specification - I tried to find proper answers to questions I had about things like legal identifier names and didn't have much luck other than seeing its official test suite. However, I think they left a bit of wiggle room in there on purpose in service of it being runtime-agnostic, and the aforementioned Javascript implementation works just fine for my purposes. There is a workable Gulp plugin out there, but that setup's only real shortcoming in the context of our requirements is its lack of support for composable layouts. There are a few solutions to that described below.

Regarding our other criteria, there are a few options already out there. Handlebars is a powerful and well-used extension of Mustache, but it doesn't really provide what I'm looking for. There's also Twitter's super-fast Hogan project, which does provide a sort of "template inheritance" also used by some other projects, but the specification is even more alarmingly vague, and I'm a bit turned off by declaring template inheritance in the templates themselves rather than in the configuration that uses them. It makes sense in some contexts, but the goal here is to keep each template DRY, and we need to have programmatic access to the layout structure in order to refresh templates at each level when necessary. The handlebars-layouts package does provide a good solution to layouts, but it wasn't built as a Gulp plugin, and it's not clear how I would satisfy our reloading requirements without sacrificing performance.

After considering all of this (and probably letting my ambition get the better of me regardless), I decided to go ahead and make my own Gulp plugin.

Let's make a Gulp plugin

As I mentioned, Gulp plugins are just functions that return a native Node Transform Stream. It just has to accept Gulp's special input object, called a Vinyl, which abstracts away the input contents. Gulp takes care of loading input files for you, users pipe them through your plugin's stream, and it spits out the result on the other side, so that users can pipe that output into other tools. Unfortunately, the NPM mustache package does not support streams - there's no reason it shouldn't be able to, but for once in my life I'll exercise restraint and save that project for another time. For now, we'll have to read the entire file and process that, which is no problem at all thanks to the Vinyl abstraction layer.

The only other challenge was coming up with a clean enough API that accomplishes all of our goals, which I'll describe below. I also wanted to do this all in Typescript, which I've used extensively before, but never as a published NPM package and never running directly in the Node runtime. This just took a bit of trial and error with my "tsconfig.json" configurations, but it was easy enough to figure out.

How it works

I ended up creating a package called gulp-mustache-layout, which is really just a thin-ish wrapper around the mustache package. Variables are bound according to options passed into the plugin - more on that below. You can instantiate a global instance of the plugin, and provide it options that are inherited by everything the plugin creates. With the instance of the plugin you can load "Layout" instances that can be chained together to render nested templates.

The user can pass in a plain Javascript object to the vars option to programmatically bind template variables. Additionally, the varLoader option allows the user to configure loading static configuration separate from their templates. When that option is supplied, the plugin performs a user-defined transformation on the template's path, reads the resulting file path into memory, invokes the provided parser function on it, and binds the resulting object to rendered templates as variables. The path transformation and parsing operation are outsourced to the user for modularity's sake, so you can do whatever you want with it.

There are currently a few peculiarities about this plugin's usage of Mustache: When one template wraps another, the inner template replaces a {{> yield}} partial invocation in the outer template, inspired by ERB's generator-like yield keyword. The bindings of the parent template can also be accessed within the child template, but unlike normal Mustache, they are scoped by the name of the parent template. For example, if my main template has a variable named foo, then I can render main.foo in a child template. Furthermore, there is a special global object in the variables that can be accessed from any scope - I use this to push things like HTML titles upstream to the top-level layout.

The options and caveats mentioned above need to be more rigorously formalized and tested, and remain subject to change in pre-release versions. Additionally, as I continue to use this plugin myself, I'll certainly run into more patterns that I can extract to simplify the end usage. For now, here is a simplified version of my current setup:

Here I have my setup configured to read the .toml file of the same name as the template, and pass that configuration on to my templates.

import GulpMustacheLayout from 'gulp-mustache-layout';
let GMLayout = new GulpMustacheLayout({ 
    varLoader: {
        path: parsed => Path.format( { ...parsed, base: '', ext: '.toml' } ),
        parser: TOML.parse,
    }
})

My layouts look something like this:

const layouts = { 
    main: GMLayout.load('src/layouts/main.mustache'),
    page: GMLayout.load('src/layouts/page.mustache'), 
    post: GMLayout.load('src/layouts/post.mustache'),
}

To render my index page, I have a simple Gulp task, which consists of the content template piped into the main layout. (The glob I've included also matches the .toml file, but the plugin ignores anything that's not a Mustache template, for the sake of convenience in writing Watch tasks shown later on)

const INDEX = "src/content/index.@(mustache|toml)"
function RenderIndex() { 
    return src(INDEX)
    .pipe(layouts.main.done())
    .pipe(dest("dist"))
}

Elsewhere in my Gulpfile, I have a task called content that builds every page (not just my index). For development mode, I have a task like this that rebuilds the index page whenever the index template changes, or everything when the main layout template changes:

export function watchContent() { 
    watch(
        layoutBase.main + '.@(mustache|toml)',
        function(cb){
            layouts.main.reload() 
            content(cb)
        }
    )

    watch(INDEX, RenderIndex)
}

And in the case of pages that have nested layouts, rather than just main, it looks like the below snippet. My pages directory contains things rendered in the "page" format like this blog post appears on; namely, my About and Projects pages. The layouts returned by the gulp-mustache-layout instance have a wrap method, which renders the template supplied as an argument within the template rendered by the called object, so templates can be arbitrarily nested within one another.

const PAGES = "src/content/pages/*.@(mustache|toml)" 
function RenderPages() { 
    return src(PAGES) 
    .pipe(
        layouts.main
        .wrap(layouts.page)
        .done()
    ) 
    .pipe(dest("dist"))
}

My actual Gulpfile is a bit more intense than these snippets, since it contains some logic specific to my setup, in addition to handling non-HTML related tasks such as CSS preprocessing and asset handling. In any case, it takes care of the high-level layouts for me but still lets me use all of the other functionality these tools provide and customize it to my needs, which was exactly my intent with this project. I can use Mustache features individually at all levels of my site’s structure while keeping a neat distinction between my markup and my configuration, and pipe that output directly to other Gulp tools (such as an HTML minifier!), leveraging Gulp’s easy parallelism and nice glob and chokidar integrations.

Finally, I've actually finished a project!

Whew! That was a mouthful! Thanks for sticking with me! In the end, I'm pretty satisfied, and the result was good enough to render this web site. This project provided me with the simple but powerful static site generation I was hoping for, with a clean config that still exposes all the fiddly bits. It performs acceptably too, with most content updates finishing after less than 30 milliseconds on my machine. I make no claims that this is the most efficient or future-proof tooling out there, but it ended up being exactly what I wanted, I hope it may suit someone else's needs too.