Dynamic directories

List of items in directories can be dynamically computed, using templates.

It is just as if directories were templates. In fact, piecutter actually implements this feature using templates. Here is the workflow when a cutter renders a directory:

  • given location, and given loader tells that resource is a directory...
  • try to get explicit tree listing, which can be either static or dynamic. At the moment, “explicit tree listing” means directory holds a file named .directory-tree:
    • handle this file as the “directory tree template”
    • render this file against data
    • the result is directory tree listing (typically as JSON)
  • else, get the implicit (and static) directory tree listing (all items in directory).
  • finally, render each item in directory tree listing.

Use cases

Since directories can be described using a template, you can use all features of template engines to render filenames, select templates or even alter data.

Here are some use cases:

  • skip or include files based on variables;
  • render a single template several times with different output filenames;
  • alter template context data for some templates;
  • include templates from third-party locations;
  • use loops, conditions, text modifiers... and all template-engine features!

Step by step tutorial

Let’s try to explain the dynamic tree templates with a story...

Given a directory

Let’s start with this simple directory:

/tmp/dynamic/
└── greeter.txt  # Contains "{{ greeter }} {{ name }}!"

First create the directory:

>>> template_dir = os.path.join(temp_dir, 'dynamic')
>>> os.mkdir(template_dir)

Then put a simple “greeter.txt” template in this directory:

>>> greeter_filename = os.path.join(template_dir, 'greeter.txt')
>>> greeter_content = "{{ greeter }} {{ name }}!"  # Jinja2
>>> open(greeter_filename, 'w').write(greeter_content)

Render directory as usual

As shown in Quickstart tutorial, we can render the directory.

First setup piecutter and data:

>>> import piecutter
>>> data = {u'greeter': u'Hello', u'name': u'world'}
>>> render = piecutter.Cutter(engine=piecutter.Jinja2Engine())
>>> render.loader.routes['file'].root = temp_dir

Then render directory. To check the result, we print attributes of each rendered file:

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))
Location: file://.../dynamic/greeter.txt
Name: greeter.txt
Path: dynamic/greeter.txt
Content: Hello world!

Introduce empty .directory-tree file

Let’s update the directory to introduce a blank .directory-tree file in directory:

>>> tree_filename = os.path.join(template_dir, '.directory-tree')
>>> open(tree_filename, 'w').write("")

So directory now looks like this:

/tmp/dynamic/
├── .directory-tree  # Blank file.
└── greeter.txt  # Contains "{{ greeter }} {{ name }}!"

What happens if we render the directory again?

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))

No output! greeter.txt file has not been rendered.

piecutter found the .directory-tree file and used it as an explicit list of files to render. Since the file is empty, nothing was rendered.

Register greeter.txt

Let’s register greeter.txt in .directory-tree file:

>>> import json
>>> items_to_render = [
...     {
...         "template": "greeter.txt"
...     }
... ]
>>> open(tree_filename, 'w').write(json.dumps(items_to_render))

And check “greeter.txt” now gets rendered:

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))
Location: file://.../dynamic/greeter.txt
Name: greeter.txt
Path: dynamic/greeter.txt
Content: Hello world!

Render greeter.txt multiple times

Each item in directory tree must tell template, and accepts optional filename and data. Let’s take advantage of those additional options to render “greeter.txt” multiple times with different filenames and different content:

>>> items_to_render = [
...     {
...         "template": "greeter.txt",
...         "filename": "hello.txt",  # Explicitely change filename.
...     },
...     {
...         "template": "greeter.txt",
...         "filename": "goodbye.txt",
...         "data": {"greeter": "Goodbye"}  # Alter context data.
...     }
... ]
>>> open(tree_filename, 'w').write(json.dumps(items_to_render))

And check the result:

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))
...     print('----')
Location: file://.../dynamic/greeter.txt
Name: hello.txt
Path: dynamic/hello.txt
Content: Hello world!
----
Location: file://.../dynamic/greeter.txt
Name: goodbye.txt
Path: dynamic/goodbye.txt
Content: Goodbye world!
----

As you can see, location attribute refers to template’s source, whereas name and path refer to rendered object.

.directory-tree is a template

.directory-tree itself is handled as a template! So we can use context data and all engine’s features to dynamically generate the items to render.

>>> items_template = """
...     [
...       {% for greeter in greeting_list|default(['hello', 'goodbye']) %}
...         {
...           "template": "greeter.txt",
...           "filename": "{{ greeter }}.txt",
...           "data": {"greeter": "{{ greeter|capitalize }}"}
...         }{% if not loop.last %},{% endif %}
...       {% endfor %}
...     ]"""
>>> open(tree_filename, 'w').write(items_template)

If we render single file .directory-tree, we get data in JSON format:

>>> tree = render('file://' + template_dir + '/.directory-tree', data)
>>> print(tree.read())  
[
    {
        "template": "greeter.txt",
        "filename": "hello.txt",
        "data": {"greeter": "Hello"}
    },
    {
        "template": "greeter.txt",
        "filename": "goodbye.txt",
        "data": {"greeter": "Goodbye"}
    }
]

And now the directory is dynamically rendered:

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))
...     print('----')
Location: file://.../dynamic/greeter.txt
Name: hello.txt
Path: dynamic/hello.txt
Content: Hello world!
----
Location: file://.../dynamic/greeter.txt
Name: goodbye.txt
Path: dynamic/goodbye.txt
Content: Goodbye world!
----

Include locations

In dynamic tree templates, items’ template is a location. By default, it is considered relative to parent directory. But it can be absolute.

Let’s update .directory-tree file so that it calls local “greeter.txt” and a remote resource (see also loaders about remote locations):

>>> items_to_render = [
...     {
...         "template": "greeter.txt",
...         "filename": "local.txt",
...     },
...     {
...         "template": "https://raw.githubusercontent.com/diecutter/diecutter/0.7/demo/templates/greetings.txt",
...         "filename": "remote.txt",
...     }
... ]
>>> open(tree_filename, 'w').write(json.dumps(items_to_render))

Then render the directory again and check both local and remote files are rendered:

>>> for item in render(u'file://' + template_dir, data):
...     print('Location: {}'.format(item.location))
...     print('Name: {}'.format(item.name))
...     print('Path: {}'.format(item.path))
...     print('Content: {}'.format(item.read()))
...     print('----')
Location: file://.../dynamic/greeter.txt
Name: local.txt
Path: dynamic/local.txt
Content: Hello world!
----
Location: https://raw.githubusercontent.com/diecutter/diecutter/0.7/demo/templates/greetings.txt
Name: remote.txt
Path: dynamic/remote.txt
Content: Hello world!
----

This feature allows you to render a template directory that includes parts of third-party templates.

Internals

The loader is the one who knows that a resource is a directory, how to get the directory tree template and how to get the static listing of files in a directory.

The cutter drives the loader, then uses the engine to render the directory tree template, and finally converts result (JSON) into actual list of locations.