API reference

Loading

tplx.Loader(roots=(), *, engines=None, undefined='strict', globals=None, encoder=None, configure=None)

Finds templates under roots and renders them with the engine each one needs.

A name is resolved against every root in order. In each directory an exact file name wins; otherwise, among the files whose stem is the name (boot.cfg finds boot.cfg.j2), the one whose suffix the highest-priority engine claims, then the lowest name. The engines are consulted highest PRIORITY first and the first whose can_process accepts the file builds it, so the suffix picks the engine.

engines defaults to the registry as it is when the loader is built. undefined is one of UNDEFINED_MODES. globals are names every render sees, under the data passed to render. encoder presents the data to the engines that cannot evaluate Python (a DataEncoder by default). configure maps an engine class to a function called once with that engine's own environment whenever a lookup builds one, for the class and its subclasses: {JinjaTemplate: lambda env: env.filters.update(q=quote)}.

Source code in src/tplx/_loader.py

def __init__(
    self,
    roots: "Iterable[Any]" = (),
    *,
    engines: "Optional[Iterable[type]]" = None,
    undefined: str = "strict",
    globals: "Optional[Mapping[str, Any]]" = None,
    encoder: "Optional[DataEncoder]" = None,
    configure: "Optional[Mapping[type, Callable[[Any], Any]]]" = None,
) -> None:
    if undefined not in UNDEFINED_MODES:
        raise ValueError(
            f"undefined must be one of {', '.join(UNDEFINED_MODES)}, "
            f"not {undefined!r}"
        )
    self.roots: list = [_as_path(root) for root in roots]
    self.engines: tuple = by_priority(
        _registry.registered_engines() if engines is None else engines
    )
    self.undefined = undefined
    self.globals: dict = dict(globals or {})
    self.encoder = encoder if encoder is not None else DataEncoder()
    self.configure: "Dict[type, Callable[[Any], Any]]" = dict(configure or {})
    for cls, function in self.configure.items():
        if not isinstance(cls, type) or not callable(function):
            raise TypeError(
                "configure maps an engine class to a callable, "
                f"not {cls!r} to {function!r}"
            )
    self._lookups: "OrderedDict[Hashable, Lookup]" = OrderedDict()

find(name, **lookup)

self.lookup(**lookup).find(name).

Source code in src/tplx/_loader.py

def find(self, name: "Names", **lookup: Any) -> Source:
    """`self.lookup(**lookup).find(name)`."""
    return self.lookup(**lookup).find(name)

get(name, **lookup)

self.lookup(**lookup).get(name).

Source code in src/tplx/_loader.py

def get(self, name: "Names", **lookup: Any) -> Template:
    """`self.lookup(**lookup).get(name)`."""
    return self.lookup(**lookup).get(name)

lookup(*, candidates=None, paths=())

The loader bound to one caller's file names and search paths.

candidates(file_name) lists the file names to try for the last part of a name, most specific first; by default only that file name. The directory part of the name is kept. paths are searched before the roots: an anchored path as it is, a relative one inside each root. Includes inside a template resolve through the same Lookup. The same arguments return the same Lookup, which caches what it compiles, while it is among the LOOKUP_CACHE_SIZE most recently used; pass a candidates function that is defined once, not a new one per call.

Source code in src/tplx/_loader.py

def lookup(
    self,
    *,
    candidates: "Optional[Candidates]" = None,
    paths: "Sequence[Any]" = (),
) -> "Lookup":
    """The loader bound to one caller's file names and search paths.

    `candidates(file_name)` lists the file names to try for the last part
    of a name, most specific first; by default only that file name. The
    directory part of the name is kept. `paths` are searched before the
    roots: an anchored path as it is, a relative one inside each root.
    Includes inside a template resolve through the same `Lookup`. The same
    arguments return the same `Lookup`, which caches what it compiles,
    while it is among the `LOOKUP_CACHE_SIZE` most recently used; pass a
    `candidates` function that is defined once, not a new one per call.
    """
    try:
        key: Hashable = (candidates, tuple(_origin(p) for p in paths))
        hash(key)
    except TypeError:
        return Lookup(self, candidates=candidates, paths=paths)
    found = self._lookups.get(key)
    if found is not None:
        self._lookups.move_to_end(key)
        return found
    found = self._lookups[key] = Lookup(self, candidates=candidates, paths=paths)
    if len(self._lookups) > LOOKUP_CACHE_SIZE:
        self._lookups.popitem(last=False)
    return found

render(name, data=None, /, *, candidates=None, paths=(), **overrides)

self.lookup(candidates=..., paths=...).render(name, data, **overrides).

A data name that is candidates or paths goes in data.

Source code in src/tplx/_loader.py

def render(
    self,
    name: "Names",
    data: "Optional[Mapping[str, Any]]" = None,
    /,
    *,
    candidates: "Optional[Candidates]" = None,
    paths: "Sequence[Any]" = (),
    **overrides: Any,
) -> "Union[str, bytes]":
    """`self.lookup(candidates=..., paths=...).render(name, data, **overrides)`.

    A data name that is `candidates` or `paths` goes in `data`.
    """
    return self.lookup(candidates=candidates, paths=paths).render(
        name, data, **overrides
    )

tplx.Lookup(loader, *, candidates=None, paths=())

A Loader bound to one set of candidates and search paths.

Build one with Loader.lookup. It caches compiled templates by name, and reuses one while its file is unchanged.

Source code in src/tplx/_loader.py

def __init__(
    self,
    loader: Loader,
    *,
    candidates: "Optional[Candidates]" = None,
    paths: "Sequence[Any]" = (),
) -> None:
    self.loader = loader
    self.candidates = candidates
    self.paths = tuple(_as_path(path) for path in paths)
    self._templates: Dict[Hashable, Template] = {}
    self._state: Dict[Hashable, Any] = {}

clear()

Forget the compiled templates and engine state; the next render resolves afresh.

A cached template is reused while its own file is unchanged, so a more specific file added later is found only after this.

Source code in src/tplx/_loader.py

def clear(self) -> None:
    """Forget the compiled templates and engine state; the next render resolves afresh.

    A cached template is reused while its own file is unchanged, so a more
    specific file added later is found only after this.
    """
    self._templates.clear()
    self._state.clear()

find(name)

Find the file for name and read its bytes.

name may be a list of names: each is tried in turn, with all of its candidates in every search path, and the first file found wins.

:raises TemplateNotFound: nothing matches, or a name has a .. segment.

Source code in src/tplx/_loader.py

def find(self, name: "Names") -> Source:
    """Find the file for `name` and read its bytes.

    `name` may be a list of names: each is tried in turn, with all of its
    candidates in every search path, and the first file found wins.

    :raises TemplateNotFound: nothing matches, or a name has a `..` segment.
    """
    names = _names(name)
    for each in names:
        found = self._find_one(each)
        if found is not None:
            return found
    raise TemplateNotFound(names[0] if len(names) == 1 else names)

get(name)

The compiled template for name, from the cache while its file is unchanged.

:raises TemplateNotFound: no file matches. :raises NoEngineError: no engine claims the file, or it is not UTF-8 and the engine that claims it needs text. :raises EngineUnavailableError: the claiming engine's extra or program is missing. :raises RenderError: the engine could not compile it.

Source code in src/tplx/_loader.py

def get(self, name: "Names") -> Template:
    """The compiled template for `name`, from the cache while its file is unchanged.

    :raises TemplateNotFound: no file matches.
    :raises NoEngineError: no engine claims the file, or it is not UTF-8
        and the engine that claims it needs text.
    :raises EngineUnavailableError: the claiming engine's extra or program
        is missing.
    :raises RenderError: the engine could not compile it.
    """
    cached = self._templates.get(_key(name))
    if cached is not None and cached.is_up_to_date:
        return cached
    source = self.find(name)
    cls, body = self._engine_for(name, source)
    try:
        template = cls(body, lookup=self, origin=source.origin)
    except TplxError:
        raise
    except Exception as exc:
        raise RenderError(
            f"{cls.__name__} could not compile {source.origin}: {exc}"
        ) from exc
    template._uptodate = source.uptodate
    self._templates[_key(name)] = template
    return template

include(name, engine)

Resolve an include written in an engine template.

name is found and given an engine exactly as get would. When that engine is engine itself, Include.text holds the source for the including engine to compile natively (keeping its blocks, macros and context); otherwise it is None, and the engine inserts render_include(name, values) instead.

Source code in src/tplx/_loader.py

def include(self, name: str, engine: type) -> "Include":
    """Resolve an include written in an `engine` template.

    `name` is found and given an engine exactly as `get` would. When that
    engine is `engine` itself, `Include.text` holds the source for the
    including engine to compile natively (keeping its blocks, macros and
    context); otherwise it is `None`, and the engine inserts
    `render_include(name, values)` instead.
    """
    source = self.find(name)
    cls, body = self._engine_for(name, source)
    native = cls is engine and isinstance(body, str)
    return Include(source, cls, body if native else None)

render(name, data=None, /, **overrides)

Render name; its names are the loader's globals, then data, then overrides.

Text, or bytes from a BINARY engine.

:raises RenderError: the engine failed; UndefinedError for a missing name in strict mode. Also everything get raises.

Source code in src/tplx/_loader.py

def render(
    self,
    name: "Names",
    data: "Optional[Mapping[str, Any]]" = None,
    /,
    **overrides: Any,
) -> "Union[str, bytes]":
    """Render `name`; its names are the loader's globals, then `data`, then `overrides`.

    Text, or `bytes` from a `BINARY` engine.

    :raises RenderError: the engine failed; `UndefinedError` for a missing
        name in `strict` mode. Also everything `get` raises.
    """
    template = self.get(name)
    values = {**self.loader.globals, **(data or {}), **overrides}
    try:
        return template.render_values(values)
    except TplxError:
        raise
    except Exception as exc:
        raise RenderError(
            f"{type(template).__name__} failed to render {template.origin}: {exc}"
        ) from exc

render_include(name, values)

Render name with its own engine and values, as text to insert.

:raises NoEngineError: the result is bytes that are not UTF-8.

Source code in src/tplx/_loader.py

def render_include(self, name: str, values: "Mapping[str, Any]") -> str:
    """Render `name` with its own engine and `values`, as text to insert.

    :raises NoEngineError: the result is bytes that are not UTF-8.
    """
    rendered = self.render(name, {str(k): unwrap(v) for k, v in values.items()})
    if isinstance(rendered, bytes):
        try:
            return rendered.decode("utf-8")
        except UnicodeDecodeError as exc:
            raise NoEngineError(
                f"{name!r} renders bytes that are not UTF-8, which cannot be "
                f"included in a text template"
            ) from exc
    return rendered

resolve(name)

The file a render of name reads and the engine that renders it.

:raises TemplateNotFound: no file matches. :raises NoEngineError: no engine claims the file.

Source code in src/tplx/_loader.py

def resolve(self, name: "Names") -> "Tuple[Source, type]":
    """The file a render of `name` reads and the engine that renders it.

    :raises TemplateNotFound: no file matches.
    :raises NoEngineError: no engine claims the file.
    """
    source = self.find(name)
    return source, self._engine_for(name, source)[0]

search_paths()

The directories searched, in order: paths first, then the roots.

Source code in src/tplx/_loader.py

def search_paths(self) -> list:
    """The directories searched, in order: `paths` first, then the roots."""
    roots = self.loader.roots or [pathlib.Path(DEFAULT_ROOT)]
    found: list = []
    for path in self.paths:
        if _is_anchored(path):
            found.append(path)
        else:
            found.extend(root / str(path) for root in roots)
    found.extend(roots)
    return found

state(key, factory)

Per-lookup state for an engine, built by factory on first use.

An engine keeps what its includes need here (a jinja Environment, a mako TemplateLookup), so every template one lookup builds shares it.

Source code in src/tplx/_loader.py

def state(self, key: Hashable, factory: Callable[[], Any]) -> Any:
    """Per-lookup state for an engine, built by `factory` on first use.

    An engine keeps what its includes need here (a jinja `Environment`, a
    mako `TemplateLookup`), so every template one lookup builds shares it.
    """
    try:
        return self._state[key]
    except KeyError:
        value = self._state[key] = factory()
        return value

tplx.Source

Bases: NamedTuple

A template file as found: its bytes, where it came from, and a freshness check.

Presenting data

tplx.DataEncoder(*, expose=(), max_depth=8)

Presents render data to the engines that cannot evaluate Python.

• view(values) for Liquid, Handlebars and Mustache: structures become lazy read-only mappings, lists stay lists, values stay as they are.
• json(values) for ERB and EPP: a JSON-serialisable copy.
• flatten(values) for the shell engine: one level of text, keyed by key(path).

One rule decides what is a structure, default(value); subclass and override it (and key, text) to change how values are encoded, as with json.JSONEncoder.default.

expose lists _-prefixed names that json and flatten keep; they drop every other one, since such names are commonly back-references (an owner, a cache) that would drag a whole program in. max_depth bounds how deep json and flatten go; a value deeper than that is written as its text.

Source code in src/tplx/_encoder.py

def __init__(self, *, expose: "Iterable[str]" = (), max_depth: int = 8) -> None:
    self.expose = frozenset(expose)
    self.max_depth = max_depth

default(value)

What value stands for: a Mapping when it is a structure, else a value.

Called for anything that is not a scalar, a mapping, a list or a tuple. The default: an object whose class defines its own __str__ is a value (written as that text); any other object with attributes is a structure of its attributes; anything else is a value.

Source code in src/tplx/_encoder.py

def default(self, value: Any) -> Any:
    """What `value` stands for: a `Mapping` when it is a structure, else a value.

    Called for anything that is not a scalar, a mapping, a list or a tuple.
    The default: an object whose class defines its own `__str__` is a value
    (written as that text); any other object with attributes is a structure
    of its attributes; anything else is a value.
    """
    if _has_own_str(value):
        return value
    if hasattr(value, "__dict__") or hasattr(value, "__slots__"):
        return DataView(value, self)
    return value

flatten(values)

values as one level of text, keyed by key(path).

List items are keyed by their index. Two paths that give one key log a warning naming both values; the later one wins.

Source code in src/tplx/_encoder.py

def flatten(self, values: "Mapping[str, Any]") -> Dict[str, str]:
    """`values` as one level of text, keyed by `key(path)`.

    List items are keyed by their index. Two paths that give one key log a
    warning naming both values; the later one wins.
    """
    result: Dict[str, str] = {}
    self._flatten(values, [], result, 0, frozenset())
    return result

json(values)

A JSON-serialisable copy of values; anything else is written as its text.

Source code in src/tplx/_encoder.py

def json(self, values: "Mapping[str, Any]") -> Any:
    """A JSON-serialisable copy of `values`; anything else is written as its text."""
    return self._json(values, 0, frozenset())

key(path)

The flattened name of the value at path: joined with _, upper-cased.

Source code in src/tplx/_encoder.py

def key(self, path: "Sequence[str]") -> str:
    """The flattened name of the value at `path`: joined with `_`, upper-cased."""
    return "_".join(path).upper()

structure(value)

value as a Mapping, a list, or a value that is neither.

Source code in src/tplx/_encoder.py

def structure(self, value: Any) -> Any:
    """`value` as a `Mapping`, a `list`, or a value that is neither."""
    if isinstance(value, self.SCALARS):
        return value
    if isinstance(value, Mapping):
        return value
    if isinstance(value, (list, tuple)):
        return list(value)
    return self.default(value)

text(value)

How flatten writes a value: None empty, a boolean true/false.

Source code in src/tplx/_encoder.py

def text(self, value: Any) -> str:
    """How `flatten` writes a value: `None` empty, a boolean `true`/`false`."""
    if value is None:
        return ""
    if isinstance(value, bool):
        return str(value).lower()
    return str(value)

view(values)

The top-level names of a data template.

Source code in src/tplx/_encoder.py

def view(self, values: "Mapping[str, Any]") -> Dict[str, Any]:
    """The top-level names of a data template."""
    return {str(key): self.wrap(value) for key, value in values.items()}

wrap(value)

value for a data template: a view for a structure, a value as it is.

Source code in src/tplx/_encoder.py

def wrap(self, value: Any) -> Any:
    """`value` for a data template: a view for a structure, a value as it is."""
    found = self.structure(value)
    if isinstance(found, DataView):
        return found
    if isinstance(found, Mapping):
        return DataView(found, self)
    if isinstance(found, list):
        return [self.wrap(item) for item in found]
    return found

tplx.DataView(obj, encoder)

Bases: Mapping

A lazy, read-only mapping over an object's items or attributes.

Nothing is copied, so a view never goes stale against what it wraps. Values are presented through the encoder that made the view. Dunder names are hidden; a single leading underscore is not.

Source code in src/tplx/_encoder.py

def __init__(self, obj: Any, encoder: "DataEncoder") -> None:
    object.__setattr__(self, "_obj", obj)
    object.__setattr__(self, "_encoder", encoder)

The engine contract

tplx.Template(source, *, lookup=None, origin='<string>')

A template an engine has compiled, ready to render.

A subclass sets EXT (the suffixes it claims), optionally PRIORITY and BINARY, and implements render_values. The loader builds it with the file's contents, the Lookup that found it and the file's origin; an engine with an include syntax resolves includes through self.lookup, so they search the same roots as the file that includes them.

Source code in src/tplx/_template.py

def __init__(
    self,
    source: "Union[str, bytes]",
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    self.source = source
    self.lookup = lookup
    self.origin = origin
    self._uptodate: "Optional[Callable[[], bool]]" = None

encoder property

The DataEncoder this template presents its data with.

is_up_to_date property

Is the file this was loaded from unchanged? True when there is no file.

undefined property

The UNDEFINED_MODES entry this template renders with.

can_process(file, source) classmethod

Does this engine render file? By default, when it claims the suffix.

Source code in src/tplx/_template.py

@classmethod
def can_process(cls, file: Any, source: "Union[str, bytes]") -> bool:
    """Does this engine render `file`? By default, when it claims the suffix."""
    return can_process_suffix(cls, file)

configure_native(native)

Apply the loader's configure functions for this engine to native.

An engine calls it once, where it builds its own environment for a lookup; every function whose key is this engine's class or a base of it runs, in the order given. Returns native.

Source code in src/tplx/_template.py

def configure_native(self, native: Any) -> Any:
    """Apply the loader's `configure` functions for this engine to `native`.

    An engine calls it once, where it builds its own environment for a
    lookup; every function whose key is this engine's class or a base of it
    runs, in the order given. Returns `native`.
    """
    if self.lookup is not None:
        for cls, function in self.lookup.loader.configure.items():
            if isinstance(self, cls):
                function(native)
    return native

render(data=None, /, **overrides)

Render with data, then overrides, as the template's top-level names.

A keyword wins over the same name in data. An engine's own exception is raised as RenderError, with it as the __cause__.

Source code in src/tplx/_template.py

def render(
    self, data: "Optional[Mapping[str, Any]]" = None, /, **overrides: Any
) -> "Union[str, bytes]":
    """Render with `data`, then `overrides`, as the template's top-level names.

    A keyword wins over the same name in `data`. An engine's own exception
    is raised as `RenderError`, with it as the `__cause__`.
    """
    values = dict(data or {})
    values.update(overrides)
    try:
        return self.render_values(values)
    except TplxError:
        raise
    except Exception as exc:
        raise RenderError(
            f"{type(self).__name__} failed to render {self.origin}: {exc}"
        ) from exc

render_values(values)

What an engine implements: render with values as the top-level names.

Source code in src/tplx/_template.py

def render_values(self, values: "Mapping[str, Any]") -> "Union[str, bytes]":
    """What an engine implements: render with `values` as the top-level names."""
    raise NotImplementedError

tplx.register_engine(cls=None, *, priority=None)

register_engine(
    cls: _Engine, *, priority: Optional[int] = None
) -> _Engine

register_engine(
    cls: None = None, *, priority: Optional[int] = None
) -> Callable[[_Engine], _Engine]

Add an engine to the registry and return it; usable as a decorator.

priority overrides the class's PRIORITY. Registering a class that is already registered does nothing beyond that. A class that would claim a suffix another registered engine claims at the same priority raises ValueError and changes nothing: give one of them a higher priority to say which wins.

Source code in src/tplx/_registry.py

def register_engine(
    cls: "Optional[_Engine]" = None, *, priority: Optional[int] = None
) -> Any:
    """Add an engine to the registry and return it; usable as a decorator.

    `priority` overrides the class's `PRIORITY`. Registering a class that is
    already registered does nothing beyond that. A class that would claim a
    suffix another registered engine claims at the same priority raises
    `ValueError` and changes nothing: give one of them a higher priority to say
    which wins.
    """

    def _register(cls):
        wanted = _priority(cls) if priority is None else int(priority)
        for other in _ENGINES:
            if other is not cls and _conflict(cls, wanted, other):
                raise ValueError(
                    f"{cls.__name__} and {other.__name__} both claim "
                    f"{template_extensions(cls) or 'every suffix'} at priority "
                    f"{wanted}; register one with a different priority"
                )
        if priority is not None:
            cls.PRIORITY = priority
        if cls not in _ENGINES:
            _ENGINES.append(cls)
        return cls

    return _register if cls is None else _register(cls)

tplx.unregister_engine(cls)

Remove an engine from the registry; True if it was registered.

Source code in src/tplx/_registry.py

def unregister_engine(cls) -> bool:
    """Remove an engine from the registry; True if it was registered."""
    if cls in _ENGINES:
        _ENGINES.remove(cls)
        return True
    return False

tplx.registered_engines()

The registered engines, highest PRIORITY first.

Source code in src/tplx/_registry.py

def registered_engines() -> tuple:
    """The registered engines, highest `PRIORITY` first."""
    _load_entry_points()
    return by_priority(_ENGINES)

Engines

tplx.engines

The shipped engines, registered on import.

An optional engine is registered whether or not its library is installed: a .mako file without the extra raises EngineUnavailableError naming it, rather than being copied as bytes because nothing claimed it. Importing this package imports no engine library.

CheetahTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Cheetah template, rendered by CT3. Needs the cheetah extra.

The data is the template's search list, as it is: $target.hostname reads a key or an attribute. #include "name" resolves through the Lookup that loaded the file, exactly as a render would: a Cheetah file is compiled by Cheetah (#include raw inserts its text as it is), any other is rendered by its own engine with the render's data and its output inserted; a name found nowhere raises TemplateNotFound under strict and renders empty otherwise. strict raises an undefined name as UndefinedError, lenient renders it empty and debug writes it back as $name. Loader(configure={CheetahTemplate: function}) receives the dict of Cheetah compiler settings.

Source code in src/tplx/engines/_cheetah.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    _, _, self._not_found = _cheetah()
    mode = self.undefined

    def factory() -> Tuple[type, str]:
        settings = self.configure_native({})
        return _bridge(lookup, type(self), mode, settings)

    bridge, prefix = (
        lookup.state(("cheetah", type(self), mode), factory)
        if lookup is not None
        else factory()
    )
    self._class = bridge.compile(source=prefix + source)

CopyTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

Return the file's bytes, unchanged; claims every suffix, consulted last.

BINARY, so the file is never decoded: a CRLF file, latin-1 text or an EFI binary all come back exactly as stored, and render returns bytes. A text file that still holds %{NAME} placeholders logs a warning on its first render, since it was probably meant for the shell engine.

Source code in src/tplx/engines/_copy.py

def __init__(
    self,
    source: "Union[bytes, str]",
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    if not isinstance(source, bytes):
        source = str(source).encode("utf-8")
    super().__init__(source, lookup=lookup, origin=origin)
    self._checked = False

EPPTemplate(source, *, lookup=None, origin='<string>')

Bases: SubprocessTemplate

A Puppet EPP template, rendered by puppet epp render (TPLX_PUPPET names another).

Every top-level name is an EPP parameter: <%= $target['hostname'] %>. The output is the template's own bytes: no newline is added at the end. CRLF in puppet's output is turned into LF, so the result does not depend on the platform; a .epp template cannot emit CRLF on purpose.

Source code in src/tplx/engines/_external.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    # Fail while loading, where the file that needs the program is named.
    self.executable()

ERBTemplate(source, *, lookup=None, origin='<string>')

Bases: SubprocessTemplate

An ERB template, rendered by the system ruby (TPLX_RUBY names another).

@context is the whole data hash and each other top-level name is also an instance variable: <%= @target['hostname'] %>; a name context is @context['context']. <%- -%> trims.

Source code in src/tplx/engines/_external.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    # Fail while loading, where the file that needs the program is named.
    self.executable()

EnvsubstTemplate(source, *, lookup=None, origin='<string>')

Bases: ShellTemplate

$NAME and ${NAME} as envsubst writes them (.envsubst, .template).

DIALECT picks which envsubst it reads like, from DIALECTS: gnu (the default: no defaults, no $$ escape, an expression it does not know is left as it is), a8m (POSIX defaults and alternates, $$ escapes) or drone (braced only; defaults, slices, replacements, trims, case, length). Setting PATTERN or FEATURES in a subclass overrides the dialect's. Values come from the render's data, flattened as for the shell engine (pass os.environ as data, or tplx render --env); names match case-insensitively, and an undefined name follows the undefined mode (debug keeps it, as nginx's own $uri should be).

class DroneEnvsubst(EnvsubstTemplate):
    DIALECT = "drone"

Source code in src/tplx/_template.py

def __init__(
    self,
    source: "Union[str, bytes]",
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    self.source = source
    self.lookup = lookup
    self.origin = origin
    self._uptodate: "Optional[Callable[[], bool]]" = None

syntax() classmethod

PATTERN and FEATURES, each the dialect's unless the class sets it.

:raises ValueError: an unknown DIALECT, PATTERN or feature.

Source code in src/tplx/engines/_shell.py

@classmethod
def syntax(cls) -> Tuple[str, FrozenSet[str]]:
    """`PATTERN` and `FEATURES`, each the dialect's unless the class sets it.

    :raises ValueError: an unknown `DIALECT`, `PATTERN` or feature.
    """
    cached = cls.__dict__.get("_syntax")
    if cached is not None:
        return cached
    if cls.DIALECT not in cls.DIALECTS:
        raise ValueError(
            f"{cls.__name__}.DIALECT must be one of {', '.join(cls.DIALECTS)}, "
            f"not {cls.DIALECT!r}"
        )
    pattern, features = cls.DIALECTS[cls.DIALECT]
    resolved = type(
        cls.__name__,
        (ShellTemplate,),
        {
            "PATTERN": cls.PATTERN if cls.PATTERN is not None else pattern,
            "FEATURES": cls.FEATURES if cls.FEATURES is not None else features,
        },
    ).syntax()
    cls._syntax = resolved
    return resolved

GoTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Go text/template template, rendered by gotpl. Needs the go extra (Python 3.11 or later).

The data is passed as it is: .Name reads a key or an attribute. Go's builtin functions only; Loader(configure={GoTemplate: function}) receives the dict of functions, so Sprig is functions.update(gotpl.funcs.sprig.text_func_map()).

{{template "name" .}} and {{block}} names that the file does not define resolve through the Lookup that loaded it, exactly as a render would: a Go file joins the template set, any other is rendered by its own engine with the pipeline's data (a mapping, or nothing) and its output inserted. A name found nowhere raises TemplateNotFound when it executes under strict and renders empty otherwise. strict raises a missing key as UndefinedError; Go has no empty mode, so lenient and debug both write <no value>. Output keeps the template's text as it is.

Source code in src/tplx/engines/_go.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    self._gotpl = gotpl = _gotpl()
    functions: Dict[str, Callable[..., Any]] = (
        dict(lookup.state(("go-functions", type(self)), self._functions))
        if lookup is not None
        else {}
    )
    functions[INCLUDE_FUNCTION] = self._include
    functions[MISSING_FUNCTION] = self._missing
    self._checks: List[Callable[[], bool]] = []
    sources = {origin: source}
    self._collect(source, sources)
    environment = gotpl.Environment(
        functions=functions,
        delimiters=self.DELIMITERS,
        missing_key=_MISSING_KEY[self.undefined],
    )
    self._root = origin
    self._engine = environment.from_sources(sources)

is_up_to_date property

Is the file unchanged, and every file it includes?

HandlebarsTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Handlebars template, rendered by pybars3. Needs the handlebars extra.

{{> name}} partials resolve through the Lookup that loaded the file, exactly as a render would: a Handlebars file is compiled by Handlebars, any other is rendered by its own engine with the template's data and its output inserted. Handlebars has no strict mode: an undefined name renders empty whatever the undefined mode says. A partial that cannot be found raises TemplateNotFound under strict and renders empty otherwise, as in Mustache. As Handlebars specifies, {{x}} is HTML-escaped; {{{x}}} writes the value as it is.

Source code in src/tplx/engines/_handlebars.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    self._compiler = _pybars()()
    self._template = self._compiler.compile(source)

JinjaTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Jinja template. Needs the jinja extra.

{% include %}, {% extends %}, {% import %} and {% from %} resolve through the Lookup that loaded the file, exactly as a render would: a Jinja file is compiled by Jinja, any other is rendered by its own engine with the current context and its output inserted. All three undefined modes apply. Output keeps the template's trailing newline and, as Jinja does by default, writes every line ending as LF.

Source code in src/tplx/engines/_jinja.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    jinja2 = self._jinja2 = _jinja2()
    mode = self.undefined
    key = ("jinja", type(self), mode)

    def factory():
        return self.configure_native(_environment(jinja2, lookup, type(self), mode))

    environment = lookup.state(key, factory) if lookup is not None else factory()
    code = environment.compile(source, name=origin, filename=origin)
    self._template = environment.template_class.from_code(
        environment, code, environment.make_globals(None), None
    )

LiquidTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Liquid template, rendered by python-liquid. Needs the liquid extra.

Liquid reads mappings, not attributes, so objects in the data are presented as read-only mappings of their attributes. {% include %} and {% render %} resolve through the Lookup that loaded the file, exactly as a render would: a Liquid file is parsed by Liquid, any other is rendered by its own engine with the render's data and its output inserted. All three undefined modes apply.

Source code in src/tplx/engines/_liquid.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    liquid, TemplateSource = _liquid()
    self._liquid = liquid
    mode = self.undefined
    key = ("liquid", type(self), mode)

    def factory():
        return self.configure_native(
            _environment(liquid, TemplateSource, lookup, type(self), mode)
        )

    environment = lookup.state(key, factory) if lookup is not None else factory()
    self._template = environment.from_string(source, name=origin)

MakoTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A mako template. Needs the mako extra.

<%include file=.../>, <%inherit file=.../> and <%namespace file=.../> resolve through the Lookup that loaded the file, exactly as a render would: a mako file is compiled by mako, any other is rendered by its own engine with the template's data and its output inserted. Names are root-relative. strict is mako's own strict_undefined (raising UndefinedError); lenient renders an undefined name empty and logs it. debug behaves as lenient: compiled mako cannot re-emit the ${...} it came from. A data name in RESERVED_NAMES is dropped with a warning.

Source code in src/tplx/engines/_mako.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    Mako, TemplateLookup, self._runtime = _mako()
    strict = self.undefined == "strict"
    key = ("mako", type(self), strict)

    def factory():
        return self.configure_native(
            _template_lookup(TemplateLookup, Mako, lookup, type(self), strict)
        )

    mako_lookup = lookup.state(key, factory) if lookup is not None else factory()
    # The lookup's template_args (strict_undefined, and whatever `configure`
    # set, such as imports or default_filters) build every template.
    self._template = Mako(
        source, lookup=mako_lookup, uri=origin, **mako_lookup.template_args
    )

MustacheTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

A Mustache template, rendered by chevron. Needs the mustache extra.

{{> name}} partials resolve through the Lookup that loaded the file, exactly as a render would: a Mustache file is parsed by Mustache, any other is rendered by its own engine with the template's data and its output inserted. Under strict a partial that cannot be found raises TemplateNotFound, otherwise it renders empty. chevron has no strict mode for names: an undefined name renders empty whatever the undefined mode says. As Mustache specifies, {{x}} is HTML-escaped; {{{x}}} or {{& x}} writes the value as it is.

Source code in src/tplx/engines/_mustache.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    self._chevron = _chevron()

ShellTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

%{UPPER_SNAKE} placeholders substituted from the flattened data.

The data is flattened by the loader's DataEncoder.flatten: by default nested names join with _ and are upper-cased ({"target": {"hostname": h}} is %{TARGET_HOSTNAME}), None renders empty and a boolean as true/false. Placeholders match names case-insensitively. %{NAME:-default} and %{NAME-default} work as in POSIX shells, as do %{NAME:1:3} (a slice) and %{NAME//pattern/text} (a replacement), and %% is a literal %. Only the braced form is on by default, so a bare % is text: a kickstart keeps %packages and a script keeps date +%Y%m%d. An expression it does not know (%{NAME:?x}) is text: its % is written and the scan goes on inside it. All three undefined modes apply.

Configured by subclassing:

class DollarTemplate(ShellTemplate):
    EXT = (".tpl", ".cfg")
    DELIMITER = "$"
    PATTERN = "all"         # ${NAME} and $NAME
    FEATURES = ShellTemplate.FEATURES | {"trim", "case"}

Source code in src/tplx/_template.py

def __init__(
    self,
    source: "Union[str, bytes]",
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    self.source = source
    self.lookup = lookup
    self.origin = origin
    self._uptodate: "Optional[Callable[[], bool]]" = None

expand(text, variables)

text with every placeholder substituted from variables (UPPER names).

Source code in src/tplx/engines/_shell.py

def expand(self, text: str, variables: "Mapping[str, str]") -> str:
    """`text` with every placeholder substituted from `variables` (UPPER names)."""
    pattern, features = self.syntax()
    delimiter = str(self.DELIMITER)
    braced = pattern in ("braced", "all")
    unbraced = pattern in ("unbraced", "all")
    parts: List[str] = []
    position = 0
    while True:
        start = text.find(delimiter, position)
        if start < 0:
            parts.append(text[position:])
            return "".join(parts)
        parts.append(text[position:start])
        after = start + len(delimiter)
        if "escape" in features and text.startswith(delimiter, after):
            parts.append(delimiter)
            position = after + len(delimiter)
            continue
        if braced and text.startswith("{", after):
            close = self._closing(text, after + 1, delimiter)
            if close is not None:
                whole = text[start : close + 1]
                found = self._braced(text[after + 1 : close], whole, variables)
                if found is not None:
                    parts.append(found)
                    position = close + 1
                    continue
        if unbraced:
            name = _NAME.match(text, after)
            if name is not None:
                whole = text[start : name.end()]
                parts.append(self._value_or_keep(name.group(), whole, variables))
                position = name.end()
                continue
        parts.append(delimiter)
        position = after

pattern() classmethod

A pattern finding where a placeholder starts (escaped, braced, named).

Cached in the class's own __dict__, so a subclass with another delimiter never reuses its parent's pattern.

Source code in src/tplx/engines/_shell.py

@classmethod
def pattern(cls) -> "re.Pattern[str]":
    """A pattern finding where a placeholder starts (`escaped`, `braced`, `named`).

    Cached in the class's own `__dict__`, so a subclass with another
    delimiter never reuses its parent's pattern.
    """
    cached = cls.__dict__.get("_compiled_pattern")
    if cached is not None:
        return cached
    pattern, _ = cls.syntax()
    delimiter = re.escape(str(cls.DELIMITER))
    alternatives = [rf"(?P<escaped>{delimiter})"]
    if pattern in ("braced", "all"):
        alternatives.append(rf"\{{(?P<braced>{_IDENT})")
    if pattern in ("unbraced", "all"):
        alternatives.append(rf"(?P<named>{_IDENT})")
    compiled = re.compile(delimiter + "(?:" + "|".join(alternatives) + ")")
    cls._compiled_pattern = compiled
    return compiled

syntax() classmethod

This class's (PATTERN, FEATURES), checked.

:raises ValueError: an unknown PATTERN or feature.

Source code in src/tplx/engines/_shell.py

@classmethod
def syntax(cls) -> Tuple[str, FrozenSet[str]]:
    """This class's `(PATTERN, FEATURES)`, checked.

    :raises ValueError: an unknown `PATTERN` or feature.
    """
    cached = cls.__dict__.get("_syntax")
    if cached is not None:
        return cached
    pattern, features = cls.PATTERN, cls.FEATURES
    if pattern is None or features is None:
        raise ValueError(f"{cls.__name__} sets no PATTERN or FEATURES")
    if pattern not in PATTERNS:
        raise ValueError(
            f"{cls.__name__}.PATTERN must be one of {', '.join(PATTERNS)}, "
            f"not {pattern!r}"
        )
    unknown = set(features) - _FEATURES
    if unknown:
        raise ValueError(
            f"{cls.__name__}.FEATURES has unknown {sorted(unknown)}; "
            f"known: {', '.join(sorted(_FEATURES))}"
        )
    cls._syntax = (pattern, frozenset(features))
    return cls._syntax

variables(values)

values as the placeholders read them: the encoder's flatten, by UPPER name.

Source code in src/tplx/engines/_shell.py

def variables(self, values: "Mapping[str, Any]") -> Dict[str, str]:
    """`values` as the placeholders read them: the encoder's `flatten`, by UPPER name."""
    return {key.upper(): text for key, text in self.encoder.flatten(values).items()}

SubprocessTemplate(source, *, lookup=None, origin='<string>')

Bases: Template

Base for an engine that runs another language's renderer.

A subclass sets COMMAND, ENV_VAR, EXT and implements command(). The data reaches the program as JSON, made by the loader's DataEncoder.json: structures become objects, anything else its text, and _-prefixed names are dropped unless the encoder's expose lists them. Undefined modes do not apply; missing data is the other language's own null.

Source code in src/tplx/engines/_external.py

def __init__(
    self,
    source: str,
    *,
    lookup: "Optional[Lookup]" = None,
    origin: str = "<string>",
) -> None:
    super().__init__(source, lookup=lookup, origin=origin)
    # Fail while loading, where the file that needs the program is named.
    self.executable()

command(program, template_path, values_path)

The argument list to run.

Source code in src/tplx/engines/_external.py

def command(self, program: str, template_path: str, values_path: str) -> List[str]:
    """The argument list to run."""
    raise NotImplementedError

executable() classmethod

The program's full path.

:raises EngineUnavailableError: it is not installed.

Source code in src/tplx/engines/_external.py

@classmethod
def executable(cls) -> str:
    """The program's full path.

    :raises EngineUnavailableError: it is not installed.
    """
    name = (os.environ.get(cls.ENV_VAR) if cls.ENV_VAR else None) or cls.COMMAND
    found = name if os.path.isabs(name) else shutil.which(name)
    if not found or not os.path.exists(found):
        raise EngineUnavailableError(
            f"the {cls.__name__.replace('Template', '').lower()} engine needs "
            f"{cls.COMMAND!r} installed and on PATH"
            + (f"; or set {cls.ENV_VAR} to its path" if cls.ENV_VAR else "")
        )
    return found

Exceptions

tplx.exceptions

The errors tplx raises. Every one derives from TplxError.

EngineUnavailableError

Bases: TplxError

The engine that claims a file cannot run here.

Its library is not installed (the message names the extra) or the program it runs is not on PATH (the message names the program).

NoEngineError

Bases: TplxError

No engine claims the file found, or the file is not text its engine can read.

RenderError

Bases: TplxError

An engine failed to compile or render a template.

The engine's own exception is the __cause__; the message names the engine and the file.

RenderTimeoutError

Bases: RenderError, TimeoutError

A program-backed engine did not finish within its timeout.

TemplateNotFound(name, message=None)

Bases: TplxError, LookupError

No search path holds a file for the name, or the name escapes them.

names holds every name tried, in order; name is the first.

Source code in src/tplx/exceptions.py

def __init__(
    self, name: "Union[str, Sequence[str]]", message: "Optional[str]" = None
) -> None:
    self.names = (name,) if isinstance(name, str) else tuple(name)
    self.name = self.names[0]
    if message is None:
        if len(self.names) == 1:
            message = f"template not found: {self.name!r}"
        else:
            tried = ", ".join(repr(each) for each in self.names)
            message = f"none of these templates was found: {tried}"
    super().__init__(message)

TplxError

Bases: Exception

Base class of every error tplx raises.

UndefinedError

Bases: RenderError, NameError

The template used a name the data does not define, in strict mode.

Testing an engine

tplx.testing

The suite every engine passes, shipped so a third-party engine runs it too.

Subclass EngineContract in a pytest test module, name the engine and a source that writes the name value, and pytest collects the checks:

from tplx.testing import EngineContract

class TestShoutyEngine(EngineContract):
    engine = ShoutyTemplate
    source = "{{ name }}"
    expected = "TPLX"

expected is what rendering source with name="tplx" returns. Every test takes pytest's tmp_path fixture.

EngineContract

Checks one engine against the Template contract.