Overlays

dotagents._overlays

Overlay support: the :class:Overlay type, plus the multi-overlay recompose.

An overlay is a directory (optionally carrying an overlay.toml manifest) that installs as a unit into <store>/overlays/<name>/, its files keeping their relative paths inside that directory. Everything that depends only on ONE overlay -- its name rules, its manifest, its setup script, its files, what it contributes to the managed AGENTS.md block -- is a method or property of :class:Overlay. Manifest keys:

• routing — lines appended to the core's "Load on demand" list.
• rules — overlay-relative paths to markdown files whose - **… bullet blocks are appended to "Always-on rules".
• requires — overlay names this one needs; overlays add installs them first.
• name, description, priority — see :meth:Overlay.read_manifest.

The operations over a set of overlays stay module functions: _compose_block folds their rules and routing into a base block, and :func:recompose_overlay_block rewrites a store's AGENTS.md with it.

DEFAULT_PRIORITY = 500 module-attribute

DEFAULT_SETUP_TIMEOUT = 300 module-attribute

OVERLAY_ROOT_NOTE = 'Overlay paths below use `$<NAME>_OVERLAY_ROOT` -- an environment variable `dotagents env` exports per installed overlay (its install dir); resolve it in a shell (`echo $ENGINEERING_OVERLAY_ROOT`; PowerShell `$env:ENGINEERING_OVERLAY_ROOT`) before opening the file with a file tool.' module-attribute

SCRIPT_NAME_RE = re.compile('^[A-Za-z0-9][A-Za-z0-9._-]*$') module-attribute

InstallResult()

Bases: object

What :meth:Overlay.install_to did. Unpacks as the historical (written, skipped, lines), skipped being the unchanged files.

Source code in src/dotagents/_overlays.py

def __init__(self) -> None:
    self.written = 0
    self.unchanged = 0
    #: Files replaced although they were not what dotagents installed
    #: (edited here): backed up first when a backup root was given.
    self.replaced: "list[str]" = []
    #: Files removed: not in the source, or cleared by ``prune``.
    self.removed: "list[str]" = []
    #: Files copied to the backup root before being replaced or removed.
    self.backed_up: "list[str]" = []
    self.lines: "list[str]" = []

backed_up = [] instance-attribute

lines = [] instance-attribute

removed = [] instance-attribute

replaced = [] instance-attribute

skipped property

unchanged = 0 instance-attribute

written = 0 instance-attribute

__iter__()

Source code in src/dotagents/_overlays.py

def __iter__(self):
    return iter((self.written, self.skipped, self.lines))

Overlay(path, store=None)

One overlay: a directory whose NAME identifies it and whose files install as a unit under <scope>/overlays/<name>/.

Construct it from the overlay's directory (a source dir or an installed dir -- both are overlays). Nothing is read at construction; every method reads the directory when called, so an Overlay made before its files land still sees them afterwards. The name-only rules (:meth:is_valid_name, :meth:normalize_name, :meth:root_var_for) are static so a caller holding just a name -- overlays add <name> -- uses the same rule as one holding a directory.

Source code in src/dotagents/_overlays.py

def __init__(
    self, path: "str | os.PathLike[str]", store: "str | os.PathLike[str] | None" = None
):
    self.path = Path(path)
    #: The store (an ``.agents`` root) this overlay was discovered in, when it
    #: came from :meth:`installed`; ``None`` for a bare directory. Not part
    #: of identity (``__eq__`` / ``__hash__`` are the path only).
    self.store: "Optional[Path]" = Path(store) if store is not None else None

DEFAULT_PRIORITY = DEFAULT_PRIORITY class-attribute instance-attribute

INSTALL_RECORD = '.dotagents-install.json' class-attribute instance-attribute

MANIFEST_NAME = 'overlay.toml' class-attribute instance-attribute

NAME_RE = re.compile('^[A-Za-z](?:[A-Za-z0-9_.-]*[A-Za-z0-9])?$') class-attribute instance-attribute

RESERVED_NAMES = frozenset(['con', 'prn', 'aux', 'nul'] + ['com%d' % i for i in range(1, 10)] + ['lpt%d' % i for i in range(1, 10)]) class-attribute instance-attribute

SETUP_SCRIPT_NAMES = ('setup.py',) class-attribute instance-attribute

SKIP_PARTS = frozenset({'.git', '.hg', '.svn', '__pycache__', '.mypy_cache', '.pytest_cache', '.ruff_cache', '.tox', '.venv', 'node_modules'}) class-attribute instance-attribute

__slots__ = ('path', 'store') class-attribute instance-attribute

install_record_path property

is_valid property

:meth:is_valid_name of :attr:name.

manifest_path property

name property

The overlay's name: its directory name.

normalized_name property

:meth:normalize_name of :attr:name.

path = Path(path) instance-attribute

priority property

The manifest priority, DEFAULT_PRIORITY when absent or unparseable (:meth:read_manifest already did the checking). Lower sorts earlier.

root_var property

:meth:root_var_for of :attr:name -- the <NAME>_OVERLAY_ROOT var.

sort_key property

The (priority, name) merge-order key.

Lower priority (default DEFAULT_PRIORITY, 500) sorts earlier; a numerically higher-priority overlay therefore lands later in the merged AGENTS.md block and wins on conflict -- the same convention _context.py uses. The tiebreaker is the manifest name (falling back to the directory name), then the directory name itself (two dirs can carry the same manifest name), so output is deterministic regardless of input order.

store = Path(store) if store is not None else None instance-attribute

__eq__(other)

Source code in src/dotagents/_overlays.py

def __eq__(self, other: object) -> bool:
    return isinstance(other, Overlay) and self.path == other.path

__fspath__()

Source code in src/dotagents/_overlays.py

def __fspath__(self) -> str:
    return str(self.path)

__hash__()

Source code in src/dotagents/_overlays.py

def __hash__(self) -> int:
    return hash(self.path)

__repr__()

Source code in src/dotagents/_overlays.py

def __repr__(self) -> str:
    return "Overlay(%r)" % str(self.path)

discover(root, store=None) classmethod

The overlays under ONE overlays/ root, sorted by name.

No registry, no manifest required: a directory under root is an overlay as long as its name passes :meth:is_valid_name (so .git/ __pycache__/dotfiles are skipped). is_dir() follows symlinks, so a symlink-to-dir counts. Empty if root is absent. This is the ONE discovery rule: :meth:installed folds it over several stores, and the overlays command's add/remove/sync use it directly for the single scope they install into. store is stamped on each result (see :attr:store).

Source code in src/dotagents/_overlays.py

@classmethod
def discover(
    cls, root: "str | os.PathLike[str]", store: "str | os.PathLike[str] | None" = None
) -> "list[Overlay]":
    """The overlays under ONE ``overlays/`` root, sorted by name.

    No registry, no manifest required: a directory under ``root`` *is* an
    overlay as long as its name passes :meth:`is_valid_name` (so ``.git``/
    ``__pycache__``/dotfiles are skipped). ``is_dir()`` follows symlinks, so a
    symlink-to-dir counts. Empty if ``root`` is absent. This is the ONE
    discovery rule: :meth:`installed` folds it over several stores,
    and the ``overlays`` command's ``add``/``remove``/``sync`` use it directly
    for the single scope they install into. ``store`` is stamped on each
    result (see :attr:`store`)."""
    root = Path(root)
    if not root.is_dir():
        return []
    found = [
        cls(p, store)
        for p in sorted(root.iterdir())
        if p.is_dir() and cls.is_valid_name(p.name)
    ]
    seen: "dict[str, str]" = {}
    for overlay in found:
        other = seen.setdefault(overlay.normalized_name, overlay.name)
        if other != overlay.name:
            _warn_once(
                "overlays %s and %s in %s are one overlay (%s) and share %s; remove one",
                other, overlay.name, root, overlay.normalized_name, overlay.root_var,
            )
    return found

files()

Files the overlay installs: everything except its manifest, its install record, VCS metadata and tool caches (:attr:SKIP_PARTS) and compiled *.pyc.

Source code in src/dotagents/_overlays.py

def files(self) -> "list[Path]":
    """Files the overlay installs: everything except its manifest, its
    install record, VCS metadata and tool caches (:attr:`SKIP_PARTS`) and
    compiled ``*.pyc``."""
    out = []
    for p in sorted(self.path.rglob("*")):
        rel = p.relative_to(self.path)
        if (
            p.is_file()
            and p.name != self.MANIFEST_NAME
            and rel.parts != (self.INSTALL_RECORD,)
            and p.suffix != ".pyc"
            and not self.SKIP_PARTS.intersection(rel.parts)
        ):
            out.append(p)
    return out

find_setup_script()

The overlay's setup script, or None if it ships none.

Call this on the installed overlay: setup runs against the copy under <scope>/overlays/<name>/ with that as its cwd, so a script can reference its own sibling files by relative path.

Source code in src/dotagents/_overlays.py

def find_setup_script(self) -> "Optional[Path]":
    """The overlay's setup script, or ``None`` if it ships none.

    Call this on the *installed* overlay: setup runs against the copy under
    ``<scope>/overlays/<name>/`` with that as its cwd, so a script can
    reference its own sibling files by relative path."""
    for name in self.SETUP_SCRIPT_NAMES:
        candidate = self.path / name
        if candidate.is_file():
            return candidate
    return None

ignore_rules()

The rules of the .gitignore / .ignore at the overlay's root (:mod:dotagents._ignore); one in a subdirectory is an ordinary file.

Source code in src/dotagents/_overlays.py

def ignore_rules(self):
    """The rules of the ``.gitignore`` / ``.ignore`` at the overlay's root
    (:mod:`dotagents._ignore`); one in a subdirectory is an ordinary file."""
    from dotagents._ignore import IgnoreRules

    return IgnoreRules.for_root(self.path)

install_to(dest_overlay_dir, dry_run, *, prune=False, backup_root=None, source=None)

Make dest_overlay_dir (<scope>/overlays/<name>/) exactly this overlay: every file it ships (see :meth:files) is installed or replaced, its overlay.toml refreshed, and every other file in the install is removed -- except what the .gitignore / .ignore at the overlay's root match, plus its install record and tool caches (:attr:SKIP_PARTS): those are the install's own (setup output, local state) and are never copied, replaced or removed. prune clears them too, leaving exactly the upstream files (and the record).

A file replaced or removed that is not what dotagents installed there (edited here, or never installed by it) is first copied to backup_root when one is given (:attr:InstallResult.backed_up, :attr:InstallResult.replaced); what prune clears is not backed up.

The install record (:attr:INSTALL_RECORD) keeps the digest of every file written and source (where the overlay came from; the previous value when None).

Source code in src/dotagents/_overlays.py

def install_to(
    self,
    dest_overlay_dir: Path,
    dry_run: bool,
    *,
    prune: bool = False,
    backup_root: "Optional[Path]" = None,
    source: "Optional[dict[str, Any]]" = None,
) -> "InstallResult":
    """Make `dest_overlay_dir` (`<scope>/overlays/<name>/`) exactly this
    overlay: every file it ships (see :meth:`files`) is installed or
    replaced, its `overlay.toml` refreshed, and every other file in the
    install is removed -- except what the ``.gitignore`` / ``.ignore`` at
    the overlay's root match, plus its install record and tool caches
    (:attr:`SKIP_PARTS`): those are the install's own (setup output,
    local state) and are never copied, replaced or removed. `prune`
    clears them too, leaving exactly the upstream files (and the record).

    A file replaced or removed that is not what dotagents installed there
    (edited here, or never installed by it) is first copied to
    `backup_root` when one is given (:attr:`InstallResult.backed_up`,
    :attr:`InstallResult.replaced`); what `prune` clears is not backed up.

    The install record (:attr:`INSTALL_RECORD`) keeps the digest of every
    file written and `source` (where the overlay came from; the previous
    value when `None`)."""
    dest = Path(dest_overlay_dir)
    installed = Overlay(dest)
    record = installed.read_install_record()
    old_files: "dict[str, str]" = record.get("files", {})
    new_files: "dict[str, str]" = {}
    result = InstallResult()
    rules = self.ignore_rules()

    def local(rel: str, target: Path) -> bool:
        """Is `target` something dotagents did not put there?"""
        return old_files.get(rel) != _file_digest(target)

    def backup(rel: str, target: Path) -> None:
        if backup_root is None or dry_run:
            return
        saved = backup_root / rel
        saved.parent.mkdir(parents=True, exist_ok=True)
        shutil.copy2(str(target), str(saved))
        result.backed_up.append(rel)

    def write(src: Path, target: Path) -> None:
        if not dry_run:
            target.parent.mkdir(parents=True, exist_ok=True)
            shutil.copy2(str(src), str(target))

    for src in self.files():
        rel = src.relative_to(self.path).as_posix()
        if rules.ignored(rel):
            continue  # the source's local state, not the overlay
        target = dest / rel
        digest = _file_digest(src)
        new_files[rel] = digest
        if target.is_file():
            if _same_content(src, target):
                result.lines.append("skip (exists): %s" % rel)
                result.unchanged += 1
                continue
            if local(rel, target):
                backup(rel, target)
                result.replaced.append(rel)
            result.lines.append("update: %s" % rel)
        else:
            result.lines.append("install: %s" % rel)
        write(src, target)
        result.written += 1

    # The manifest is dotagents' own metadata: always the source's.
    if self.manifest_path.is_file():
        target = dest / self.MANIFEST_NAME
        if target.is_file() and _same_content(self.manifest_path, target):
            result.unchanged += 1
        else:
            result.lines.append(
                "%s: %s" % ("update" if target.is_file() else "install", self.MANIFEST_NAME)
            )
            write(self.manifest_path, target)
            result.written += 1

    # Everything else in the install: removed, unless it is the install's
    # own (ignored, a cache) and `prune` was not asked for. An own
    # directory is never walked into (a `.venv` can be large): skipped
    # whole, or with `prune` removed whole.
    if dest.is_dir():
        keep = {self.MANIFEST_NAME, self.INSTALL_RECORD}

        def join(base: str, name: str) -> str:
            return "%s/%s" % (base, name) if base else name

        def drop(rel: str, target: Path, own: bool) -> None:
            if not own and local(rel, target):
                backup(rel, target)
            result.lines.append("remove (%s): %s" % ("pruned" if own else "not in the source", rel))
            result.removed.append(rel)
            if not dry_run:
                target.unlink()

        for dirpath, dirnames, filenames in os.walk(str(dest)):
            base = Path(dirpath).relative_to(dest).as_posix()
            base = "" if base == "." else base
            descend = []
            for name in sorted(dirnames):
                rel = join(base, name)
                target = Path(dirpath) / name
                if target.is_symlink():
                    filenames.append(name)  # a link is an entry, not a tree
                    continue
                own = name in self.SKIP_PARTS or rules.ignored(rel, True)
                if not own:
                    descend.append(name)
                elif prune:
                    result.lines.append("remove (pruned): %s/" % rel)
                    result.removed.append(rel + "/")
                    if not dry_run:
                        shutil.rmtree(str(target))
            dirnames[:] = descend
            for name in sorted(filenames):
                rel = join(base, name)
                if rel in keep or rel in new_files:
                    continue
                own = name.endswith(".pyc") or rules.ignored(rel)
                if own and not prune:
                    continue
                drop(rel, Path(dirpath) / name, own)
        if not dry_run:
            for dirpath, dirnames, filenames in os.walk(str(dest), topdown=False):
                here = Path(dirpath)
                if here != dest and not any(here.iterdir()):
                    rel = here.relative_to(dest).as_posix()
                    if prune or not rules.ignored(rel, True):
                        here.rmdir()

    if not dry_run:
        dest.mkdir(parents=True, exist_ok=True)
        installed._write_install_record({
            "source": source if source is not None else record.get("source"),
            "files": new_files,
        })
    return result

installed(*stores) classmethod

Every overlay a session uses, across stores (.agents-shaped roots) given in precedence order -- Scope.stores: the system store, the user store, then the project's <project>/.agents (a user scope has no project store). A None store is skipped.

The overlays come back store by store, each store's sorted by name, with every result stamped with the :attr:store it came from. Shadowing: an overlay whose name (compared normalized, so my_overlay and my-overlay are one) also appears in a LATER store is dropped -- the later store's copy REPLACES it, so the project's bin/lib/ env/cmds/CONTEXT.md/root var are the only ones that resolve, not both stacked. This is the one function behind the contract-A walk (Scope.paths), env's overlay roots, context's sources/placeholders/skills and overlays list/show, so they can never disagree about what is installed.

Source code in src/dotagents/_overlays.py

@classmethod
def installed(cls, *stores: "str | os.PathLike[str] | None") -> "list[Overlay]":
    """Every overlay a session uses, across ``stores`` (``.agents``-shaped
    roots) given in precedence order -- ``Scope.stores``: the system store,
    the user store, then the project's ``<project>/.agents`` (a user scope
    has no project store). A ``None`` store is skipped.

    The overlays come back store by store, each store's sorted by name,
    with every result stamped with the :attr:`store` it came from.
    **Shadowing:** an overlay whose name (compared normalized, so
    ``my_overlay`` and ``my-overlay`` are one) also appears in a LATER store
    is dropped -- the later store's copy REPLACES it, so the project's bin/lib/
    env/cmds/CONTEXT.md/root var are the only ones that resolve, not both
    stacked. This is the one function behind the contract-A walk
    (``Scope.paths``), ``env``'s overlay roots, ``context``'s
    sources/placeholders/skills and ``overlays list``/``show``, so they can
    never disagree about what is installed."""
    per_store = [
        cls.discover(Path(store) / "overlays", store)
        for store in stores
        if store is not None
    ]
    result: "list[Overlay]" = []
    for i, found in enumerate(per_store):
        later = {o.normalized_name for rest in per_store[i + 1 :] for o in rest}
        result.extend(o for o in found if o.normalized_name not in later)
    return result

is_valid_name(name) staticmethod

A dir under overlays/ is an overlay iff its name matches :data:NAME_RE: a leading ASCII letter, then ASCII letters/digits/_/ ./-. Dots are allowed mid-name (foo.bar, v1.2) but NOT as the first char, so .git/.hidden and __pycache__ (leading _) and 2fast (leading digit) are excluded. The contract-A level names (user, project, system, project-root, default, overlay -- :data:dotagents._scope.LEVEL_NAMES) are reserved: an overlay's dir name is its level label in the walk, and one of these would collide with the per-level filename keys. So are Windows device names (:data:RESERVED_NAMES, with or without an extension).

Source code in src/dotagents/_overlays.py

@staticmethod
def is_valid_name(name: str) -> bool:
    """A dir under ``overlays/`` is an overlay iff its name matches
    :data:`NAME_RE`: a leading ASCII letter, then ASCII letters/digits/``_``/
    ``.``/``-``. Dots are allowed mid-name (``foo.bar``, ``v1.2``) but NOT as
    the first char, so ``.git``/``.hidden`` and ``__pycache__`` (leading ``_``)
    and ``2fast`` (leading digit) are excluded. The contract-A level names
    (``user``, ``project``, ``system``, ``project-root``, ``default``,
    ``overlay`` -- :data:`dotagents._scope.LEVEL_NAMES`) are reserved: an
    overlay's dir name is its level label in the walk, and one of these
    would collide with the per-level filename keys. So are Windows device
    names (:data:`RESERVED_NAMES`, with or without an extension)."""
    from dotagents._scope import LEVEL_NAMES

    return (
        bool(Overlay.NAME_RE.match(name))
        and name.lower() not in LEVEL_NAMES
        and name.split(".", 1)[0].lower() not in Overlay.RESERVED_NAMES
    )

normalize_name(name) staticmethod

THE canonical overlay name: lowercase, _ and . -> -.

Every place that refers to an overlay by name goes through this one function, so My_Overlay and my-overlay are one overlay everywhere: overlays add installs to <scope>/overlays/<normalize_name(name)>/ and resolves the source dir by the same name, shadowing across stores compares it, and :meth:root_var_for derives the <NAME>_OVERLAY_ROOT env var from it. A dot is legal in a dir name (:meth:is_valid_name) but no shell variable can carry it, so v1.2 and v1-2 would share V1_2_OVERLAY_ROOT; normalizing it to - makes them one overlay rather than two that alias.

Source code in src/dotagents/_overlays.py

@staticmethod
def normalize_name(name: str) -> str:
    """THE canonical overlay name: lowercase, ``_`` and ``.`` -> ``-``.

    Every place that refers to an overlay by name goes through this one
    function, so ``My_Overlay`` and ``my-overlay`` are one overlay everywhere:
    ``overlays add`` installs to ``<scope>/overlays/<normalize_name(name)>/``
    and resolves the source dir by the same name, shadowing across stores
    compares it, and :meth:`root_var_for` derives the ``<NAME>_OVERLAY_ROOT``
    env var from it. A dot is legal in a dir name (:meth:`is_valid_name`)
    but no shell variable can carry it, so ``v1.2`` and ``v1-2`` would share
    ``V1_2_OVERLAY_ROOT``; normalizing it to ``-`` makes them one overlay
    rather than two that alias."""
    return name.lower().replace("_", "-").replace(".", "-")

read_install_record()

{"source": {...} | None, "files": {rel: sha256}} of an installed overlay; empty for a source dir or an install that predates records.

Source code in src/dotagents/_overlays.py

def read_install_record(self) -> "dict[str, Any]":
    """``{"source": {...} | None, "files": {rel: sha256}}`` of an installed
    overlay; empty for a source dir or an install that predates records."""
    import json

    try:
        data = json.loads(self.install_record_path.read_text(encoding="utf-8"))
    except (OSError, ValueError):
        return {}
    if not isinstance(data, dict):
        return {}
    files = data.get("files")
    data["files"] = {
        str(k): str(v) for k, v in files.items() if isinstance(v, str)
    } if isinstance(files, dict) else {}
    return data

read_manifest()

Parse overlay.toml, returning name / description / routing / rules / requires / priority / setup_timeout (seconds, 0 = none; None when the manifest does not set it) / scripts ([scripts] as written: launcher name -> script path relative to the overlay, or False; :meth:scripts adds the detected ones).

A missing or unreadable manifest yields empty contributions -- an overlay is allowed to be just a directory of files.

Source code in src/dotagents/_overlays.py

def read_manifest(self) -> "dict[str, object]":
    """Parse `overlay.toml`, returning name / description / routing / rules /
    requires / priority / setup_timeout (seconds, ``0`` = none; ``None``
    when the manifest does not set it) / scripts (``[scripts]`` as
    written: launcher name -> script path relative to the overlay, or
    ``False``; :meth:`scripts` adds the detected ones).

    A missing or unreadable manifest yields empty contributions -- an overlay
    is allowed to be just a directory of files."""
    empty: "dict[str, object]" = {
        "name": self.name, "description": "", "routing": [], "rules": [],
        "requires": [], "priority": DEFAULT_PRIORITY, "setup_timeout": None, "scripts": {},
    }
    path = self.manifest_path
    if not path.is_file():
        return empty
    try:
        text = path.read_text(encoding="utf-8")
    except (OSError, UnicodeDecodeError) as exc:
        _warn_once("cannot read %s (%s); ignoring it", path, exc)
        return empty
    doc = parse_manifest_text(text, str(path))
    if doc is None:
        return empty
    name, description, priority = doc.get("name"), doc.get("description"), doc.get("priority")
    # `requires` names become install paths (`<scope>/overlays/<name>`):
    # only a valid overlay name may pass, so `../x` or a level name can
    # never escape `overlays/` or leave an orphan the commands cannot see.
    requires: "list[str]" = []
    for raw in _string_list(doc.get("requires")):
        if not self.is_valid_name(raw):
            _warn_once("overlay %s: ignoring invalid requires entry %r", self.name, raw)
            continue
        dep = self.normalize_name(raw)
        if dep not in requires:
            requires.append(dep)
    return {
        "name": name if isinstance(name, str) and name else self.name,
        "description": description if isinstance(description, str) else "",
        "routing": _string_list(doc.get("routing")),
        "rules": _string_list(doc.get("rules")),
        "requires": requires,
        "priority": priority if isinstance(priority, int) and not isinstance(priority, bool)
        else DEFAULT_PRIORITY,
        "setup_timeout": _setup_timeout(doc.get("setup_timeout"), self.name),
        "scripts": _scripts(doc.get("scripts"), self.name),
    }

root_var_for(name) staticmethod

The env-var name carrying an overlay's installed root: <NAME>_OVERLAY_ROOT.

NAME is :meth:normalize_name of the overlay upper-cased, with - (and a mid-name ., which no shell variable can carry) turned into _: my-overlay / My_Overlay -> MY_OVERLAY_OVERLAY_ROOT, v1.2 -> V1_2_OVERLAY_ROOT. Because it starts from the same normalized name the install dir uses, the variable for overlays/<n>/ is always root_var_for(n) (:attr:root_var on the instance). Both consumers go through here: dotagents env EMITS the variable and dotagents context expands the <NAME_OVERLAY_ROOT> placeholder to the same path.

Source code in src/dotagents/_overlays.py

@staticmethod
def root_var_for(name: str) -> str:
    """The env-var name carrying an overlay's installed root: ``<NAME>_OVERLAY_ROOT``.

    ``NAME`` is :meth:`normalize_name` of the overlay upper-cased, with ``-``
    (and a mid-name ``.``, which no shell variable can carry) turned into
    ``_``: ``my-overlay`` / ``My_Overlay`` -> ``MY_OVERLAY_OVERLAY_ROOT``,
    ``v1.2`` -> ``V1_2_OVERLAY_ROOT``. Because it starts from the same
    normalized name the install dir uses, the variable for ``overlays/<n>/``
    is always ``root_var_for(n)`` (:attr:`root_var` on the instance). Both
    consumers go through here: ``dotagents env`` EMITS the variable and
    ``dotagents context`` expands the ``<NAME_OVERLAY_ROOT>`` placeholder to
    the same path."""
    return re.sub(r"[^A-Z0-9]", "_", Overlay.normalize_name(name).upper()) + "_OVERLAY_ROOT"

rule_blocks(rel_paths)

Extract - **… bullet blocks from each referenced markdown file.

Returns (blocks, warnings). Only the leading run of bullets is taken: a rules file may carry explanatory prose under a ## heading (the engineering overlay's rules/ENGINEERING.md documents why its rules are not in the base), and that must not be merged into the core. A path that does not exist is reported, not fatal -- a broken optional overlay must never block the base config landing.

Source code in src/dotagents/_overlays.py

def rule_blocks(self, rel_paths: "list[str]") -> "tuple[list[str], list[str]]":
    """Extract `- **…` bullet blocks from each referenced markdown file.

    Returns (blocks, warnings). Only the leading run of bullets is taken: a
    rules file may carry explanatory prose under a `## ` heading
    (the engineering overlay's rules/ENGINEERING.md documents *why* its rules are not in the base), and
    that must not be merged into the core. A path that does not exist is
    reported, not fatal -- a broken optional overlay must never block the
    base config landing."""
    blocks: "list[str]" = []
    warnings: "list[str]" = []
    try:
        root = self.path.resolve()
    except OSError:
        root = self.path
    for rel in rel_paths:
        src = self.path / rel
        # Contained: an absolute path or one that resolves outside the
        # overlay would merge bullets from any file into AGENTS.md.
        try:
            inside = not os.path.isabs(rel) and not rel.startswith(("/", "\\")) and (
                src.resolve() == root or root in src.resolve().parents
            )
        except OSError:
            inside = False
        if not inside:
            warnings.append("rules file outside the overlay: %s" % rel)
            continue
        if not src.is_file():
            warnings.append("rules file not found: %s" % rel)
            continue
        try:
            text = src.read_text(encoding="utf-8")
        except (OSError, UnicodeDecodeError) as exc:
            warnings.append("rules file unreadable: %s (%s)" % (rel, exc))
            continue
        # Everything from the first bullet up to the first `## ` heading after it.
        start = re.search(r"(?m)^- \*\*", text)
        if start is None:
            warnings.append("no rules found in: %s" % rel)
            continue
        body = text[start.start():]
        end = re.search(r"(?m)^## ", body)
        if end is not None:
            body = body[: end.start()]
        blocks.append(body.rstrip())
    return blocks, warnings

run_setup(*, agents_dir, dry_run, logger, scope_root=None, scope_level=None, base_env=None, timeout=None)

Run the overlay's idempotent setup script if it ships one.

Returns the script's exit code, or None when the overlay has no setup script (nothing to run -- not an error). Presence of setup.py at the overlay root is the opt-in; the runner never second-guesses a script the user chose to install.

Idempotency is the overlay author's contract: the script must be safe to run on every add/sync (check-then-act). The runner only invokes it:

• cwd = the installed overlay dir, so the script sees its own files.
• env is base_env (default os.environ; overlays passes the assembled dotagents env, so the script imports any overlay's lib and calls any overlay's bin by name), plus AGENTS_HOME = agents_dir, the USER store -- what the variable means everywhere else, so a dotagents call the script makes resolves the same stores -- plus AGENTS_SCOPE_ROOT = the store the overlay is installed into (scope_root; the user store for a -g install, the project's .agents otherwise), AGENTS_SCOPE = user / project (scope_level, when given) and AGENTS_OVERLAY_DIR = its own installed dir.
• the script runs under AGENTS_PYTHON from that env (the scope venv, which has every overlay's requirements and libs), else the interpreter running dotagents.
• it is stopped after timeout seconds -- else the manifest's setup_timeout, else :data:DEFAULT_SETUP_TIMEOUT; 0 means no limit -- and a stopped script is a failure (exit 124).

A non-zero exit is returned so the caller can raise a clear error -- never a silent skip.

Source code in src/dotagents/_overlays.py

def run_setup(
    self, *, agents_dir: Path, dry_run: bool, logger,
    scope_root: "Optional[Path]" = None, scope_level: "Optional[str]" = None,
    base_env: "Optional[dict[str, str]]" = None, timeout: "Optional[int]" = None,
) -> "Optional[int]":
    """Run the overlay's idempotent ``setup`` script if it ships one.

    Returns the script's exit code, or ``None`` when the overlay has no setup
    script (nothing to run -- not an error). Presence of ``setup.py`` at the
    overlay root is the opt-in; the runner never second-guesses a script the
    user chose to install.

    **Idempotency is the overlay author's contract**: the script must be safe
    to run on every ``add``/``sync`` (check-then-act). The runner only
    invokes it:

    * **cwd** = the installed overlay dir, so the script sees its own files.
    * **env** is ``base_env`` (default ``os.environ``; ``overlays`` passes the
      assembled ``dotagents env``, so the script imports any overlay's
      ``lib`` and calls any overlay's ``bin`` by name), plus
      ``AGENTS_HOME`` = ``agents_dir``, the USER store
      -- what the variable means everywhere else, so a ``dotagents``
      call the script makes resolves the same stores -- plus
      ``AGENTS_SCOPE_ROOT`` = the store the overlay is installed into
      (``scope_root``; the user store for a ``-g`` install, the project's
      ``.agents`` otherwise), ``AGENTS_SCOPE`` = ``user`` / ``project``
      (``scope_level``, when given) and ``AGENTS_OVERLAY_DIR`` = its own
      installed dir.
    * the script runs under ``AGENTS_PYTHON`` from that env (the scope
      venv, which has every overlay's requirements and libs), else the
      interpreter running dotagents.
    * it is stopped after ``timeout`` seconds -- else the manifest's
      ``setup_timeout``, else :data:`DEFAULT_SETUP_TIMEOUT`; ``0`` means no
      limit -- and a stopped script is a failure (exit 124).

    A non-zero exit is returned so the caller can raise a clear error --
    never a silent skip."""
    script = self.find_setup_script()
    if script is None:
        return None

    logger.info("running setup for %s (%s)", self.name, script.name)
    if dry_run:
        logger.info("[dry-run] would run %s in %s", script.name, self.path)
        return 0

    env = dict(os.environ if base_env is None else base_env)
    env["AGENTS_HOME"] = str(agents_dir)
    env["AGENTS_SCOPE_ROOT"] = str(scope_root if scope_root is not None else agents_dir)
    if scope_level:
        env["AGENTS_SCOPE"] = scope_level
    env["AGENTS_OVERLAY_DIR"] = str(self.path)

    from dotagents._env import interpreter

    # AGENTS_PYTHON when the env names one (the scope venv: overlay libs
    # and requirements importable), else the interpreter running dotagents.
    cmd = [interpreter(env), str(script)]

    try:
        res = subprocess.run(cmd, cwd=str(self.path), env=env,
                             timeout=self.setup_timeout(timeout) or None)
    except subprocess.TimeoutExpired:
        logger.error(
            "setup for %s did not finish in %ds and was stopped (raise it with "
            "setup_timeout in overlay.toml or --setup-timeout; 0 = no limit)",
            self.name, self.setup_timeout(timeout),
        )
        return 124
    except OSError as exc:
        logger.error("setup for %s failed to start: %s", self.name, exc)
        return 1
    if res.returncode != 0:
        logger.error("setup for %s exited %d", self.name, res.returncode)
    return res.returncode

scripts()

The scripts that get a launcher (:mod:dotagents._launchers): each top-level bin/<name>.py whose stem is a valid script name, then [scripts] from the manifest, which adds or replaces a path by name, or with false drops a detected one.

Source code in src/dotagents/_overlays.py

def scripts(self) -> "dict[str, str]":
    """The scripts that get a launcher (:mod:`dotagents._launchers`):
    each top-level ``bin/<name>.py`` whose stem is a valid script name,
    then ``[scripts]`` from the manifest, which adds or replaces a path
    by name, or with ``false`` drops a detected one."""
    found: "dict[str, str]" = {}
    bin_dir = self.path / "bin"
    if bin_dir.is_dir():
        for entry in sorted(bin_dir.iterdir()):
            stem = entry.name[:-3]
            if entry.name.endswith(".py") and entry.is_file() and SCRIPT_NAME_RE.match(stem) \
                    and stem.split(".")[0].lower() not in self.RESERVED_NAMES:
                found[stem] = "bin/" + entry.name
    for name, path in self.read_manifest()["scripts"].items():  # type: ignore[union-attr]
        if path is False:
            found.pop(name, None)
        else:
            found[name] = path  # type: ignore[assignment]
    return found

setup_timeout(override=None)

Seconds the setup script may run: override (--setup-timeout), else the manifest's setup_timeout, else :data:DEFAULT_SETUP_TIMEOUT. 0 means no limit.

Source code in src/dotagents/_overlays.py

def setup_timeout(self, override: "Optional[int]" = None) -> int:
    """Seconds the setup script may run: ``override`` (``--setup-timeout``),
    else the manifest's ``setup_timeout``, else
    :data:`DEFAULT_SETUP_TIMEOUT`. ``0`` means no limit."""
    if override is not None:
        return max(0, override)
    manifest = self.read_manifest()["setup_timeout"]
    return DEFAULT_SETUP_TIMEOUT if manifest is None else manifest  # type: ignore[return-value]

sort_by_priority(overlays) classmethod

overlays (instances or dirs) sorted by (priority, name).

Deterministic regardless of the caller's order (add passes invocation order, sync discovery order): a high-priority overlay's lines must always land last. See :attr:sort_key.

Source code in src/dotagents/_overlays.py

@classmethod
def sort_by_priority(
    cls, overlays: "Iterable[Overlay | str | os.PathLike[str]]"
) -> "list[Overlay]":
    """`overlays` (instances or dirs) sorted by `(priority, name)`.

    Deterministic regardless of the caller's order (`add` passes invocation
    order, `sync` discovery order): a high-priority overlay's lines must
    always land last. See :attr:`sort_key`."""
    return sorted((cls(o) for o in overlays), key=lambda o: o.sort_key)

parse_manifest_text(text, origin='overlay.toml')

The manifest document: tomllib/tomli when importable, else the hand reader. None (and a warning naming origin) when it is not valid TOML.

Source code in src/dotagents/_overlays.py

def parse_manifest_text(text: str, origin: str = "overlay.toml") -> "Optional[dict[str, object]]":
    """The manifest document: ``tomllib``/``tomli`` when importable, else the
    hand reader. ``None`` (and a warning naming ``origin``) when it is not
    valid TOML."""
    toml = _toml_module()
    if toml is None:
        return _hand_parse(text)
    try:
        return toml.loads(text)
    except ValueError as exc:  # TOMLDecodeError is a ValueError
        _warn_once("%s is not valid TOML (%s); ignoring it", origin, exc)
        return None

recompose_overlay_block(agents_md, base_block, overlays, dry_run, logger)

Rebuild AGENTS.md's managed block from the pristine base over ALL installed overlays in (priority, name) order, in place.

base_block is the base overlay's managed block (no overlay content), overlays is every installed overlay in the scope (instances or dirs), and _compose_block folds them in sorted order -- so the block is a pure function of which overlays are installed, never when each was added (an incremental append could only put a newcomer last).

Only the managed block (between the dotagents markers) is rewritten; content outside the markers is untouched. Returns True if the file changed, False on a no-op (nothing to merge, file/markers absent, or block already correct).

Source code in src/dotagents/_overlays.py

def recompose_overlay_block(
    agents_md: Path,
    base_block: str,
    overlays: "Iterable[Overlay | str | os.PathLike[str]]",
    dry_run: bool,
    logger,
) -> bool:
    """Rebuild AGENTS.md's managed block from the *pristine* base over ALL installed
    overlays in `(priority, name)` order, in place.

    `base_block` is the base overlay's managed block (no overlay content),
    `overlays` is every installed overlay in the scope (instances or dirs), and
    `_compose_block` folds them in sorted order -- so the block is a pure
    function of *which* overlays are installed, never *when* each was added
    (an incremental append could only put a newcomer last).

    Only the managed block (between the dotagents markers) is rewritten; content
    outside the markers is untouched. Returns True if the file changed, False on a
    no-op (nothing to merge, file/markers absent, or block already correct).
    """
    from dotagents._fs import write_text_lf
    from dotagents._merge import _extract_block, find_block

    if not agents_md.is_file():
        # No `init` here yet (an `overlays add` into a fresh project): create
        # the block rather than drop the overlays' rules and routing.
        if not dry_run:
            write_text_lf(agents_md, _compose_block(_extract_block(base_block), list(overlays), logger) + "\n")
        logger.info("%s %s with the managed block", "would create" if dry_run else "created", agents_md)
        return True
    existing = agents_md.read_text(encoding="utf-8")
    span = find_block(existing)
    if span is None:
        logger.warning(
            "AGENTS.md has no dotagents managed block; overlay rules/routing not "
            "merged (run `dotagents init` first)"
        )
        return False

    # Compose over the pristine base block, not the current (already-merged) one, so
    # every install of the same overlay set yields identical output.
    pristine = _extract_block(base_block)
    merged = _compose_block(pristine, list(overlays), logger)

    start, end = span
    if existing[start:end] == merged:
        return False
    new_text = existing[:start] + merged + existing[end:]
    if not dry_run:
        write_text_lf(agents_md, new_text)
    return True