Configuration layers

Beyond CLI arguments, duho can pull defaults from environment variables and a config file in JSON, TOML, YAML or INI. The full precedence ladder, highest first:

CLI args  >  instance values  >  env var  >  config file  >  class default

A value supplied by any layer also un-requires that field: a field with no class default that's set in the config file no longer has to be passed on the command line.

Environment variables

Annotate a field with Meta(env="VAR_NAME"):

from duho import Args, Arg, Meta

class Deploy(Args):
    """Deploy the app."""

    token: Arg[str, Meta(env="DEPLOY_TOKEN")] = ""
    "Auth token"

$ export DEPLOY_TOKEN=abc123
$ deploy                      # token == "abc123"
$ deploy --token override     # token == "override"   (CLI wins)

The env value is converted with the field's own type, so Meta(env="PORT") on an int field yields an int — and a bad value produces the same clear error argparse would give. A bool field reads 1, true, yes, on, y, t as True and 0, false, no, off, n, f or the empty string as False, case-insensitively; duho.parse_bool(text) applies the same rule to any string (duho.text.BOOL_TRUE and BOOL_FALSE are the token tables) and raises ValueError for anything else.

One prefix for every field: `_env_prefix_`

Set _env_prefix_ on a class and each of its fields without a Meta(env=...) reads the variable PREFIX + FIELD_NAME.upper(), with the layering, conversion and (from env X) help note Meta(env=...) gives:

class Serve(duho.Cmd):
    _env_prefix_ = "MYAPP_"

    port: int = 8080          # MYAPP_PORT
    host: str = "127.0.0.1"   # MYAPP_HOST
    token: Arg[str, Meta(env="API_TOKEN")] = ""   # API_TOKEN: an explicit name wins

The prefix can be a function of the command class, so each command has its own (_env_prefix_ = staticmethod(lambda cls: "MYAPP_" + cls.__name__.upper() + "_")). Subclasses inherit it. The fields the presets add (-v, --config, --format, ...) never get a name.

Config files

Set _config_ on the class, or pass config= to duho.parse / duho.main (the keyword argument wins; duho.parse_globals and duho.app take it too). Every entry point layers env and the class's _config_ the same way: parse, main, parse_globals, app, and a parser from duho.parser(cls):

class Deploy(Args):
    _config_ = "~/.config/myapp/config.toml"

result = duho.parse(Deploy, config="./deploy.toml")

Top-level keys map to the root command's fields. A table named after a subcommand maps to that subcommand's fields:

# deploy.toml
token = "abc123"
verbose = true

[install]
target = "prod"

Unknown keys are ignored (with a debug log line), so a config file can carry settings for several versions of your tool without breaking older ones. A tool that would rather catch a misspelled key can say so; see Unknown keys.

A _config_ path that does not exist yet is skipped (with a debug log) instead of raising — a class attribute is allowed to point at a file an app hasn't written yet. A path you name yourself stays strict: an explicit config= argument, --config and the _config_env_ variable must exist, since you named the file directly.

Unknown keys

_config_validate_ on the root class decides what happens to a key in the file that nothing applies. A key counts as unknown at its own level: it is neither a field of that command nor a table named after one of its subcommands. It is reported as a dotted path, so targte under [install] is install.targte, and a table that names no command is reported as the table's name.

Value	Effect
`None` or `"ignore"`	the default: unknown keys are skipped
`"warn"`	one warning on the `duho.config` logger naming the file and the keys
`"strict"`	a one-line error (exit status 2, no usage lines, no traceback) naming the file and the keys
a function	called as `f(args, config, unknown)` or `f(args, config)` to check the whole file; see below

from duho import Cli

class Deploy(Cli):
    _config_ = "deploy.toml"
    _config_validate_ = "strict"

    port: int = 8000

$ deploy
deploy: error: duho: config file deploy.toml: unknown keys: 'prot', 'install.targte'

A function checks the whole file, not only its names. It is called once the command line is parsed, with the parsed instance of the selected command (what __call__ gets as self), the loaded mapping, and a tuple of the unknown paths (empty when there are none). A function that takes only two arguments, f(args, config), is called the same way without the unknown paths, which are then ignored as by default. Either form is called whether or not the file has unknown keys. A ValueError it raises with a message becomes the same one-line error; any other exception propagates.

from duho import Cli

def check(args, config, unknown):
    if unknown:
        raise ValueError("unknown keys " + ", ".join(unknown))
    if args.port < 1024:                 # the value after the file AND the command line
        raise ValueError("port must be 1024 or above")
    if "tls" in config and "cert" not in config["tls"]:
        raise ValueError("[tls] needs a cert")

class Deploy(Cli):
    _config_ = "deploy.toml"
    _config_validate_ = check

    port: int = 8000

The error and the warning name keys and the file, never a value from it. Nothing runs, a function included, unless a file was read: a _config_ that does not exist yet is skipped, and so is the check. --help and --version work whatever the file holds, and a function is not called for them or for a command line argparse refuses. Any other value for _config_validate_ is a ValueError naming the class. The attribute is read on the root class, and applies whichever way the file was chosen and whichever format or _config_loader_ read it. duho.parse_globals reads the file but does not run the check, because the commands duho.app registers afterwards are not known to it yet.

The `--config` flag: `ConfigArgs`

Most tools let the user name the file. ConfigArgs is a mixin that adds --config FILE (-c) and reads the file it names, with no other attribute to set:

from duho import Cli, ConfigArgs, LoggingArgs

class Deploy(ConfigArgs, LoggingArgs, Cli):
    """Deploy the app."""

    port: int = 8000
    "Port to listen on"

$ deploy -c prod.yaml          # port comes from prod.yaml
$ deploy -c prod.yaml --port 1 # the command line wins
$ deploy -c missing.yaml       # deploy: error: argument --config/-c: no such file: missing.yaml

The path has ~ expanded and must be a file, so a mistyped path is a usage error (exit status 2) with no traceback. The field named by _config_field_ is the file's own path, so it is never a settings key: a config key inside the file is not applied, and _config_validate_ counts it as unknown.

A tool that reads its --config file itself, with its own loader or its own idea of what the file holds, can keep the option and turn duho's reading off: set _config_field_ = None in the class body (class Deploy(ConfigArgs, LoggingArgs, Cli) with _config_field_ = None). The field is still parsed and checked to be a file, and the program reads it. List ConfigArgs before Cli: Cli declares _config_field_ = None, and class Deploy(Cli, ConfigArgs) is a TypeError naming the class. Because the class names a config source, --help lists the reversible --no-* form of its bool fields whether or not a file was given.

Choosing the config file at run time

_config_ fixes the path in the class. To let the user or the environment choose it, set _config_env_ and/or _config_field_ on the root:

from typing import Optional
from duho import Cmd

class Tool(Cmd):
    _config_ = "~/.config/tool/config.toml"
    _config_env_ = "TOOL_CONFIG"      # the path is read from $TOOL_CONFIG
    _config_field_ = "config"         # ...or from --config

    config: Optional[str] = None
    "Config file"

    def __call__(self):
        print(self.config)

Highest first:

an explicit config= argument to main/parse/parse_globals/app;
the _config_field_ field, when the user gave it on the command line or through its own env var (its class default does not count);
the _config_env_ variable, when it is set;
_config_.

An empty path means no file, wherever a path can be given: config="", the _config_field_ field given empty (--config "" or --config=, or its own env var set to the empty string) and a _config_env_ variable set to the empty string. Nothing is read then, _config_ included, and no _config_loader_ or _config_validate_ function runs. With ConfigArgs, --config "" leaves config as None. It is how a user runs a tool whose _config_ names a file with none of that file's values. An unset variable still falls through to _config_, and a non-empty path that does not exist is still an error.

A path chosen by the field or the variable is strict, like config=: a missing file raises FileNotFoundError, where a missing _config_ is skipped. A _config_field_ naming a field the class does not declare is a ValueError naming the class, raised before parsing. A tree served over MCP reads the file _config_ names but applies neither of these two attributes. It reads that file once, at the first call in the process, and keeps it: a later change to the file, or to the variable that locates it, is not seen until the process restarts, so an in-process test cannot vary the default file between calls.

An env var set to the empty string is treated as unset (falls through to config/class default) for every field except a bare str field, which keeps the empty string as its value. The variable that names the config file itself is the exception: empty there means no file, as above.

A default location in the user's config directory

duho.config.user_path(app, *names) is the path <config dir>/<app>/<names...>, where the config dir is APPDATA (when it is an absolute path, else ~/AppData/Roaming) on Windows and XDG_CONFIG_HOME (when it is an absolute path, else ~/.config) elsewhere. It resolves each time duho looks for the file, not when the class is defined, so it is safe as a class attribute:

class Tool(duho.Cli):
    _config_ = duho.config.user_path("tool", "config.toml")

str(path) is the resolved path and repr shows the call. An empty app is a ValueError. Like any _config_, the file is skipped when it does not exist.

Formats

The file's suffix picks the reader, from the duho.config backends:

Format	Suffixes	Reads with	Writes with	Needs
JSON	`.json`	`json`	`json`	nothing
TOML	`.toml`	`tomllib` (3.11+), else `tomli`	`tomli_w`	`pip install duho[config]` on 3.9 and 3.10, and to write
YAML	`.yaml`, `.yml`	`yaml.safe_load`	`yaml.safe_dump`	`pip install duho[yaml]`
INI	`.ini`, `.cfg`	`configparser`	`configparser`	nothing

A name no backend claims (app.conf, or no suffix at all) is read as TOML. Every format gives the same nested shape: top-level keys map to the root command's fields and a table (object, mapping, section) named for a subcommand maps to that subcommand's fields.

# deploy.yaml
token: abc123
verbose: true
install:
  target: prod

An INI file is plain text: the keys of [DEFAULT] are the top level, every other section is a table, sections do not inherit [DEFAULT], key case is kept, % is not special, and every value is a string that duho converts with the field's type.

; deploy.ini
[DEFAULT]
token = abc123
verbose = true

[install]
target = prod

duho stays zero-dependency by default: a parser library is imported only when a file of its format is read. If it is missing, reading that file is a one-line error naming the extra (duho[yaml], duho[config]), exit status 2; --help and --version still work.

Malformed config files

A malformed file, or one whose top level is not a table, is one line, PROG: error: MESSAGE, naming the file and the format, with exit status 2 and no usage lines under duho.main, duho.parse and duho.app alike, not a raised exception. A value in the file that fails to convert for its field is still an argparse usage error. The message carries the parser's problem and position and never a value from the document, so a secret in a bad file is not echoed. It can name a key, a section, or a YAML alias or tag, cut to 120 characters:

app: error: duho: invalid JSON in config file /etc/app.json: Expecting property name ...
app: error: duho: invalid YAML in config file /etc/app.yaml: found character '\t' ... (at line 2, column 1)
app: error: duho: config file /etc/app.json must contain a table/object at the top level, got list

Reading and writing files yourself: `duho.config`

The backends are public. load and dump pick a backend by the file name, or by a format name you pass; loads and dumps work on text:

from duho import config

config.dump({"port": 8000, "install": {"target": "prod"}}, "deploy.json")
settings = config.load("deploy.json")           # {"port": 8000, "install": {...}}

text = config.dumps(settings, "ini")            # "[DEFAULT]\nport = 8000\n..."
same = config.loads(text, "ini")                # every INI value is a string

dump builds the whole document before it touches the file, writes UTF-8, and leaves an existing file as it was when the data cannot be written (an INI file cannot hold a list or a table nested two deep). config.backend_names() lists the formats and config.backend_for(path) returns the backend a file name selects. Failures are duho.config.ConfigError (a ValueError with path, lineno and colno), UnsupportedFormatError and ConfigDependencyError (with .extra).

Adding a format

Subclass ConfigBackend, set name, and implement loads and dumps. Defining the class registers it, so config.load("a.hocon"), and a _config_ path ending .hocon, use it:

from duho import config

class KeyValueBackend(config.ConfigBackend):
    name = "kv"
    aliases = ("keyvalue",)
    suffixes = (".kv",)

    def loads(self, text):
        try:
            return dict(line.split("=", 1) for line in text.splitlines() if line)
        except ValueError:
            raise config.ConfigError("a line without '='") from None

    def dumps(self, data):
        return "".join(f"{key}={value}\n" for key, value in data.items())

assert config.loads("a=1\nb=2\n", "kv") == {"a": "1", "b": "2"}

A name, alias or suffix a registered backend already holds is a ValueError. To take over a built-in format, pass replace=True in the class statement (class MyYaml(config.ConfigBackend, replace=True) with name = "yaml"). A subclass that sets no name of its own, such as class Loud(config.JSONBackend), is not registered; use it in a set, below.

Choosing which formats a tool reads

By default a tool reads every registered format. Set _config_backends_ on the root to limit it to the ones you list, as registered names, backend classes or instances. A file whose name no listed suffix matches is then a one-line error that lists the accepted suffixes:

from duho import Cli, ConfigArgs

class Deploy(ConfigArgs, Cli):
    _config_backends_ = ["toml", "json"]   # no YAML, no INI

Changing a backend's names and suffixes

An instance may carry other name, aliases and suffixes than its class, which is how a tool reads .conf files as INI, or a suffix-less file as YAML. The suffix "" matches any file name, and always loses to a longer suffix:

from duho import Cli, ConfigArgs, config

class Deploy(ConfigArgs, Cli):
    _config_backends_ = [
        "json",
        config.INIBackend(suffixes=(".conf", ".ini")),
        config.YAMLBackend(suffixes=(".yaml", "")),
    ]

A later item in the list wins a name or suffix an earlier one also claims, so a subclass placed last replaces a built-in for this tool only, leaving the registry alone:

class StrictJSON(config.JSONBackend):
    def loads(self, text):
        data = super().loads(text)
        if not isinstance(data, dict):
            raise config.ConfigError("the top level must be an object")
        return data

class Deploy(ConfigArgs, Cli):
    _config_backends_ = ["json", "toml", StrictJSON]   # StrictJSON reads .json

The same backends= keyword is accepted by config.load, loads, dump, dumps, backend_for, get_backend and backend_names.

A format with no backend: `_config_loader_`

For a one-off, set a class-level _config_loader_: a Callable[[Path], dict] that duho calls instead of the backends. It receives the expanded Path and returns the config dict; the layering, precedence and subcommand-table rules are the same.

class Deploy(duho.Cli):
    _config_ = "./deploy.hjson"
    _config_loader_ = staticmethod(lambda path: my_hjson.loads(path.read_text()))

A loader that takes two positional arguments is also told where its path came from, as a second argument: "argument" (config=), "field" (the _config_field_ field), "env" (the _config_env_ variable) or "default" (_config_, also for a tree served over MCP). A program can then check a file it found on its own differently from one the user named. A loader of one argument is called as before.

def load(path, source):
    try:
        return my_hjson.loads(path.read_text())
    except ValueError:
        if source == "default":
            return {}  # a stale default file is ignored; a named one is an error
        raise

class Deploy(duho.Cli):
    _config_ = "./deploy.hjson"
    _config_loader_ = staticmethod(load)

An error raised inside the loader that is a duho.CommandError or matches the root's _errors_ is reported like a command's error (the _report_error_ hook or the <prog>: error: <message> line, with its status), once the parse reaches it, so --help still works; any other exception propagates to the caller unchanged. Only the backends, and the check that the loaded value is a mapping, report through the parser: a loader that returns something that is not a mapping is a one-line error naming the file. For anything you would use twice, a ConfigBackend subclass is the better home.

When the root has a _report_error_, the one-line error for a malformed or unreadable file is given to it, as a duho.exceptions.ConfigError with status 2, instead of being printed.

Where did this value come from?

duho.value_sources(parsed) reports which layer won for each field of a parsed instance — what duho.parse returns, or self inside a command's __call__:

result = duho.parse(Deploy, [], config="./deploy.toml")

duho.value_sources(result)
# {"token": "env", "verbose": "config", "target": "default"}

Each value is one of "cli", "instance" (a field that came from an instance passed to duho.parse), "env", "config", or "default". This is the fastest way to answer "why is this setting not what I expect".

Which config file is in force

duho.config_source(parsed) returns the file the parse read and where its path came from, for the root or any subcommand instance, or None when no file was read:

source = duho.config_source(self)
# ConfigSource(path=PosixPath('/etc/deploy.toml'), source='default')

source is "argument" (config=), "field" (the _config_field_ field), "env" (_config_env_) or "default" (_config_), the vocabulary a two-argument _config_loader_ receives. It is None for an empty path, a default file that does not exist, and an instance built directly rather than parsed.