Connecting
Every target is described by a config: the transport's settings, without a live connection. A config is built in one of two ways, and both give the same object:
- by class,
SSHConfig("nas.example", username="admin"): the target is the one positional argument, everything else is a keyword (a second positional argument raisesTypeError); - by URI,
HostConfig("ssh://admin@nas.example"), which picks the config class from the scheme.Host(uri)does the same and returns the host.
config.open() creates the host, and with config as host: connects and
always closes. Nothing connects before that.
Credentials
Secrets never live in the URI a config renders. They reach a config three ways:
- as keyword arguments to the class, or as
**secretstoHostConfig(uri, **secrets)/Host(uri, **secrets); - as the password in a URI's userinfo,
ssh://admin:secret@nas.example. It is accepted on the way in, moved into the credentials, and never rendered back out:str(config)andconfig.connection_uricarry no password or key. A keyword password replaces it, andusername=replaces the URI's user, so one URI serves several logins:HostConfig("ssh://admin:old@nas", username="deploy", password="new").password=""means no password; - as extra lines of the password field, split by
parse_credentials: the first line is the password and each latername:valueline is passed as the credential keywordname, so a second factor travels in the same field.
Each config names the credentials it accepts (uri_credentials); any other
keyword raises ValueError naming it, so a typo such as passwrd= never
builds a config without a password. username= is one on SSH, WinRM and
qga+ssh.
On SSH and WinRM password may also be a callable returning the password,
called only when one is needed: SSH when the server asks after key
authentication failed, WinRM when the chosen leg authenticates with a
password. That is how a program asks its user only when it has to.
from hostctl import HostConfig, SSHConfig, parse_credentials
config = HostConfig("ssh://admin:secret@nas.example")
assert isinstance(config, SSHConfig)
assert "secret" not in str(config) and "secret" not in repr(config)
assert parse_credentials("hunter2\notp:123456") == ("hunter2", {"otp": "123456"})
try:
HostConfig("ssh://admin@nas.example", passwrd="secret")
except ValueError as error:
print(error) # unknown credential argument: passwrd
Percent-encode /, ? and # in a password written into a URI; @ and :
parse as written. redact_uri(uri) strips the password from a URI for a log.
Timeouts
Every config takes timeout=, spelled the same way everywhere. A number or
None is the total: the default deadline of run(timeout=), None
meaning no limit. A Timeout names the other fields too:
from hostctl import DEFAULT, HostConfig, SSHConfig, Timeout
config = SSHConfig("nas.example", timeout=Timeout(600, connect=10))
assert config.timeout == Timeout(total=600, connect=10)
assert SSHConfig("nas.example", timeout=600).timeout == Timeout(total=600)
same = HostConfig("ssh://nas.example?timeout=600&timeout_connect=10")
assert same.timeout == config.timeout
# What each DEFAULT became: SSH connects within 30 s unless told otherwise.
assert SSHConfig("nas.example").timeouts == Timeout(total=None, connect=30)
assert Timeout().read is DEFAULT
| field | bounds | URI key |
|---|---|---|
total |
a whole command; the default of run(timeout=) |
timeout |
connect |
establishing a connection, session or port | timeout_connect |
read |
one reply, or one read | timeout_read |
write |
one write | timeout_write |
Each field is DEFAULT (the transport's own), None (no limit) or a positive
number of seconds; a serial read or write may also be 0, pyserial's
"do not wait" (every other transport refuses a zero). In a URI an empty value is None and an absent key is
DEFAULT, and str(config) writes only the fields that are not DEFAULT.
A transport supports only some fields, and setting one it cannot honour raises
ValueError naming the config and the field rather than being ignored:
| config | total |
connect |
read |
write |
|---|---|---|---|---|
| Local | none | -- | -- | -- |
| SSH | none | 30 s | -- | -- |
| WinRM | 30 s | 30 s | -- | -- |
| Container | -- | 60 s | -- | -- |
| QEMU guest | none | the read value (unix and ssh transports) |
10 s | -- |
| Serial | none | -- | 0.1 s | 10 s |
| System URIs | the provider's own | -- | -- | -- |
run(timeout=) overrides the total for one call, and run(timeout=None)
lifts it, WinRM's 30 s included. The per-transport sections below say what
each field bounds there; the
API header
has the full rules.
Local
| URI | local: (local:?timeout=60 with a total) |
| class | LocalConfig() |
| timeouts | total |
| extra | none |
from hostctl import LocalConfig
with LocalConfig() as host:
print(host.run(["python", "-c", "print('hello')"], encoding="utf-8").stdout)
SSH
| URI | ssh://[user@]host[:port][?query] |
| query keys | shell, pathname, executable, login_shell, verify, and the timeout keys |
| credentials | username, password, client_keys, known_hosts |
| keyword only | ssh_config (default USER_SSH_CONFIG), connect_options (never in a URI) |
| extra | ssh |
from hostctl import HostConfig, SSHConfig, Timeout
config = SSHConfig(
"nas.example",
port=2222,
username="admin",
client_keys="~/.ssh/id_ed25519",
known_hosts="~/.ssh/known_hosts",
timeout=Timeout(connect=10.0),
)
same = HostConfig(
"ssh://admin@nas.example:2222?timeout_connect=10",
client_keys="~/.ssh/id_ed25519",
known_hosts="~/.ssh/known_hosts",
)
assert str(same) == str(config)
- Account and port. With no
usernamethe ssh_config'sUserapplies, else the local account name; with noportitsPort, else 22, assshdoes without-l/-p. A value you pass wins.ssh://hnames no port;ssh://h:22names 22.default_username="root"takes the local account's place: the ssh_config'sUserstill wins over it, andusernameover both. - Host keys.
known_hosts=()(the default) verifies the server's key against the ssh_config'sUserKnownHostsFile, else~/.ssh/known_hosts, assshdoes; a path, bytes or a sequence of them names the trusted keys.verify=False(?verify=false) is the one way to skip the check: it replaces the usual lookup, while aknown_hostsyou name is still checked.known_hosts=Noneraises. The ssh_config is honoured as written,UserKnownHostsFile noneincluded. - One connection. Commands and paths share one connection and one login,
opened by whichever comes first (building an absolute path connects
nothing); paths open SFTP channels on it, so every setting here applies
once. After
close()the next use opens a new one, and a path taken beforeclose()keeps working on it. - ssh_config. The connection reads your
~/.ssh/config, assshdoes:ssh_config=USER_SSH_CONFIG, the default, finds it when the connection opens, and a missing file reads nothing.ssh_config=Nonereads none, for a service or a test that must not depend on a personal file. Apathlib.Pathnames another file. Astris the configuration's text, not a file name, and so are bytes, a stream such asio.StringIO, and any object withread_text()-- a path on another host included, read when the first connection opens. Text reaches asyncssh through a private temporary file that exists only while the connection opens;repr()shows<ssh_config text>, never the text. The ssh_config cannot come from a URI, because aProxyCommandstarts a local process. - Timeouts.
timeout.connect(default 30 s) bounds connecting only: TCP, key exchange and authentication;Noneleaves asyncssh's own. The total (no default) bounds a command.readandwriteraise. - Opening it returns a
PosixHost, or aWindowsHostwhenpathnameis Windows. For a Windows OpenSSH server passshell=POWERSHELL(orCMD) andpathname="windows";shell="auto"andpathname="auto"probe instead of guessing.
import pathlib
from hostctl import SSHConfig
SSHConfig("nas") # ~/.ssh/config, as ssh reads it
SSHConfig("nas", ssh_config=None) # no configuration at all
SSHConfig("nas", ssh_config=pathlib.Path("ci/ssh_config")) # a file
SSHConfig("nas", ssh_config="Host nas\n HostName 10.0.0.5\n") # text
WinRM
| URI | winrm://user@host[:port][?query] (HTTP), winrms://... (HTTPS) |
| query keys | auth, verify, message_encryption, provider, cleartext_auth, and the timeout keys |
| credentials | username, password |
| extra | winrm; psrp for runspaces (Python 3.10+); winrm-credssp, winrm-kerberos for those auth values |
from hostctl import HostConfig, WinRMConfig
config = WinRMConfig(
"win.example", username="admin", password="secret", tls=True, verify=False
)
same = HostConfig("winrms://admin@win.example?verify=false", password="secret")
assert str(same) == str(config)
assert "secret" not in str(config)
- TLS.
winrms://(ortls=True) is HTTPS. The certificate is checked against the operating system's trust on every leg: the Windows certificate store on the native leg, and on pywinrm and PSRP the platform's own verifier throughtruststoreon Python 3.10+ (on 3.9 the store as Python'ssslmodule loads it) -- or, whenREQUESTS_CA_BUNDLEorCURL_CA_BUNDLEis set, that bundle alone. certifi is not used.verify="ca.pem"(?verify=ca.pem, a file or a hashed directory) trusts that bundle instead, on pywinrm and PSRP; the native leg refuses one, since PowerShell remoting trusts only the store.verify=False(?verify=false) skips every certificate check. What the store holds differs by platform and Python version; see the contracts ledger. - Cleartext.
auth="basic","plaintext"or"ssl"over plain HTTP would send the password in cleartext, so it raisesValueErrorunlesscleartext_auth=True(?cleartext_auth=true). NTLM (the default), Kerberos and CredSSP encrypt the message. - No password. On a Windows client, a config with no password uses the
current Windows login (native PowerShell remoting);
usernamemust name that user. - Timeouts. Two fields, both 30 s by default. The total is the default
for
run(timeout=): the same total deadline per command, which a per-calltimeout=overrides;timeout=None(?timeout=) sets none, so a command that never finishes blocks the call, and on the native leg so does a host that stops answering once the session is open.timeout.connectbounds only establishing the connection, as on SSH, and a host that never accepts it raisesConnectionError.Timeout(connect=None)(?timeout_connect=) keeps each leg's library default: PowerShell's 180 s on the native leg, pypsrp's 30 s on PSRP, and on pywinrm its single 30 s HTTP timeout, which then bounds connecting too. How long one WS-Man request may wait is fixed and not configurable, soreadandwriteraise.
Containers
| URI | docker://container[?query] |
| query keys | engine_url, verify, username, workdir, executable, shell, pathname, timeout_connect |
| credentials | client_factory (a callable returning a Docker client), client_cert (a (certificate, key) pair of PEM paths) |
| extra | container |
from hostctl import ContainerConfig, HostConfig
config = ContainerConfig("application", username="app", workdir="/srv")
same = HostConfig("docker://application?username=app&workdir=/srv")
assert str(same) == str(config)
With no engine_url the Docker client comes from the environment
(docker.from_env(), which reads DOCKER_HOST and its TLS settings); with one
it is docker.DockerClient(base_url=engine_url). An https:// engine URL is
TLS: by default the certificate is checked against the operating system's
trust store, as on pywinrm (REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE instead
when set); verify="ca.pem" trusts that CA bundle instead, verify=False
checks nothing, and either holds even when REQUESTS_CA_BUNDLE is set. A
daemon run with --tlsverify also wants a client certificate:
client_cert=("cert.pem", "key.pem") (one PEM holding both is passed
twice) presents it, alongside whatever verify says. It is never shown in
repr() or the connection URI, and a missing file raises
FileNotFoundError before connecting. For anything else -- a pre-built
client -- pass client_factory= (and leave verify and client_cert
alone).
config = ContainerConfig(
"application",
engine_url="https://docker.example:2376",
verify="ca.pem",
client_cert=("cert.pem", "key.pem"),
)
timeout.connect is the client's request timeout (the SDK's 60 s by
default), passed as timeout= to the client or the factory only when set;
it bounds connecting to the Engine and each API reply. A container has no
total: an Engine exec cannot be cancelled, so timeout=<n> raises and so
does run(timeout=<n>).
QEMU guests
| URI, local libvirt | qemu+libvirt:///domain[?connection=LIBVIRT_URI] |
| URI, Unix socket | qga+unix:///SOCKET_PATH?domain=DOMAIN (the path percent-encoded) |
| URI, socket over SSH | qga+ssh://user@hypervisor/domain[?socket_path=PATH&verify=false] |
| query keys | max_reply_size, shell, pathname, the timeout keys, plus the ones above |
| credentials | transport_factory, serial_console, path_helper; on qga+ssh, also username, password, client_keys, known_hosts |
| extra | qemu-libvirt for libvirt; ssh for qga+ssh |
from hostctl import HostConfig, QEMUConfig, SSHConfig
local = HostConfig("qemu+libvirt:///vm-id?connection=qemu:///system")
socket = HostConfig("qga+unix:///%2Frun%2Fqga.sock?domain=vm-id")
assert socket.socket_path == "/run/qga.sock"
tunneled = QEMUConfig(
"vm-id",
transport="ssh",
ssh=SSHConfig("hypervisor.example", username="root"),
)
assert isinstance(HostConfig(str(tunneled)), QEMUConfig)
The SSH tunnel takes an SSHConfig, so its host keys follow the SSH rules
above; qga+ssh defaults socket_path to /run/qemu-server/DOMAIN.qga.
timeout.read (10 s) bounds one guest-agent round trip, not a command;
timeout.connect bounds opening a unix or ssh agent stream (by default it
shares the first reply's deadline) and is refused on libvirt, which opens its
own; the total bounds a command. The tunnel's SSH connection itself follows
its SSHConfig's timeout.connect.
Serial consoles
| URI | serial:///dev/ttyUSB0, serial:///COM3, serial:///loop:// ([?query]) |
| query keys | baudrate, bytesize, parity, stopbits, xonxoff, rtscts, dsrdtr, inter_byte_timeout, exclusive, and the timeout keys |
| credentials | profile, serial_factory, serial_port |
| extra | serial (unless a port is injected) |
from hostctl import HostConfig, PosixConsoleProfile, PromptConsoleProfile, SerialConfig
windows_port = HostConfig("serial:///COM3?baudrate=9600")
posix_port = HostConfig("serial:///dev/ttyUSB0")
assert posix_port.device == "/dev/ttyUSB0"
assert windows_port.device == "COM3"
network_port = SerialConfig("rfc2217://console.example:4001")
linux_console = SerialConfig(
"COM3",
profile=PosixConsoleProfile(login=[("login: ", "root"), ("Password: ", "secret")]),
)
switch_console = SerialConfig(
"COM4",
profile=PromptConsoleProfile(
"switch# ",
login=[("Username: ", "admin"), ("Password: ", "secret")],
),
)
A Linux or other POSIX shell on the line takes PosixConsoleProfile, which
frames output, stderr and the exit status without reading the echo; a network
device's console takes PromptConsoleProfile (see
Running commands).
The device reads the way a file: URI's path does (RFC 8089):
serial:///dev/ttyUSB0 and serial:/dev/ttyUSB0 are /dev/ttyUSB0. A
device pyserial resolves by name rather than as a path drops the URI's leading
slash, as file:///C:/x is C:/x: a Windows port (serial:///COM3 is
COM3, serial:///%5C%5C.%5CCOM10 is \\.\COM10) and a pyserial URL
(serial:///loop:// and serial:///loop%3A%2F%2F are both loop://). A
relative device is the opaque form, serial:ttyUSB0. Percent-encode only what
a URI path cannot hold (a space, ?, #); an encoded slash right after the
leading one is part of the device, so serial:///%2Fdev%2FttyUSB0 is still
/dev/ttyUSB0. serial:////dev/ttyUSB0 is refused, naming the three-slash
spelling. SerialConfig(device) takes the device as written, and
str(config) renders the readable form that reads back as the same device. A
console has no
username or password of its own; a login is the profile's login= steps,
whose secret=True values are redacted from transcripts. RFC 2217 is not
encrypted. timeout.read (0.1 s) and timeout.write (10 s) are pyserial's
read and write timeouts; inter_byte_timeout stays a field of its own,
since it is a framing gap within one read rather than a bound on one.
System URIs
posix://, windows:// and ios:// describe the system and name its
providers instead of owning a transport: executor and path are repeatable
query keys naming registered providers (local, ssh, sftp, winrm, or
your own). A transport provider takes its config, credentials included, from
provider_options, which is never rendered into the URI. A system config's
timeout= sets only the total (?timeout=); left DEFAULT, the provider
applies its own (a WinRM provider its config's 30 s), and connecting follows
the transport config in provider_options.
from hostctl import HostConfig, PosixHost, SSHConfig
config = HostConfig(
"posix://nas.example?executor=ssh&path=sftp",
provider_options={"ssh": SSHConfig("nas.example", username="admin")},
)
host = config.open()
assert isinstance(host, PosixHost)
host.close()
local = HostConfig("posix://localhost?executor=local&path=local")
Systems and providers covers provider selection, and Extending hostctl adding a scheme of your own.