syshttp.proxy

syshttp.proxy

Proxy routing: syshttp's settings as one proxylib proxy map.

Settings are read through an :class:~syshttp.env.HTTPEnv (env=, default HTTPEnv()), each name under each of its prefixes (SYSHTTP_PREFIX); below they are named as written, under the empty prefix.

:func:proxy_map is a proxylib ProxyMap that reads the settings at every lookup and answers, in order:

1. SYSHTTP_PROXY -- a proxy for this package's traffic, distinct from the machine's so other tools (git, npm, corporate CLIs) are not forced through it. Its credential is SYSHTTP_PROXY_AUTH (a header value sent verbatim, in SYSHTTP_PROXY_AUTH_HEADER, default Proxy-Authorization), else the URL's userinfo as Basic. SYSHTTP_PROXY_TYPE=prefix[:/endpoint] makes it a URL-prefix gateway: GET https://api.example.com/x becomes GET <proxy><endpoint>https://api.example.com/x.
2. The machine's proxy: the *_PROXY variables when any is set (per scheme, as proxylib's EnvProxyConfig reads them, the lowercase spelling first under each prefix), else the system's configuration for the URL (WinHTTP on Windows, CFNetwork on macOS, libproxy or the desktop's settings elsewhere; PAC scripts and WPAD included), unless SYSHTTP_SYSTEM_PROXY is off. HTTP_PROXY_AUTH is its credential (Proxy-Authorization).
3. DIRECT.

NO_PROXY / no_proxy hosts go direct before any of it (proxylib's BypassRules: suffixes, CIDR, ports, loopback). An answer is a list (PROXY a; PROXY b; DIRECT): proxylib's client integrations try it in order and raise the last error when no entry connects; a client that sends through one proxy only (urllib, a curl command line) takes it through proxylib.Probed, which raises :class:NoReachableProxyError.

auth_header(*, env=None)

The name of the header SYSHTTP_PROXY_AUTH is sent in: SYSHTTP_PROXY_AUTH_HEADER, else Proxy-Authorization. A value with a colon or whitespace is not a header name: a ValueError.

Source code in src/syshttp/proxy.py

def auth_header(*, env: Optional[HTTPEnv] = None) -> str:
    """The name of the header ``SYSHTTP_PROXY_AUTH`` is sent in:
    ``SYSHTTP_PROXY_AUTH_HEADER``, else ``Proxy-Authorization``. A value with
    a colon or whitespace is not a header name: a ``ValueError``."""
    env = _env(env)
    name = env.get(PROXY_AUTH_HEADER)
    if not name:
        return DEFAULT_AUTH_HEADER
    if ":" in name or any(ch.isspace() for ch in name):
        raise ConfigurationError("%s must be a header name, not %r" % (_var(env, PROXY_AUTH_HEADER), name))
    return name

configured_proxy(*, env=None)

SYSHTTP_PROXY as a proxylib Proxy with its credential and kind, or None when it is unset. SYSHTTP_PROXY_AUTH replaces the URL's userinfo; SYSHTTP_PROXY_TYPE=prefix makes a gateway. A malformed value raises ValueError (ProxyParseError for the URL).

Source code in src/syshttp/proxy.py

def configured_proxy(*, env: Optional[HTTPEnv] = None) -> Optional[Proxy]:
    """``SYSHTTP_PROXY`` as a proxylib ``Proxy`` with its credential and kind,
    or ``None`` when it is unset. ``SYSHTTP_PROXY_AUTH`` replaces the URL's
    userinfo; ``SYSHTTP_PROXY_TYPE=prefix`` makes a gateway. A malformed value
    raises ``ValueError`` (``ProxyParseError`` for the URL)."""
    env = _env(env)
    raw = env.get(PROXY)
    if not raw:
        return None
    found = Proxy.parse(raw)
    kind, endpoint = proxy_type(env=env)
    if kind == "prefix":
        if found.scheme not in ("http", "https"):
            raise ConfigurationError("%s=prefix needs an http or https %s" % (_var(env, PROXY_TYPE), _var(env, PROXY)))
        found = Proxy("prefix+" + found.scheme, found.host, found.port, found.username, found.password, path=endpoint)
    value = env.get(PROXY_AUTH)
    if not value:
        return found
    return _with_credential(found, auth_header(env=env), _checked(_var(env, PROXY_AUTH), value))

proxy_map(proxy=None, noproxy=None, proxy_user=None, *, env=None)

syshttp's routing as a proxylib ProxyMap (proxylib.lookup(m, url) reads it; every proxylib integration takes it). The settings are read at every lookup, so one map serves a long-lived client. It always answers (never KeyError).

proxy (curl's -x) replaces the settings with that proxy, always a plain one whose credential is its own userinfo, never the configured SYSHTTP_PROXY_AUTH; "" means direct. noproxy (--noproxy) replaces NO_PROXY. proxy_user (-U user:password) becomes the credential of whichever proxy answers, replacing the configured one. A malformed setting raises ValueError at lookup, only when it would be used.

Source code in src/syshttp/proxy.py

def proxy_map(
    proxy: Optional[str] = None,
    noproxy: Optional[str] = None,
    proxy_user: Optional[str] = None,
    *,
    env: Optional[HTTPEnv] = None,
) -> Any:
    """syshttp's routing as a proxylib ``ProxyMap`` (``proxylib.lookup(m,
    url)`` reads it; every proxylib integration takes it). The settings are
    read at every lookup, so one map serves a long-lived client. It always
    answers (never ``KeyError``).

    ``proxy`` (curl's ``-x``) replaces the settings with that proxy, always a
    plain one whose credential is its own userinfo, never the configured
    ``SYSHTTP_PROXY_AUTH``; ``""`` means direct. ``noproxy`` (``--noproxy``)
    replaces ``NO_PROXY``. ``proxy_user`` (``-U user:password``) becomes the
    credential of whichever proxy answers, replacing the configured one. A
    malformed setting raises ``ValueError`` at lookup, only when it would be
    used."""
    return _SettingsMap(proxy, noproxy, proxy_user, env)

proxy_type(*, env=None)

("connect", None) for a standard proxy, or ("prefix", "/endpoint") for a URL-prefix gateway (endpoint defaults to /), from SYSHTTP_PROXY_TYPE. Anything else is a configuration error, raised.

Source code in src/syshttp/proxy.py

def proxy_type(*, env: Optional[HTTPEnv] = None) -> Tuple[str, Optional[str]]:
    """``("connect", None)`` for a standard proxy, or ``("prefix", "/endpoint")``
    for a URL-prefix gateway (endpoint defaults to ``/``), from
    ``SYSHTTP_PROXY_TYPE``. Anything else is a configuration error, raised."""
    env = _env(env)
    raw = env.get(PROXY_TYPE)
    if not raw:
        return "connect", None
    kind, _, endpoint = raw.partition(":")
    kind = kind.strip().lower()
    if kind == "connect":
        return "connect", None
    if kind != "prefix":
        raise ConfigurationError("%s must be 'connect' or 'prefix[:/endpoint]', not %r" % (_var(env, PROXY_TYPE), raw))
    endpoint = endpoint.strip() or "/"
    if not endpoint.startswith("/"):
        endpoint = "/" + endpoint
    return "prefix", endpoint

system_map()

The system's proxy configuration as a proxylib ProxyMap (proxylib.auto_proxy with no environment variables: the OS resolver, then the OS settings), read once per :data:SYSTEM_PROXY_MAX_AGE seconds, each URL's answer kept as long. None when it cannot be read (logged at DEBUG). A URL it cannot answer raises KeyError.

Source code in src/syshttp/proxy.py

def system_map() -> Any:
    """The system's proxy configuration as a proxylib ``ProxyMap``
    (``proxylib.auto_proxy`` with no environment variables: the OS resolver,
    then the OS settings), read once per :data:`SYSTEM_PROXY_MAX_AGE` seconds,
    each URL's answer kept as long. ``None`` when it cannot be read (logged at
    DEBUG). A URL it cannot answer raises ``KeyError``."""
    global _system, _system_at
    with _system_lock:
        now = time.monotonic()
        if _system_at and now - _system_at <= SYSTEM_PROXY_MAX_AGE:
            return _system
        try:
            _system = proxylib.Cached(_Quiet(proxylib.auto_proxy(environ={})), ttl=SYSTEM_PROXY_MAX_AGE)
        except Exception as exc:
            _log.debug("the system proxy configuration could not be read: %s", exc)
            _system = None
        _system_at = now
        return _system

system_proxy_enabled(*, env=None)

SYSHTTP_SYSTEM_PROXY as a boolean, default on. Any other value than 1/true/yes/on or 0/false/no/off (any case) is a ValueError naming the variable.

Source code in src/syshttp/proxy.py

def system_proxy_enabled(*, env: Optional[HTTPEnv] = None) -> bool:
    """``SYSHTTP_SYSTEM_PROXY`` as a boolean, default on. Any other value than
    ``1``/``true``/``yes``/``on`` or ``0``/``false``/``no``/``off`` (any case)
    is a ``ValueError`` naming the variable."""
    env = _env(env)
    raw = env.get(SYSTEM_PROXY)
    if raw is None:
        return True
    if raw.lower() in _TRUE:
        return True
    if raw.lower() in _FALSE:
        return False
    raise ConfigurationError("%s must be one of %s, not %r" % (_var(env, SYSTEM_PROXY), "/".join(_TRUE + _FALSE), raw))