Skip to content

System settings

proxylib.system

This machine's proxy settings.

  • :func:auto_proxy -- the proxy map a program should use: the proxy environment variables when any is set, else the OS's own resolver backed by the OS settings.
  • :func:system_config -- the OS or desktop settings, as a :class:ProxyConfig.
  • :class:WinHTTPProxyMap, :class:CFNetworkProxyMap, :class:LibProxyMap -- the OS resolvers (Windows, macOS, libproxy).

__all__ = ('CFNetworkProxyMap', 'LibProxyMap', 'ProxyConfig', 'WinHTTPProxyMap', 'auto_proxy', 'system_config') module-attribute

CFNetworkProxyMap(deadline=5.0)

Bases: ProxyMap

A proxy map answered by macOS's resolver (CFNetworkCopyProxiesForURL).

CFNetwork applies the system's bypass list itself and runs a configured PAC (CFNetworkExecuteProxyAutoConfigurationURL), so no JS engine is needed. A PAC run is bounded by deadline seconds; when it overruns, or CFNetwork cannot answer, the lookup has no opinion (KeyError) rather than DIRECT, so a chain can ask the next map. macOS only: off it, every lookup raises KeyError.

Source code in src/proxylib/system/_darwin.py
386
387
def __init__(self, deadline: float = 5.0) -> None:
    self.deadline = deadline

__slots__ = ('deadline',) class-attribute instance-attribute

deadline = deadline instance-attribute

__getitem__(url)

Source code in src/proxylib/system/_darwin.py
395
396
397
398
399
def __getitem__(self, url: str) -> "Tuple[Optional[Proxy], ...]":
    entries = _resolve_proxies_for_url(url, self.deadline)
    if entries is None:
        raise KeyError(url)
    return tuple(Proxy.parse(entry) if entry else None for entry in entries) or (None,)

__repr__()

Source code in src/proxylib/system/_darwin.py
389
390
def __repr__(self) -> str:
    return f"CFNetworkProxyMap(deadline={self.deadline!r})"

close()

Nothing to free: CFNetwork state is per lookup.

Source code in src/proxylib/system/_darwin.py
392
393
def close(self) -> None:
    """Nothing to free: CFNetwork state is per lookup."""

LibProxyMap(lib=None)

Bases: ProxyMap

A proxy map answered by libproxy (px_proxy_factory_get_proxies).

lib is the loaded library; by default it is found on the linker path. When libproxy is not installed, or cannot answer, a lookup has no opinion (KeyError) rather than DIRECT.

One libproxy factory, made on the first lookup, serves every lookup so its PAC and settings caches are reused; :meth:close releases it (garbage collection does too).

Source code in src/proxylib/system/_posix/_libproxy.py
63
64
65
66
def __init__(self, lib: "Optional[ctypes.CDLL]" = None) -> None:
    self._lib = lib if lib is not None else _get_libproxy()
    self._factory = None
    self._lock = threading.Lock()

__slots__ = ('_lib', '_factory', '_lock') class-attribute instance-attribute

__del__()

Source code in src/proxylib/system/_posix/_libproxy.py
74
75
76
77
78
def __del__(self) -> None:
    try:
        self.close()
    except AttributeError:  # __init__ did not finish
        pass

__getitem__(uri)

Source code in src/proxylib/system/_posix/_libproxy.py
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def __getitem__(self, uri: str) -> "Tuple[Optional[Proxy], ...]":
    lib = self._lib
    if lib is None:
        raise KeyError(uri)

    with self._lock:
        if not self._factory:
            self._factory = lib.px_proxy_factory_new()
        if not self._factory:
            raise KeyError(uri)
        proxies_array = lib.px_proxy_factory_get_proxies(self._factory, uri.encode("utf-8"))
    if not proxies_array:
        # libproxy failed to answer, which is not a DIRECT decision.
        raise KeyError(uri)
    try:
        results: "list[Optional[Proxy]]" = []
        i = 0
        while proxies_array[i] is not None:
            entry = proxies_array[i].decode("utf-8")
            results.append(None if entry.startswith("direct://") else Proxy.parse(entry))
            i += 1
        return tuple(results) or (None,)
    finally:
        lib.px_proxy_factory_free_proxies(proxies_array)

close()

Source code in src/proxylib/system/_posix/_libproxy.py
68
69
70
71
72
def close(self) -> None:
    with self._lock:
        factory, self._factory = self._factory, None
    if factory and self._lib is not None:
        self._lib.px_proxy_factory_free(factory)

ProxyConfig(source='none', state='unset', auto_detect=False, pac_url=None, pac_script=None, proxies=dict(), bypass=BypassRules.NONE) dataclass

This machine's proxy settings, as one OS or desktop stores them.

source names where they were read: "windows", "macos", "gnome", "mate", "kde", "networkmanager", or "none". state is "direct" when the settings say to use no proxy, "configured" when they name auto-detection, a PAC or a proxy, and "unset" when there were no settings to read.

proxies maps a request scheme (http, https, ftp) to the proxy for it, and "socks" to the proxy for every scheme without its own entry; a string value is parsed with :meth:Proxy.parse. bypass applies to those proxies only, as it does on every OS.

auto_detect = False class-attribute instance-attribute

bypass = BypassRules.NONE class-attribute instance-attribute

pac_script = None class-attribute instance-attribute

pac_url = None class-attribute instance-attribute

proxies = field(default_factory=dict) class-attribute instance-attribute

source = 'none' class-attribute instance-attribute

state = 'unset' class-attribute instance-attribute

__post_init__()

Source code in src/proxylib/system/_config.py
57
58
59
60
61
62
63
64
def __post_init__(self) -> None:
    if self.state not in _STATES:
        raise ValueError(f"state must be one of {_STATES}, not {self.state!r}")
    proxies = {
        key.lower(): value if isinstance(value, Proxy) else Proxy.parse(value)
        for key, value in self.proxies.items()
    }
    object.__setattr__(self, "proxies", MappingProxyType(proxies))

to_proxymap(*, wpad=False, timeout=10.0, on_missing_engine='raise')

The proxy map these settings describe.

Sources are asked in the order browsers use: auto-detection (WPAD), the PAC, then the proxies. A source that fails -- no WPAD server, a PAC that does not download or run -- passes to the next and is tried again after 30 s; when none answers, the answer is DIRECT. WPAD runs only when auto_detect is set, or when wpad is true and the state is "unset"; a "direct" state is DIRECT for every URL. Nothing is fetched until the first lookup.

timeout bounds each WPAD probe and PAC download. on_missing_engine is as for :class:~proxylib.PacScript.

Source code in src/proxylib/system/_config.py
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
def to_proxymap(
    self, *, wpad: bool = False, timeout: float = 10.0, on_missing_engine: str = "raise"
) -> ProxyMap:
    """The proxy map these settings describe.

    Sources are asked in the order browsers use: auto-detection (WPAD),
    the PAC, then the proxies. A source that fails -- no WPAD server, a
    PAC that does not download or run -- passes to the next and is tried
    again after 30 s; when none answers, the answer is DIRECT. WPAD runs
    only when ``auto_detect`` is set, or when ``wpad`` is true and the
    state is ``"unset"``; a ``"direct"`` state is DIRECT for every URL.
    Nothing is fetched until the first lookup.

    ``timeout`` bounds each WPAD probe and PAC download.
    ``on_missing_engine`` is as for :class:`~proxylib.PacScript`.
    """
    if self.state == "direct":
        return SimpleProxyMap()
    maps: "List[ProxyMap]" = []
    if self.auto_detect or (wpad and self.state == "unset"):
        from ..pac import wpad as _wpad

        maps.append(
            _Deferred(
                "WPAD",
                functools.partial(_wpad.discover, timeout=timeout, on_missing_engine=on_missing_engine),
            )
        )
    if self.pac_url:
        from .. import pac

        maps.append(
            _Deferred(
                self.pac_url,
                functools.partial(pac.load, self.pac_url, timeout=timeout, on_missing_engine=on_missing_engine),
            )
        )
    if self.pac_script:
        from .. import pac

        maps.append(
            _Deferred(
                "the PAC script", functools.partial(pac.loads, self.pac_script, on_missing_engine=on_missing_engine)
            )
        )
    maps.append(manual_map(self.proxies, self.bypass))
    return maps[0] if len(maps) == 1 else ChainProxyMap(*maps)

WinHTTPProxyMap()

Bases: ProxyMap

A proxy map answered by WinHTTP's resolver (WinHttpGetProxyForUrl).

WinHTTP does the WPAD discovery (DHCP and DNS), fetches the PAC with the user's NTLM/Kerberos credentials when challenged, and runs it, so no JS engine is needed. When the settings name neither auto-detection nor a PAC, or WinHTTP fails, the manual proxy settings answer; with none, DIRECT.

One WinHTTP session serves every lookup, and WinHTTP caches PAC results per session; :meth:close releases it (garbage collection does too). After close a lookup has no opinion (KeyError). Windows only: construction raises OSError elsewhere.

Source code in src/proxylib/system/_nt.py
234
235
236
237
238
239
def __init__(self) -> None:
    self._hsession = None
    libs = _load()
    self._hsession = libs.winhttp.WinHttpOpen("proxylib", _WINHTTP_ACCESS_TYPE_NO_PROXY, None, None, 0)
    if self._hsession:
        libs.winhttp.WinHttpSetTimeouts(self._hsession, *([_SESSION_TIMEOUT_MS] * 4))

__slots__ = ('_hsession',) class-attribute instance-attribute

__del__()

Source code in src/proxylib/system/_nt.py
249
250
251
252
253
def __del__(self) -> None:
    try:
        self.close()
    except (AttributeError, OSError):  # __init__ did not finish
        pass

__getitem__(url)

Source code in src/proxylib/system/_nt.py
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
def __getitem__(self, url: str) -> "Tuple[Optional[Proxy], ...]":
    if not self._hsession:
        raise KeyError(url)
    ie = _read_ie_proxy_config()
    auto_detect = bool(ie and ie.auto_detect)
    auto_config_url = ie.auto_config_url if ie else None

    if auto_detect or auto_config_url:
        proxy, _bypass, ok, last_error = _query_proxy_for_url(
            self._hsession, url, auto_detect, auto_config_url, auto_logon=False
        )
        # Retry with fAutoLogonIfChallenged only after a real NTLM/Kerberos
        # challenge: setting it on every call turns off WinHTTP's caching.
        if not ok and last_error == _ERROR_WINHTTP_LOGIN_FAILURE:
            proxy, _bypass, ok, last_error = _query_proxy_for_url(
                self._hsession, url, auto_detect, auto_config_url, auto_logon=True
            )
        if ok:
            if not proxy:
                return (None,)
            return tuple(_resolve_for_scheme(url, proxy)) or (None,)
        log.debug("WinHttpGetProxyForUrl failed for %s with error %d", url, last_error)

    if ie is None:
        return (None,)
    config = config_from_settings(ie)
    return manual_map(config.proxies, config.bypass)[url]

__repr__()

Source code in src/proxylib/system/_nt.py
241
242
def __repr__(self) -> str:
    return "WinHTTPProxyMap()" if self._hsession else "WinHTTPProxyMap(<closed>)"

close()

Source code in src/proxylib/system/_nt.py
244
245
246
247
def close(self) -> None:
    if self._hsession:
        _load().winhttp.WinHttpCloseHandle(self._hsession)
        self._hsession = None

auto_proxy(*, system=True, no_proxy=None, wpad=False, timeout=10.0, on_missing_engine='raise', environ=None)

The proxy map for this machine. It answers every URL.

When any proxy variable (HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, ...) is set in environ (default os.environ), those variables and NO_PROXY alone decide, as in :func:urllib.request.getproxies: a scheme without a variable goes DIRECT, and the OS is not consulted. A variable may hold a PAC location (see :class:EnvProxyConfig).

Otherwise the OS settings decide. With system=True the OS's own resolver answers when there is one -- WinHTTP on Windows, CFNetwork on macOS, libproxy elsewhere when it is installed -- and a URL it cannot answer goes to :meth:ProxyConfig.to_proxymap of :func:system_config; system=False uses only the latter. wpad, timeout and on_missing_engine are passed to it: WPAD runs only when the OS asks for auto-detection, or with wpad=True when it has no settings.

no_proxy rules (BypassRules or NO_PROXY text) are applied over everything.

Source code in src/proxylib/system/__init__.py
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
def auto_proxy(
    *,
    system: bool = True,
    no_proxy: "Union[BypassRules, str, Iterable[str], None]" = None,
    wpad: bool = False,
    timeout: float = 10.0,
    on_missing_engine: str = "raise",
    environ: "Optional[Mapping[str, str]]" = None,
) -> ProxyMap:
    """The proxy map for this machine. It answers every URL.

    When any proxy variable (``HTTP_PROXY``, ``HTTPS_PROXY``, ``ALL_PROXY``,
    ...) is set in ``environ`` (default ``os.environ``), those variables and
    ``NO_PROXY`` alone decide, as in :func:`urllib.request.getproxies`: a
    scheme without a variable goes DIRECT, and the OS is not consulted. A
    variable may hold a PAC location (see :class:`EnvProxyConfig`).

    Otherwise the OS settings decide. With ``system=True`` the OS's own
    resolver answers when there is one -- WinHTTP on Windows, CFNetwork on
    macOS, libproxy elsewhere when it is installed -- and a URL it cannot
    answer goes to :meth:`ProxyConfig.to_proxymap` of :func:`system_config`;
    ``system=False`` uses only the latter. ``wpad``, ``timeout`` and
    ``on_missing_engine`` are passed to it: WPAD runs only when the OS asks
    for auto-detection, or with ``wpad=True`` when it has no settings.

    ``no_proxy`` rules (``BypassRules`` or ``NO_PROXY`` text) are applied
    over everything.
    """
    env = os.environ if environ is None else environ
    proxymap: ProxyMap
    if _has_proxy_variable(env):
        log.debug("using the proxy environment variables")
        proxymap = ChainProxyMap(EnvProxyConfig(env), SimpleProxyMap())
    else:
        fallback = system_config().to_proxymap(wpad=wpad, timeout=timeout, on_missing_engine=on_missing_engine)
        resolver = _os_resolver() if system else None
        log.debug("OS resolver: %r", resolver)
        proxymap = ChainProxyMap(resolver, fallback) if resolver is not None else fallback
    if no_proxy is not None:
        proxymap = Bypass(proxymap, no_proxy)
    return proxymap

system_config()

This machine's proxy settings, from the OS or desktop; the environment variables are not read.

Windows: the current user's Internet Options. macOS: System Configuration (scutil --proxy). Elsewhere: GNOME, MATE, KDE and NetworkManager, the first with a proxy configured, else the first with settings. With none readable, ProxyConfig() (source="none", state="unset").

Source code in src/proxylib/system/__init__.py
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
def system_config() -> ProxyConfig:
    """This machine's proxy settings, from the OS or desktop; the environment
    variables are not read.

    Windows: the current user's Internet Options. macOS: System Configuration
    (``scutil --proxy``). Elsewhere: GNOME, MATE, KDE and NetworkManager, the
    first with a proxy configured, else the first with settings. With none
    readable, ``ProxyConfig()`` (``source="none"``, ``state="unset"``).
    """
    if _PLATFORM == "win32":
        config = _nt.read_config()
    elif _PLATFORM == "darwin":
        config = _darwin.read_config()
    else:
        config = _posix.read_config()
    if config is None:
        config = ProxyConfig()
    log.debug("system proxy settings: %r", config)
    return config

proxylib.system._config.ProxyConfig(source='none', state='unset', auto_detect=False, pac_url=None, pac_script=None, proxies=dict(), bypass=BypassRules.NONE) dataclass

This machine's proxy settings, as one OS or desktop stores them.

source names where they were read: "windows", "macos", "gnome", "mate", "kde", "networkmanager", or "none". state is "direct" when the settings say to use no proxy, "configured" when they name auto-detection, a PAC or a proxy, and "unset" when there were no settings to read.

proxies maps a request scheme (http, https, ftp) to the proxy for it, and "socks" to the proxy for every scheme without its own entry; a string value is parsed with :meth:Proxy.parse. bypass applies to those proxies only, as it does on every OS.

auto_detect = False class-attribute instance-attribute

bypass = BypassRules.NONE class-attribute instance-attribute

pac_script = None class-attribute instance-attribute

pac_url = None class-attribute instance-attribute

proxies = field(default_factory=dict) class-attribute instance-attribute

source = 'none' class-attribute instance-attribute

state = 'unset' class-attribute instance-attribute

__post_init__()

Source code in src/proxylib/system/_config.py
57
58
59
60
61
62
63
64
def __post_init__(self) -> None:
    if self.state not in _STATES:
        raise ValueError(f"state must be one of {_STATES}, not {self.state!r}")
    proxies = {
        key.lower(): value if isinstance(value, Proxy) else Proxy.parse(value)
        for key, value in self.proxies.items()
    }
    object.__setattr__(self, "proxies", MappingProxyType(proxies))

to_proxymap(*, wpad=False, timeout=10.0, on_missing_engine='raise')

The proxy map these settings describe.

Sources are asked in the order browsers use: auto-detection (WPAD), the PAC, then the proxies. A source that fails -- no WPAD server, a PAC that does not download or run -- passes to the next and is tried again after 30 s; when none answers, the answer is DIRECT. WPAD runs only when auto_detect is set, or when wpad is true and the state is "unset"; a "direct" state is DIRECT for every URL. Nothing is fetched until the first lookup.

timeout bounds each WPAD probe and PAC download. on_missing_engine is as for :class:~proxylib.PacScript.

Source code in src/proxylib/system/_config.py
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
def to_proxymap(
    self, *, wpad: bool = False, timeout: float = 10.0, on_missing_engine: str = "raise"
) -> ProxyMap:
    """The proxy map these settings describe.

    Sources are asked in the order browsers use: auto-detection (WPAD),
    the PAC, then the proxies. A source that fails -- no WPAD server, a
    PAC that does not download or run -- passes to the next and is tried
    again after 30 s; when none answers, the answer is DIRECT. WPAD runs
    only when ``auto_detect`` is set, or when ``wpad`` is true and the
    state is ``"unset"``; a ``"direct"`` state is DIRECT for every URL.
    Nothing is fetched until the first lookup.

    ``timeout`` bounds each WPAD probe and PAC download.
    ``on_missing_engine`` is as for :class:`~proxylib.PacScript`.
    """
    if self.state == "direct":
        return SimpleProxyMap()
    maps: "List[ProxyMap]" = []
    if self.auto_detect or (wpad and self.state == "unset"):
        from ..pac import wpad as _wpad

        maps.append(
            _Deferred(
                "WPAD",
                functools.partial(_wpad.discover, timeout=timeout, on_missing_engine=on_missing_engine),
            )
        )
    if self.pac_url:
        from .. import pac

        maps.append(
            _Deferred(
                self.pac_url,
                functools.partial(pac.load, self.pac_url, timeout=timeout, on_missing_engine=on_missing_engine),
            )
        )
    if self.pac_script:
        from .. import pac

        maps.append(
            _Deferred(
                "the PAC script", functools.partial(pac.loads, self.pac_script, on_missing_engine=on_missing_engine)
            )
        )
    maps.append(manual_map(self.proxies, self.bypass))
    return maps[0] if len(maps) == 1 else ChainProxyMap(*maps)