requests / urllib integrations

proxylib.integrations.requests

requests integration.

ProxyMapAdapter (needs the requests extra: proxylib[requests]) is the recommended way to wire a ProxyMap into a requests.Session — it hooks HTTPAdapter.send(), so it sees the real request URL. For the simpler proxies-dict style (requests.get(url, proxies=...)) use :class:proxylib.integrations.dict.ProxyDict, which isn't requests-specific.

__all__ = ('ProxyMapAdapter',) module-attribute

ProxyMapAdapter(proxymap, *args, **kwargs)

Bases: HTTPAdapter

A Transport Adapter that resolves the proxy per-request from a ProxyMap.

Sees the real request.url (scheme, host and path), matching what a PAC file's FindProxyForURL is meant to receive::

session = requests.Session()
adapter = ProxyMapAdapter(proxymap)
session.mount("http://", adapter)
session.mount("https://", adapter)

Proxies explicitly passed to Session.proxies/request(proxies=...) at the same scheme://hostname key requests itself resolves by (see requests.utils.select_proxy) still take precedence over the ProxyMap result; a less-specific key (e.g. a bare "http" entry, or an env-var-injected one when trust_env=True) does not, since a per-request ProxyMap decision is more specific by definition -- that's the reason this adapter exists over a plain proxies dict.

Source code in src/proxylib/integrations/requests.py

def __init__(self, proxymap: ProxyMap, *args: Any, **kwargs: Any) -> None:
    self.proxymap = proxymap
    super().__init__(*args, **kwargs)

proxymap = proxymap instance-attribute

send(request, **kwargs)

Source code in src/proxylib/integrations/requests.py

def send(self, request: "PreparedRequest", **kwargs: Any):
    proxies = dict(kwargs.get("proxies") or {})
    try:
        entries = self.proxymap[request.url]
    except KeyError:
        entries = None  # no opinion: leave `proxies` as-is (env/session settings apply)

    if entries is not None:
        hostname = urlsplit(request.url).hostname
        if hostname:
            scheme = urlsplit(request.url).scheme
            resolved = next(iter(entries), None)
            # The most-specific key select_proxy() checks (scheme://
            # hostname, ahead of a bare scheme/"all") -- this is what
            # actually outranks requests' own env-proxy injection
            # (Session.merge_environment_settings runs before this
            # adapter's send(), via `proxies.setdefault(scheme, ...)`
            # at the *scheme* level only, so it can never pre-occupy
            # this more specific key). `None` here means DIRECT:
            # select_proxy()'s `if proxy:` check treats it the same
            # as "not configured".
            proxies.setdefault(
                f"{scheme}://{hostname}",
                resolved.as_uri() if resolved is not None else None,
            )
    kwargs["proxies"] = proxies
    return super().send(request, **kwargs)

proxylib.integrations.urllib

urllib.request integration.

ProxyMapHandler resolves the proxy per-request from a ProxyMap, rather than a static {scheme: proxy_url} dict like the stdlib urllib.request.ProxyHandler normally uses -- the same upgrade :class:proxylib.integrations.requests.ProxyMapAdapter is over a plain proxies dict, and for the same reason: a static dict can't see the request's host/path, which most ProxyMap rules (PAC scripts, path-dependent NO_PROXY) need.

__all__ = ('ProxyMapHandler',) module-attribute

ProxyMapHandler(proxymap)

Bases: ProxyHandler

A urllib.request handler that resolves proxies per-request from a ProxyMap::

opener = urllib.request.build_opener(ProxyMapHandler(proxymap)) opener.open("https://example.com")

or, to affect every urllib.request.urlopen() call:

urllib.request.install_opener(opener)

Source code in src/proxylib/integrations/urllib.py

def __init__(self, proxymap: ProxyMap) -> None:
    # proxies={}: skip ProxyHandler.__init__'s per-protocol setattr(), which
    # would otherwise shadow the http_open/https_open/ftp_open methods
    # below with static, env-derived ones (its default when proxies=None).
    super().__init__(proxies={})
    self.proxymap = proxymap

ftp_open = _resolve class-attribute instance-attribute

http_open = _resolve class-attribute instance-attribute

https_open = _resolve class-attribute instance-attribute

proxymap = proxymap instance-attribute

proxylib.integrations.dict

Plain proxies-dict integration: ProxyDict duck-types the {scheme_or_url: "proxy://uri"} mapping requests and similar libraries accept, resolving from a ProxyMap.

__all__ = ('ProxyDict',) module-attribute

ProxyDict(proxymap)

Duck-types the plain {scheme_or_url: "proxy://uri"} proxies dict that requests and similar libraries accept, resolving from a ProxyMap.

The simpler integration when a Transport Adapter/handler isn't wanted::

requests.get(url, proxies=ProxyDict(proxymap))

Consumers of a proxies dict look keys up by scheme/host, not the full request URL, so path-dependent rules (most PAC scripts) won't apply -- use ProxyMapAdapter/ProxyMapHandler when that fidelity matters.

Lookups return the first proxy's URI string; DIRECT raises KeyError (a missing key means "no proxy", matching the dict convention).

Deliberately composes a ProxyMap rather than subclassing it -- __getitem__ here returns a str (a proxy URI), not Iterable[Optional[Proxy]], which would be an LSP violation if this were treated as one.

Source code in src/proxylib/integrations/dict.py

def __init__(self, proxymap: ProxyMap) -> None:
    self.proxymap = proxymap

__slots__ = ('proxymap',) class-attribute instance-attribute

proxymap = proxymap instance-attribute

__contains__(key)

Source code in src/proxylib/integrations/dict.py

def __contains__(self, key: object) -> bool:
    try:
        self[key]
        return True
    except KeyError:
        return False

__getitem__(uri)

Source code in src/proxylib/integrations/dict.py

def __getitem__(self, uri: str) -> str:
    try:
        proxy = next(iter(self.proxymap[uri]))
        if proxy is None:
            raise KeyError(uri)
        return proxy.as_uri()
    except StopIteration:
        raise KeyError(uri)

copy()

Source code in src/proxylib/integrations/dict.py

def copy(self) -> "ProxyDict":
    return type(self)(self.proxymap)

get(uri, default=None)

Source code in src/proxylib/integrations/dict.py

def get(self, uri: str, default=None):
    try:
        return self[uri]
    except KeyError:
        return default

setdefault(url, value)

No-op: requests calls this to merge env proxies; the ProxyMap wins.

Source code in src/proxylib/integrations/dict.py

def setdefault(self, url: str, value: str) -> None:
    """No-op: requests calls this to merge env proxies; the ProxyMap wins."""