Schemes
Every built-in implementation shares one contract: exists(), is_dir(),
iterdir(), glob(), walk(), open()/read_text()/write_bytes(),
copy()/move(), rm() and PathSyncer are derived from a few primitives
(stat, _open, listing, _mkdir, unlink, rmdir, rename, chmod).
Where a backend lacks a primitive, the operation raises
NotImplementedError; move() falls back to copy + delete when rename() is
unavailable or the target is elsewhere.
URIPath("scheme://...") returns the class registered for the scheme. None of
the schemes below needs a third-party client library. sftp:, s3:, gs:,
az:, repo:, github: and gitlab: are provided by other distributions (see
"Other distributions").
Capability matrix
| Implementation | Read | Write | Append | List | mkdir | Delete | rename | chmod |
|---|---|---|---|---|---|---|---|---|
file: (FileURIPath) |
Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
data: (DataURIPath) |
Yes | No | No | No | No | No | No | No |
zip: (ZipURIPath) |
Yes | Local file: archive |
No | Yes | Local archive | Local archive | Local archive | No |
tar: (TarURIPath) |
Yes | No | No | Yes | No | No | No | No |
archive: / archive+zip: / archive+tar: |
As zip: or tar: for the detected or pinned format |
|||||||
ftp: / ftps: (FTPURIPath) |
Yes | Yes | Yes (APPE) |
Yes | Yes | Yes | Yes | SITE CHMOD, if the server has it |
http: / https: (HTTPURIPath) |
Yes | Yes (PUT) |
Yes (rewrite or PATCH) |
Yes (HTML index) | No | Yes (DELETE) |
No | No |
dav: / davs: (DAVURIPath) |
Yes | Yes | No | Yes (PROPFIND) |
Yes (MKCOL) |
Yes | Yes (MOVE) |
No |
stat() works everywhere. st_mtime is real on every implementation except
data:, where it is 0. readlink() is implemented
by the archive schemes; no scheme in this package implements symlink_to().
Common behavior
- Errors map to pathlib's exception types (
FileNotFoundError,PermissionError,FileExistsError,IsADirectoryError,NotADirectoryError,OSError(ENOTEMPTY)); network timeouts raiseTimeoutError. Transport exceptions are not chained, because their text can carry credentials. - Credentials in a URI are redacted from
str()/repr()(the password; forrepo:/github:/gitlab:the whole userinfo). A backend (session, connection, token) is reused only for the same scheme, userinfo, host and port, so joining a path onto another host never sends it there. rename()stays on one endpoint: a target on another host or archive raisesNotImplementedError, andmove()copies and deletes instead. Astrtarget is a path (not URI syntax); a relative one is a sibling.- Query and fragment stay on every path that is derived from another by
changing only the path (
/,parent,with_name(), a listing's children), so a token in the query follows the names taken from it; the children of anhttp:ordav:listing are the exception, since the server names them. - Case: a host keeps the spelling it was given, and
http://NASA/xandhttp://nasa/xare one server and equal URIs. - Backends are passed as
URIPath(uri, backend=...)orpath.with_backend(backend), and inherited by every derived path. Without one, paths derived from one another (root / name,parent, ...) share the one backend the first of them builds, so 20 children of a fresh root use one connection; paths constructed separately do not share.
Local files and data
file:(FileURIPath) wraps aLocalPath(filepath) and delegates all I/O to it.rename()accepts local targets only. A path built from a local path object is written with the empty authority, aspathlib.Path.as_uri()writes it (file:///etc/passwd).data:(DataURIPath, RFC 2397) keeps the whole content in the URI (data:[<mediatype>][;base64],<data>): a read-only single file with amediatypeand acharsetproperty (read_text()decodes with it). Its path is the payload, soname,parent,/and the other path algebra raiseNotImplementedError.
HTTP and WebDAV
http(s):(HTTPURIPath) reads withGET(uncompressed) andstat()s withHEAD, falling back toGETon 405 and following a redirect once, at itsLocation; a final URL whose path ends in/is a directory. Listing parses Apache/nginx-style HTML indexes; a non-HTML response raisesNotADirectoryError, and an HTML file cannot be told apart from an index page. An index is read like a file: one that is cut short or stalls raises (OSErrororTimeoutError) instead of listing part of it, and one larger thanpathapi.http.MAX_REPLY_BYTES(8 MiB) raisesOSError(EFBIG); assign a larger number to list it. Configure it withpath.with_transport(transport, headers=None, timeout=(10, 60), write_method="PUT", append_mode="rewrite"); the default transport is the built-in one (see "Transports" below).- Writes send
write_methodwith the whole body on close.open("x")callsstat()first: a file that exists raisesFileExistsError, and so does any failure of that check other than not-found (nothing is sent). The upload carriesIf-None-Match: *, so a server that honours it answers 412 (FileExistsError) if the file appeared meanwhile. open("a"):append_mode="rewrite"downloads, appends and re-uploads (works on any server, not atomic);append_mode="patch"sendsPATCHwith aContent-Rangestarting at the size theHEADreply states (OSErrorif it has noContent-Length) and never falls back. A refusedPATCHraisesPermissionError(401, 403, 405, 501) orOSErrorwith the HTTP status (other codes, such as 400). Anopen("a")that raises, because reading the current content or theHEADfailed, has uploaded nothing.unlink()refuses a directory;rmdir()requires an empty one.- Requests time out after
(10, 60)seconds (connect, read) unless atimeoutis given;timeout=Nonewaits forever. - The library follows redirects itself, up to 30, for
GET,HEADandOPTIONS, but not forPROPFINDor the writes:PROPFIND,PUT,PATCH,DELETE,MKCOL,MOVEand a customwrite_methodare never followed. A 307 or 308 (forPROPFINDalso a 301, 302 or 303) to the same scheme, host and port is sent once more with the same method, headers and body; any other 3xx, a redirect to another origin and a second redirect raiseOSError(EIO) naming the status and theLocation. Point the path at the final URL if the server redirects. - Credentials in the URL (
https://user:pw@host/) are sent as anAuthorization: Basicheader, never inside the request URL; the octets of the percent-escapes are the credential (caf%C3%A9goes out as UTF-8 bytes). The default transport never reads~/.netrc. - When a
GETorHEADis redirected to a different scheme, host or port, the library dropsAuthorizationand nothing else: headers other thanAuthorizationfromwith_transport(headers=...)(X-Api-Key, a vendor token header) go to the new host too. Keep a secret inAuthorization, or point the path at the final URL. - A header or URL with a line break or other control character is a
ValueError, raised before anything is sent. - The mode of
open()is matched exactly:"r+","w+"and"a+"raiseNotImplementedError. dav(s):(DAVURIPath) is WebDAV (RFC 4918) over the equivalenthttp(s):URL, with the samewith_transport().PROPFINDgives real directory metadata,MKCOL/PUT/DELETE/MOVEfull writes. A reply that is not amultistatusdocument, or that names neither the collection nor any member of it, raisesOSError(EIO), and a listing keeps only the direct members of the collection it asked about.unlink()refuses a collection andrmdir()checks that it is empty (WebDAVDELETEis recursive);rm(recursive=True)is a singleDELETE.rename()does not overwrite an existing target (FileExistsError). Uploads usewrite_method(defaultPUT) fromwith_transport(); append mode is not supported, soappend_modehas no effect.
Transports
http: and dav: send through a transport:
any object with send(method, url, *, headers, body=None, timeout,
stream=False) returning an HTTPResponse, and close(): the contract of
pathapi.http. The
library, not the transport, follows redirects, adds credentials and maps a
status to an exception, so every transport behaves alike.
HTTPTransport(the default) is standard library only (http.client,ssl).HTTPTransport(ssl_context=None, verify=True, proxies=None):https:is verified against the system trust store, orssl_context;verify=Falseis the one switch that turns it off. It keeps one idle connection per thread and server, and sends a repeatable request again once when the server had closed that connection.- Proxies: without
proxies=the environment (HTTP_PROXY,HTTPS_PROXY,ALL_PROXY,NO_PROXY) decides.proxies=is an object indexed with the request URL that returnsNone(direct), a proxy, or an order of them to try when one cannot be reached;KeyErrormeans direct. A proxy is a URL or an object with aurl, so a proxy resolver of that shape works as it is. Onlyhttp://proxies are supported (CONNECTforhttps:); for SOCKS or anhttps://proxy use therequeststransport ofpathapi-requests. - The
requeststransport is provided by thepathapi-requestsdistribution (~/.netrc,REQUESTS_CA_BUNDLEand certifi, SOCKS proxies, digest and NTLM authentication, cookies, mounted adapters). Once it is installed,path.with_session(...)andPATHAPI_HTTP_TRANSPORT=requestsuse it; without it they raise anImportErrorthat names it.pathapi.http.transport_names()lists the transports, and a distribution adds its own by declaring an entry point in thepathapi.http_transportsgroup.
FTP
ftp(s): (FTPURIPath) uses stdlib ftplib, through
pathapi.ftp.FTPPath, which needs no URI: FTPPath("/pub/x",
host="h", user="u", tls=True) is the same tree without a ftp:// string.
FTPURIBackend(timeout=30.0, ssl_context=None, verify=True)configures it. Sockets time out after 30 seconds by default (timeout=Nonewaits forever). It opens a connection for whichever server a path names, which is why it is notpathapi.ftp.FTPBackend: that one belongs to a single host.ftps:verifies the server certificate and host name before logging in and reuses the TLS session on data connections. For a private CA passFTPURIBackend(ssl_context=ssl.create_default_context(cafile=...));FTPURIBackend(verify=False)turns verification off, and logs a warning on theuripathlogger each time one is built.- A backend is a context manager:
with FTPURIBackend() as backend:closes the connections of every thread when the block ends, asbackend.close()does. - Paths built without
backend=share one default backend, with one cached connection per server and thread, closed when the thread ends. A connection idle for more thanIDLE_PROBE_SECONDS(1 second) is probed withNOOPbefore use and replaced if the server dropped it. - A listing uses
MLSD(type, size and UTC modification time in one round trip) andstat()usesMLSTwhere the server offers it; without them the listing falls back toNLSTand the stat toSIZE/CWD. - Reads download the whole file into memory; writes are buffered in memory
and uploaded on close.
chmod()usesSITE CHMODand raisesNotImplementedErrorwhen the server lacks it.
Other distributions
These schemes and this transport are known to uripath and provided by other
distributions. A path of one of the schemes builds, joins, compares and prints
without it; the first operation that needs the server (stat(), exists(),
open(), iterdir(), ...) raises an ImportError that names what to install.
| Provides | Install |
|---|---|
sftp: |
pip install "sftppath[paramiko]" (or [asyncssh]) |
s3: |
pip install "objpath[s3]" |
gs: |
pip install "objpath[gs]" |
az: |
pip install "objpath[az]" |
repo:, github:, gitlab: |
pip install repopath (read-only paths for GitHub and GitLab repositories) |
the requests HTTP transport |
pip install pathapi-requests |
Archives
zip:/tar: address an entry inside an archive:
zip:<archive-uri>!/<inner-path> (the Java-style !/ separator of JAR URLs).
The <archive-uri> is any absolute URI with an explicit scheme, so
zip:file:///backups/site.zip!/index.html and
zip:ftp://host/nightly.zip!/index.html work the same way. name,
parent, glob(), ... operate on the inner path.
The members are read and written by pathapi.archive
(ZipPath, TarPath, ArchivePath), which take the archive as any Path and
need no URI: ZipPath("index.html", archive="/backups/site.zip").
- Writing (new members, overwriting,
mkdir(),unlink(),rmdir(),rename()) works for zip archives whose outer URI is a localfile:path. Each change replaces the archive atomically (temporary file, thenos.replace) and keeps the other members' metadata, the archive comment and any leading bytes. Every other outer scheme is read-only and is read into memory. with path.batch():holds every change made to a zip while the block is open and replaces the archive once when it ends, instead of once per change. Inside the block every path of the archive reads the held changes and the file is not touched; an exception leaves the archive byte for byte as it was.copy(recursive=True)into a zip andPathSynceronto one open it themselves: copying 300 files is one rewrite, not 300, and a copy that fails part way leaves the archive unchanged. Another thread using the archive waits for the block.- Links. A tar symlink and a zip entry whose Unix mode says link are
symbolic links:
is_symlink(),readlink()andlstat()work, andstat(),open()anditerdir()follow them inside the archive. A link that is absolute, climbs out of the archive or points at nothing isFileNotFoundError, a loopOSError(ELOOP); at most 40 links are followed.walk()andglob()list a link and do not enter it unless asked to,rm()removes the link and never its target, andcopy(follow_symlinks=False)recreates it. A listing that does follow links (walk(follow_symlinks=True),copy()'s default,PathSyncer) follows one directory link per path and never a link to its own directory or an ancestor, which is an error for that entry: spell the path to go further. Nothing is written through a link, and none can be created. tar:(also.tar.gz/.tar.bz2/.tar.xz) is read-only.archive:detects the format from the outer name, then from the file's magic bytes, on first use (building or printing a path reads nothing);archive+zip:/archive+tar:pin it. All spellings of one archive share one open handle. A remote outer is read once and kept while a path to it lives;path.refresh()makes the next use read it again.- Cost. Listing and
stat()are served from a table built once per open archive. A member is read into memory up to 16 MiB and into a temporary file beyond that. A zip change copies the members it does not touch without decompressing them. A new zip member is stored deflated. - An archive inside an archive is addressed by nesting
(
zip:zip:file:///outer.zip!/inner.zip!/x.txt) and is read-only; a!/inside a member name is written%21/. - Member names are normalized as POSIX relative paths, the same way for
every format:
./x,a//b,a/./banda/b/../call resolve, so a member lists and reads under one name however the archive was written (tar -C dir .andshutil.make_archiveprefix every member with./). The spelling as written still works. - Members whose names would escape the archive (
..past the root, or an absolute path) have no name inside it: never listed, never readable. A name that merely a Windows destination would misread --C:drive.txt, or one containing\-- is a normal member, since both are ordinary filenames on POSIX; copying or extracting it onto a Windows path is what refuses it. Exception types are the POSIX ones on every platform.
See Divergences from pathlib for the reasons behind these
choices. Every class and backend signature is in the API reference:
file:/data:,
http:/dav:, ftp:,
and archives.