Extending
Two equally first-class ways to add a new path-addressable resource. In
both, you implement a small, documented method surface; everything else
(open/read_text/write_text/glob/walk/touch/rm/copy/move/
exists/is_dir/is_file/...) is derived automatically from the
protocols in pathlib_next.protocols.
- Track A -- subclass
Pathdirectly: for any custom path-addressable resource that isn't naturally a URI (e.g. a database-backed virtual filesystem, an archive member, a key-value store).MemPathis the reference exemplar. - Track B -- subclass
UriPath: for a new URI scheme (http:,sftp:, ...). Registers automatically and gets pure-path parsing (join, query, fragment) for free fromUri.
Whichever track you pick, run the shared contract test suite against your implementation -- see Testing your implementation below.
Track A: subclass Path
Required (pure-path side, from the Pathname ABC):
segments # property -> sequence of path component strings
parts # property -> whatever "parts" means for your type
parent # property -> the logical parent
with_segments(*segments) # construct a same-type instance from new segments
as_uri() # a URI string identifying this path (can be a custom scheme)
relative_to(other) # or raise NotImplementedError if not meaningful
Optional I/O, implement whichever your resource actually supports -- leave
the rest as the inherited @notimplemented stubs (derived helpers either
fall back, e.g. move() falls back to copy+unlink when rename() isn't
implemented, or raise NotImplementedError cleanly):
iterdir() # yield child instances
_scandir() # optional: yield (name, FileStat|None) pairs
# instead, if listing your resource can
# cheaply include stat metadata -- speeds
# up walk()/glob() (see Track B's
# "_scandir: listing with metadata" below,
# which applies here too)
stat(*, follow_symlinks=True) # -> a FileStatLike (utils.stat.FileStat is a
# ready-made concrete one)
_open(mode, buffering) # -> a *binary* IOBase; open()/read_text()/
# write_bytes()/copy() are all derived
# from this one method
_mkdir(mode) # create just this directory (mkdir() layers
# parents=/exist_ok= handling on top)
unlink(), rmdir()
rename(target)
chmod(mode, *, follow_symlinks=True)
MemPath (src/pathlib_next/mempath.py) implements exactly this surface
over a backend of nested dicts (MemPathBackend; a dict value is a
directory, a bytearray value is a file) -- read it end to end as a
worked example; it's under 200 lines.
Track B: subclass UriPath
The pure-path side (parsing, join, query/fragment, with_*) comes free
from Uri. Register your scheme and implement the I/O surface:
from pathlib_next.uri import UriPath
class MyPath(UriPath):
__SCHEMES = ("myscheme",) # name-mangled per-class; redeclare in every
# subclass, don't inherit it
def _listdir(self):
... # yield child *names* (str), not instances --
# UriPath.iterdir() wraps each into a child
def stat(self, *, follow_symlinks=True):
...
def _open(self, mode="r", buffering=-1):
...
def _mkdir(self, mode): ...
def unlink(self, missing_ok=False): ...
def rmdir(self): ...
def rename(self, target): ...
def chmod(self, mode, *, follow_symlinks=True): ...
Importing the module that defines your subclass is enough to register it
(UriPath._schemesmap() walks __subclasses__() and caches the result) --
UriPath("myscheme://host/path") then dispatches to MyPath automatically.
_scandir: listing with metadata
If your remote listing call already returns type/size/mtime for every
child in one round trip (an HTML directory index, WebDAV PROPFIND, SFTP
listdir_attr, FTP MLSD, an S3 list_objects_v2 page, ...), override
_scandir() instead of (or alongside) _listdir():
def _scandir(self):
for name, meta in my_one_shot_listing_call(self.path):
yield name, FileStat(st_size=meta.size, st_mtime=meta.mtime,
is_dir=meta.is_dir)
UriPath.iterdir() is derived from _scandir() and pre-seeds each child
with its FileStat as a single-use hint: the child's first stat() call
returns the hint directly (no request), and every call after that re-fetches
for real -- so a live mutation is never masked by a stale value. walk()/
glob() then classify directories vs. files from this same hint, turning a
remote-tree walk from O(entries) round trips into O(dirs). If you don't
override _scandir(), it falls back to _listdir() + one stat() per
child (no round-trip savings, but nothing breaks) -- _listdir()/
iterdir() remain fully supported on their own for schemes that have no
richer listing call to offer. See HttpPath/DavPath/SftpPath/FtpPath/
S3Path (src/pathlib_next/uri/schemes/) for worked examples.
Optional: override _initbackend() to lazily create per-instance
connection/session state (see HttpBackend/SftpBackend/MemPathBackend
for the pattern -- a NamedTuple or small class holding a session/client,
propagated to children via with_segments/_make_child_relpath).
FileUri, HttpPath, and SftpPath (src/pathlib_next/uri/schemes/) are
the three built-in worked examples, in increasing order of complexity
(FileUri is ~70 lines wrapping LocalPath; SftpPath adds connection
pooling; HttpPath adds HTML-scraping-based listing and HEAD/GET stat
fallback).
Testing your implementation
To ensure custom implementations comply with pathlib_next's expected behaviors, the library offers three contract levels in pathlib_next.testing which can be mixed into your pytest suite:
PurePathContract: Covers logical pure-path operations (joining, parents, stems, suffix checks, and glob matching) that do not require any physical I/O.ReadPathContract: ExtendsPurePathContractto verify read-only I/O capabilities (such asexists(),is_file(),is_dir(),read_text(),iterdir(), andstat()). This level requires a pre-populatedrootfixture.PathContract: ExtendsReadPathContractto verify full read/write/modify capabilities (such asmkdir(),write_text(),unlink(),rm(),copy(),move(), andtouch()).
Which Contract is Run by Built-in Schemes?
- Full
PathContract: Executed againstLocalPath,MemPath,FileUri, andSftpPath. ReadPathContract: Executed againstDataUri(with directory listing skipped),ZipUri(read-only mode),TarUri, andHttpPath(against a local test HTTP server).- Unit Mocks only:
FtpPath,DavPath, andS3Pathare tested via unit fakes because fully spinning up their real servers locally to satisfy the generic test suite is non-trivial.
Example: Running the full contract
Subclass the appropriate contract with a root fixture providing a writable directory:
import pytest
from pathlib_next.testing import PathContract
class TestMyPath(PathContract):
@pytest.fixture
def root(self, tmp_path):
return MyPath(tmp_path) # an empty, writable directory
See pathlib_next's own tests/test_contract.py for concrete examples wiring these contracts to existing schemes.