Skip to content

Extending

Two equally first-class ways to add a new path-addressable resource. In both, you implement a small, documented method surface; everything else (open/read_text/write_text/glob/walk/touch/rm/copy/move/ exists/is_dir/is_file/...) is derived automatically from the protocols in pathlib_next.protocols.

  • Track A -- subclass Path directly: for any custom path-addressable resource that isn't naturally a URI (e.g. a database-backed virtual filesystem, an archive member, a key-value store). MemPath is the reference exemplar.
  • Track B -- subclass UriPath: for a new URI scheme (http:, sftp:, ...). Registers automatically and gets pure-path parsing (join, query, fragment) for free from Uri.

Whichever track you pick, run the shared contract test suite against your implementation -- see Testing your implementation below.

Track A: subclass Path

Required (pure-path side, from the Pathname ABC):

segments        # property -> sequence of path component strings
parts           # property -> whatever "parts" means for your type
parent          # property -> the logical parent
with_segments(*segments)   # construct a same-type instance from new segments
as_uri()        # a URI string identifying this path (can be a custom scheme)
relative_to(other)          # or raise NotImplementedError if not meaningful

Optional I/O, implement whichever your resource actually supports -- leave the rest as the inherited @notimplemented stubs (derived helpers either fall back, e.g. move() falls back to copy+unlink when rename() isn't implemented, or raise NotImplementedError cleanly):

iterdir()                       # yield child instances
_scandir()                      # optional: yield (name, FileStat|None) pairs
                                 #    instead, if listing your resource can
                                 #    cheaply include stat metadata -- speeds
                                 #    up walk()/glob() (see Track B's
                                 #    "_scandir: listing with metadata" below,
                                 #    which applies here too)
stat(*, follow_symlinks=True)   # -> a FileStatLike (utils.stat.FileStat is a
                                 #    ready-made concrete one)
_open(mode, buffering)          # -> a *binary* IOBase; open()/read_text()/
                                 #    write_bytes()/copy() are all derived
                                 #    from this one method
_mkdir(mode)                    # create just this directory (mkdir() layers
                                 #    parents=/exist_ok= handling on top)
unlink(), rmdir()
rename(target)
chmod(mode, *, follow_symlinks=True)

MemPath (src/pathlib_next/mempath.py) implements exactly this surface over a backend of nested dicts (MemPathBackend; a dict value is a directory, a bytearray value is a file) -- read it end to end as a worked example; it's under 200 lines.

Track B: subclass UriPath

The pure-path side (parsing, join, query/fragment, with_*) comes free from Uri. Register your scheme and implement the I/O surface:

from pathlib_next.uri import UriPath

class MyPath(UriPath):
    __SCHEMES = ("myscheme",)   # name-mangled per-class; redeclare in every
                                 # subclass, don't inherit it

    def _listdir(self):
        ...                      # yield child *names* (str), not instances --
                                  # UriPath.iterdir() wraps each into a child

    def stat(self, *, follow_symlinks=True):
        ...

    def _open(self, mode="r", buffering=-1):
        ...

    def _mkdir(self, mode): ...
    def unlink(self, missing_ok=False): ...
    def rmdir(self): ...
    def rename(self, target): ...
    def chmod(self, mode, *, follow_symlinks=True): ...

Importing the module that defines your subclass is enough to register it (UriPath._schemesmap() walks __subclasses__() and caches the result) -- UriPath("myscheme://host/path") then dispatches to MyPath automatically.

_scandir: listing with metadata

If your remote listing call already returns type/size/mtime for every child in one round trip (an HTML directory index, WebDAV PROPFIND, SFTP listdir_attr, FTP MLSD, an S3 list_objects_v2 page, ...), override _scandir() instead of (or alongside) _listdir():

def _scandir(self):
    for name, meta in my_one_shot_listing_call(self.path):
        yield name, FileStat(st_size=meta.size, st_mtime=meta.mtime,
                              is_dir=meta.is_dir)

UriPath.iterdir() is derived from _scandir() and pre-seeds each child with its FileStat as a single-use hint: the child's first stat() call returns the hint directly (no request), and every call after that re-fetches for real -- so a live mutation is never masked by a stale value. walk()/ glob() then classify directories vs. files from this same hint, turning a remote-tree walk from O(entries) round trips into O(dirs). If you don't override _scandir(), it falls back to _listdir() + one stat() per child (no round-trip savings, but nothing breaks) -- _listdir()/ iterdir() remain fully supported on their own for schemes that have no richer listing call to offer. See HttpPath/DavPath/SftpPath/FtpPath/ S3Path (src/pathlib_next/uri/schemes/) for worked examples.

Optional: override _initbackend() to lazily create per-instance connection/session state (see HttpBackend/SftpBackend/MemPathBackend for the pattern -- a NamedTuple or small class holding a session/client, propagated to children via with_segments/_make_child_relpath).

FileUri, HttpPath, and SftpPath (src/pathlib_next/uri/schemes/) are the three built-in worked examples, in increasing order of complexity (FileUri is ~70 lines wrapping LocalPath; SftpPath adds connection pooling; HttpPath adds HTML-scraping-based listing and HEAD/GET stat fallback).

Testing your implementation

To ensure custom implementations comply with pathlib_next's expected behaviors, the library offers three contract levels in pathlib_next.testing which can be mixed into your pytest suite:

  1. PurePathContract: Covers logical pure-path operations (joining, parents, stems, suffix checks, and glob matching) that do not require any physical I/O.
  2. ReadPathContract: Extends PurePathContract to verify read-only I/O capabilities (such as exists(), is_file(), is_dir(), read_text(), iterdir(), and stat()). This level requires a pre-populated root fixture.
  3. PathContract: Extends ReadPathContract to verify full read/write/modify capabilities (such as mkdir(), write_text(), unlink(), rm(), copy(), move(), and touch()).

Which Contract is Run by Built-in Schemes?

  • Full PathContract: Executed against LocalPath, MemPath, FileUri, and SftpPath.
  • ReadPathContract: Executed against DataUri (with directory listing skipped), ZipUri (read-only mode), TarUri, and HttpPath (against a local test HTTP server).
  • Unit Mocks only: FtpPath, DavPath, and S3Path are tested via unit fakes because fully spinning up their real servers locally to satisfy the generic test suite is non-trivial.

Example: Running the full contract

Subclass the appropriate contract with a root fixture providing a writable directory:

import pytest
from pathlib_next.testing import PathContract

class TestMyPath(PathContract):
    @pytest.fixture
    def root(self, tmp_path):
        return MyPath(tmp_path)   # an empty, writable directory

See pathlib_next's own tests/test_contract.py for concrete examples wiring these contracts to existing schemes.