Skip to content

fsspec-tools

Two libraries for using fsspec filesystems from Python and from your own CLI.

  • vosfs connects fsspec-aware applications to OpenCADC VOSpace.
  • fss-cli supplies POSIX-shaped commands for a Typer application, using the filesystem instances your application provides. See the CLI integration guide.

The VOSpace filesystem

vosfs is an asynchronous fsspec filesystem for the OpenCADC Cavern VOSpace service. It registers the vos protocol so fsspec-aware Python tools — and a bounded set of scientific-stack consumers (pandas, NumPy, Dask, Zarr, PyArrow) — can read, write, inspect, and mutate OpenCADC VOSpace paths.

vosfs targets the OpenCADC VOSpace profile only; it does not claim generic IVOA VOSpace 2.1 conformance.

Install

uv add "git+https://github.com/opencadc/fsspec-tools@main#subdirectory=libs/vosfs"

Authenticated quickstart

Set exactly one credential source. A CADC proxy certificate is a combined certificate-chain and private-key PEM file; use an absolute path because shell shortcuts such as ~ are not expanded by Python libraries.

export VOSFS_CERT_FILE=/absolute/path/to/cadcproxy.pem

The normal synchronous fsspec API is available even though vosfs implements network I/O asynchronously.

import fsspec

fs = fsspec.filesystem("vos", endpoint_url="https://staging.canfar.net/arc")
try:
    print(fs.ls("/home/<cadc-username>"))
finally:
    fs.close()

No credential yet?

A CADC proxy certificate comes from cadc-get-cert. See Getting a credential.

Where to go next

  • The User Guide covers construction, credentials, and the supported and unsupported behavior.
  • Recipes are task-first snippets: read a CSV into pandas, upload a directory, open a FITS file, checkpoint to Zarr.
  • Troubleshooting is organized by the error you saw.
  • The API Reference documents the public interface.
  • fss-cli turns this (or any async fsspec filesystem) into shell-shaped commands inside your own CLI.
  • The normative surface is the capability contract in libs/vosfs/docs/design/trd.md.

To contribute, follow the contributor guide.