Data commands¶

CANFAR Data beta

The canfar data commands and Python storage tools in canfar.storage have been available in beta since client v1.5.0. Your existing storage permissions still apply. Share your experience or get help.

Use canfar data to copy files between your computer and CANFAR storage, inspect remote files, and manage directories. Each path starts with a storage name: local means the computer running the command; configured names such as arc and vault refer to remote services.

These commands were added after v1.4.1. Check the installation guide before following the examples. For transfers without installing Python, use Storage Management in your browser.

Sources and operands¶

Run canfar login for the identity provider (IDP) that owns the remote service. Default arc and vault storage uses CADC credentials (canfar login cadc), even if your active compute server belongs to SRCNet. Then use a storage name followed by an absolute path:

storage-identifier:/absolute/path
local:/absolute/path

The default CADC configuration provides arc, vault, and local when its default Server record is present:

canfar data ls -lh arc:/home/user
canfar data ls -lh vault:/
canfar data ls -lh local:/tmp

Storage Identifiers are configuration keys, not protocols. There is no active:/ source, bare local-path shorthand, empty :/path source, or canfar storage command: name files on your computer local:/PATH, not /PATH. A data command sees all configured sources, not only the active Server Selection. Each data command's --help ends with this syntax and the configured identifiers, and a usage error caused by a bare local path adds a local:/PATH hint.

Commands¶

Use canfar data --help for the installed upstream options. The available commands are:

Command	Purpose
`basename`, `dirname`	Transform a path string.
`info`, `size`, `stat`, `test`	Inspect a file or evaluate a predicate.
`ls`, `ll`	List directory contents; `ls` accepts `-A`, `-l`, and `-h`, while `ll` accepts `-A` and `-h`.
`du`	Estimate file space usage; supports `-s` and `-h`.
`find`, `tree`	Traverse recursively; `find` supports `--maxdepth` and `--type f\|d`, while `tree` supports `--maxdepth`.
`head`, `tail`, `cat`	Read leading bytes, trailing bytes, or file contents.
`cp`	Copy files or one directory; `-R`/`-r` enables recursive copy.
`mv`	Move or rename files on one mapped filesystem.
`mkdir`, `rmdir`, `unlink`, `rm`	Create directories, remove empty directories, remove one file, or remove files.

Recursive removal is disabled by CANFAR policy, so data rm has no -R or -r option. data mv does not implement a cross-source move; copy between sources and verify the destination before removing the source separately.

Examples¶

List and inspect a remote object:

canfar data ls -lh arc:/home/user
canfar data info arc:/home/user/file.fits
canfar data cat arc:/home/user/file.fits

Copy between local and remote sources:

canfar data cp local:/tmp/file.fits arc:/home/user/file.fits
canfar data cp arc:/home/user/file.fits local:/tmp/file.fits

A single-file copy always transfers the file, even when the destination already holds identical content; only a rerun recursive copy skips verified files. A copy between two storage identifiers, such as vault: to arc:, downloads the file to a temporary file on the computer running the command and then uploads it, so that computer needs room for the file. A copy within one identifier uses that service's own copy.

Recursive copy is available for one directory and its descendants:

canfar data mkdir -p arc:/home/user
canfar data cp -R local:/tmp/dataset arc:/home/user

An existing destination directory receives the source directory by name, so this example writes arc:/home/user/dataset. Rerun the same command against the same existing parent directory to resume a partial copy. Matching files are skipped only after content checksums or byte comparison confirm equality; files with different contents are replaced. Without compatible checksums, verification can download both copies even when no upload is needed.

For a cross-source move, make the verification and removal explicit:

canfar data cp vault:/folder/file.fits arc:/home/user/file.fits
canfar data info arc:/home/user/file.fits
canfar data rm vault:/folder/file.fits

Data command stdout belongs to the embedded command. CANFAR does not prepend the active-Server banner or add a JSON/YAML envelope, and data commands do not provide the CANFAR -o/--output option.

For Python access, explicit Storage Identifier resolution and cache guidance are documented in Data Access.