atompack.hub¶
Classes¶
Read-only view over one atompack file or a lexicographically ordered shard set. |
Functions¶
|
Download a single .atp file or shard directory from a Hugging Face repo. |
|
Upload a local .atp file or shard directory to a Hugging Face repo. |
|
Download atompack data from Hugging Face and return a read-only reader. |
|
Open a local .atp file or shard directory as a read-only reader. |
Module Contents¶
- class atompack.hub.AtompackReader(paths: Sequence[pathlib.Path], databases: Sequence[atompack._atompack_rs.PyAtomDatabase])[source]¶
Read-only view over one atompack file or a lexicographically ordered shard set.
- atompack.hub.download(repo_id: str, path_in_repo: str | None = None, *, revision: str | None = None, repo_type: str = 'dataset', token: str | bool | None = None, cache_dir: str | pathlib.Path | None = None, local_dir: str | pathlib.Path | None = None, local_files_only: bool = False, force_download: bool = False) pathlib.Path[source]¶
Download a single .atp file or shard directory from a Hugging Face repo.
- atompack.hub.upload(source: str | pathlib.Path, repo_id: str, path_in_repo: str | None = None, *, repo_type: str = 'dataset', revision: str | None = None, token: str | None = None, commit_message: str | None = None, create_repo: bool = True, private: bool = False, use_xet: bool | None = None) Any[source]¶
Upload a local .atp file or shard directory to a Hugging Face repo.
By default, large single-shard uploads temporarily disable Xet so the Hub can use multipart LFS upload, which is typically much faster for large compressed .atp files. Pass use_xet=True to force Xet, or use_xet=False to disable it explicitly.
- atompack.hub.open(repo_id: str, path_in_repo: str | None = None, *, revision: str | None = None, repo_type: str = 'dataset', token: str | bool | None = None, cache_dir: str | pathlib.Path | None = None, local_dir: str | pathlib.Path | None = None, local_files_only: bool = False, force_download: bool = False) AtompackReader[source]¶
Download atompack data from Hugging Face and return a read-only reader.
- atompack.hub.open_path(source: str | pathlib.Path) AtompackReader[source]¶
Open a local .atp file or shard directory as a read-only reader.