[Utils] Add get_cached_repo_tree utility - #4513
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| return [ | ||
| RepoFile(path=path, size=entry.size, oid=entry.blob_id, xetHash=entry.xet_hash) | ||
| for path, entry in tree_entries.items() | ||
| ] |
There was a problem hiding this comment.
LFS metadata omitted from RepoFile
Medium Severity
get_cached_repo_tree builds each RepoFile from TreeCacheEntry with path, size, oid, and xetHash only. Cached lfs_sha256 and lfs_size are never passed into RepoFile, so LFS-tracked files always have lfs=None despite the tree cache storing that metadata when snapshot_download runs.
Reviewed by Cursor Bugbot for commit b907c2f. Configure here.
There was a problem hiding this comment.
on purpose, not needed in practice
let's add them in the future if they are really needed. path + size + xet_hash is already enough
get_cached_repo_tree utility
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f10a15f. Configure here.
|
This PR has been shipped as part of the v1.23.0 release. |


Follow-up PR after #4394.
This PR adds a utility to retrieve the cached repo tree from snapshot_download. First mentioned in huggingface/diffusers#14118 (comment). Should help
diffusersintegration to define the structure of repo (with its variants, etc.) even in offline mode. cc @sayakpaulSummary
Adds a power-user helper
get_cached_repo_tree(repo_id, repo_type=..., revision=..., cache_dir=...)that returns the repo's cached/treelisting (populated as a side effect ofsnapshot_download) as alist[RepoFile], without any network call.HfApi.list_repo_tree: it always returns the full list,RepoFileonly (noRepoFolder), and has norecursive/expand/path/... options — everything cached is returned.errors.CachedRepoTreeNotFoundErrorwhen nothing is cached for the requested revision.Example
Note
Low Risk
Additive public API over existing on-disk tree cache; read-only local filesystem access with clear errors when cache is missing.
Overview
Adds
get_cached_repo_tree, a read-only helper that returns the full cached repo file tree aslist[RepoFile]from thetrees/<commit_hash>.jsondata written bysnapshot_download, with no Hub request. Revisions can be commit hashes or branch/tag names resolved via cachedrefs/;local_dirreads trees fromlocal_dir/.cache/huggingface/while still resolving refs from the main cache.Introduces
CachedRepoTreeNotFoundErrorwhen no ref or tree cache exists, documents both in the package reference, exportsget_cached_repo_treefrom the public API, and adds unit tests intest_tree_cache.py. CLI repo-create tests expectspace_template=Noneoncreate_repo(unrelated assertion fix).Reviewed by Cursor Bugbot for commit 6557128. Bugbot is set up for automated code reviews on this repo. Configure here.