@@ -1,5 +1,51 @@
 # Changelog for oda_reader
 
+## Unreleased
+
+- **Adds `download_crdf()`, `download_crdf_file()` and `list_crdf_files()`** for the OECD
+  Climate-Related Development Finance workbooks at `webfs.oecd.org/climate/`, read over plain
+  HTTP from either perspective. File names come from the live folder listing, and each year is
+  read from the narrowest workbook that holds it. Every header layout, current and archived,
+  maps to one set of columns through `schemas/mappings/crdf_schema.json`, with `price_year`,
+  `vintage`, `source_file`, `file_version` and `file_version_kind` added to each row.
+  `file_version` is the file host's validator for the cached copy read, and `file_version_kind`
+  says whether it is an `etag` or a `last_modified` date. Each workbook is read from a private
+  snapshot of the cached copy, so a concurrent refresh cannot change the rows under that label. An unknown layout raises
+  `CRDFSchemaError`, which lists the unmapped and missing columns. A folder listing entry that is
+  not a plain file name also raises `CRDFSchemaError`.
+- **Adds `register_bulk_file()` for bulk files downloaded in a browser.** It validates the file,
+  copies it into the bulk file cache under the key a download would use, and returns the cached
+  path. A registered entry never expires, so the TTL sweep never deletes it. Online reads serve it
+  while the file host's version token matches the one recorded, or cannot be read, and download
+  a republished file when the token changes. `dataset` takes `"crs"`, `"crs-reduced"`,
+  `"dac1"`, `"dac2a"`, `"dac2b"` or `"multisystem"`. Bulk files download whole, a single-byte
+  request reads the version token first, and a renamed file still resolves: online reads take
+  the name from the SDMX dataflow annotations and offline reads find the copy by dataset.
+- **Adds `offline=True` to `bulk_download_crs()`, the DAC1, DAC2a, DAC2b and multisystem bulk
+  readers, and `bulk_download_parquet()`.** The reader makes no network request and reads the
+  cached or registered copy whatever its age. With nothing cached it raises the new
+  `BulkFileNotCachedError`, which prints the `register_bulk_file` call. A CRS year file
+  registers with `url=` and reads offline through `bulk_download_parquet(url=..., offline=True)`.
+  When two cached copies match a dataset, the one stored last is read, even within the same second.
+- **`BulkDownloadChallengeError` prints the fix.** Its message names the file to download in a
+  browser and the `register_bulk_file` call that registers it, including the version token the
+  reader saw. The error gains `filename`, `dataset` and `version` attributes.
+- **Bulk downloads honour `Retry-After` on 429 and 503.** A wait of up to 300 seconds is taken
+  and the request retried, up to three times. A longer wait, a missing header or a fourth
+  throttled response raises `BulkDownloadHTTPError`, which now carries `retry_after`.
+- **Adds `CRS_BULK_LABEL` and `CRS_REDUCED_BULK_LABEL`** to `oda_reader.crs`. `CacheManager`
+  gains `register()`, `find_entry()` and `snapshot()`, and `list_records()` reports each entry's `source` and
+  `url`.
+- **The bulk-file cache keeps a dataset's only copy past its TTL.** In 1.11.4 the startup sweep
+  deleted every downloaded file past its TTL, including a dataset's only copy, which a scripted
+  download may not be able to replace. The sweep now deletes an expired download only when a newer
+  copy of the same dataset is cached. An only copy stays: an online read refetches it and an offline read uses it.
+  A file downloaded by URL alone stays until a download replaces it or the cache is cleared.
+- **A cache hit fills in the URL and dataset a cached record lacks**, so a file downloaded before
+  they were recorded becomes readable with `offline=True`.
+- **A bulk URL with spaces and its `%20` form share one cache entry.** Registration, downloads
+  and offline reads all key the cache on the percent-encoded URL.
+
 ## 1.11.4 (2026-10-21)
 
 - **The bulk-file cache keeps every file it downloads until its TTL.** It used to keep only the
 
@@ -23,6 +23,25 @@ For large-scale analysis, bulk downloads are faster and more reliable than repea
 - Exploratory analysis with changing queries
 - You only need recent data
 
+## How ODA Reader fetches bulk files
+
+OECD publishes the bulk files on `webfs-dcd.oecd.org`. ODA Reader downloads each file whole,
+keeps a local copy in the [bulk file cache](caching.md#bulk-file-cache) and answers every query
+from that copy. It never reads a bulk file remotely.
+
+Before a download, ODA Reader sends one single-byte request to read the file's version token
+(its `ETag`, or `Last-Modified` when no `ETag` is sent). It downloads the file only when it has
+no copy, when the token has changed, or when the copy is past its 90-day TTL. Otherwise it reads
+the copy it has. See [Cache invalidation on republish](caching.md#cache-invalidation-on-republish).
+
+OECD may change bulk file names, for example by adding a timestamp. ODA Reader does not hard-code
+them. On each online read it looks up the current file name in the SDMX dataflow annotations,
+and an offline read finds the cached copy by dataset, whatever name it was stored under.
+
+A scripted request can meet a Cloudflare challenge; see
+[Cloudflare challenge](#cloudflare-challenge-bulkdownloadchallengeerror). The supported route
+then is to download the file in a browser and pass it to `register_bulk_file`.
+
 ## CRS bulk downloads
 
 The full Creditor Reporting System dataset is available as a bare parquet file (1.18 GB, not zipped). ODA Reader can download and load it for you.
@@ -450,20 +469,71 @@ See [DAC2b bulk download](#dac2b-bulk-download) above for the full column list a
 
 **Slow download**: Bulk downloads depend on OECD's file server speed. Try again later if slow. Once downloaded, files are cached. Each attempt has a 10-second connect / 60-second read timeout, so a stalled connection fails with an error.
 
-### Cloudflare challenge (`BulkDownloadChallengeError`)
+**HTTP 429 or 503**: When the file server answers with a `Retry-After` of 300 seconds or less, ODA Reader waits that long and tries again, up to three times. A longer wait, a missing `Retry-After`, or a fourth throttled response raises `BulkDownloadHTTPError`, and its `retry_after` attribute holds the wait the server asked for in seconds.
 
-Cloudflare can require an interactive browser check on an OECD bulk-file
-request. These checks require browser execution. ODA Reader raises
-`BulkDownloadChallengeError` and includes the `cf_ray` value supplied by
-OECD/Cloudflare. Include that value when reporting the failure to OECD.
+### Cloudflare challenge (`BulkDownloadChallengeError`)
 
 OECD supports downloading the bulk files in a browser. Scripted downloads,
 including the ones ODA Reader makes, are not a service OECD has committed
 to, so OECD's file server can challenge them at any time. A pipeline that
 must run unattended should be ready to handle `BulkDownloadChallengeError`.
 
-When bulk access is unavailable, choose a narrowly filtered API request
-explicitly.
+When ODA Reader meets a challenge it raises `BulkDownloadChallengeError`. Its message names
+the file and prints the call that registers a copy downloaded in a browser:
+
+```text
+Cloudflare presented an interactive challenge for https://webfs-dcd.oecd.org/.../CRS.parquet.
+Open that URL in a browser to download CRS.parquet, then register the file so
+oda-reader reads it from the cache:
+
+    import oda_reader
+    oda_reader.register_bulk_file('~/Downloads/CRS.parquet', dataset='crs', url='https://webfs-dcd.oecd.org/.../CRS.parquet', version='"5f1c..."')
+```
+
+The error also carries these values as attributes: `url`, `filename`,
+`dataset` (set when a named reader such as `bulk_download_crs()` made the
+request), `version` (the host's token, when it could be read) and `cf_ray`.
+
+#### Register a file downloaded in a browser
+
+Open the URL in a browser, let it save the file, then register it:
+
+```python
+import oda_reader
+
+oda_reader.register_bulk_file(
+    "~/Downloads/CRS.parquet", dataset="crs", url="<url from the error>"
+)
+crs = oda_reader.bulk_download_crs()  # reads the registered copy
+```
+
+`register_bulk_file` checks that the file is a valid zip or parquet payload,
+copies it into the bulk file cache and returns the cached path. You can
+delete the download afterwards. The registered copy does not expire. Online
+reads keep serving it until OECD publishes a new version of the file.
+`bulk_download_crs(offline=True)` reads it with no network request. A CRS year file registers with `url=` and reads
+offline through `bulk_download_parquet(url=..., offline=True)`. See
+[Registered files](caching.md#registered-files) and
+[Reading offline](caching.md#reading-offline).
+
+`dataset` takes one of these names:
+
+| `dataset` | Reader |
+|---|---|
+| `"crs"` | `bulk_download_crs()` |
+| `"crs-reduced"` | `bulk_download_crs(reduced_version=True)` |
+| `"dac1"` | `bulk_download_dac1()` |
+| `"dac2a"` | `bulk_download_dac2a()` |
+| `"dac2b"` | `bulk_download_dac2b()` |
+| `"multisystem"` | `bulk_download_multisystem()` |
+
+Without `url`, ODA Reader looks the URL up in OECD's SDMX metadata, which
+needs network access to `sdmx.oecd.org`. `version` is the file host's
+version token. The call the error prints includes it when the reader could
+read it.
+
+If the browser download is also blocked, report the `cf_ray` value to OECD
+and use a narrowly filtered API request instead.
 
 ```python
 from oda_reader import download_dac1
@@ -495,8 +565,8 @@ dimension.
 
 **`BulkPayloadCorruptError`**: The OECD's bulk endpoint occasionally serves a
 truncated or malformed file (a bad zip, or a bare parquet file that fails to
-parse). The corrupt entry is removed automatically before the exception is
-raised, so the next call cleanly re-downloads. Retry the call, or pass
+parse). The corrupt download is deleted before the exception is raised and
+never replaces the cached copy, so the next call cleanly re-downloads. Retry the call, or pass
 `use_raw_cache=False` to skip the cache for that invocation.
 
 ## Next steps
 
@@ -145,28 +145,37 @@ earlier versions stays on disk.
 
 ### Bulk file cache
 
-The bulk file cache (used by `bulk_download_crs`, `download_crs_file`,
-`bulk_download_dac1`, `bulk_download_dac2a`, `bulk_download_dac2b`, and `bulk_download_multisystem`)
-is governed separately
-because the files are large (up to 1.18 GB each):
-
-- **Expiry**: every bulk file you download stays cached until its TTL
-  below runs out, so disk use grows with the number of distinct files you
-  fetch. Expired files are deleted the first time a Python session makes a
-  bulk download call.
+The bulk file cache is used by `bulk_download_crs`, `download_crs_file`, `bulk_download_dac1`,
+`bulk_download_dac2a`, `bulk_download_dac2b`, `bulk_download_multisystem` and `download_crdf`.
+ODA Reader downloads each bulk file whole, keeps the copy here and answers queries from it. It
+never reads a bulk file remotely. A scripted request can meet a Cloudflare challenge; see
+[Cloudflare challenge](bulk-downloads.md#cloudflare-challenge-bulkdownloadchallengeerror). A
+browser download registered with `register_bulk_file` then fills the cache (see
+[Registered files](#registered-files)).
+
+The cache is governed separately because the files are large (up to 1.18 GB each):
+
+- **Expiry**: a downloaded bulk file past its TTL below is deleted the first
+  time a Python session makes a bulk download call, but only once a newer
+  copy of the same dataset is cached. A dataset's only copy stays: an online
+  read refetches it and an offline read uses it. A file downloaded by URL
+  alone, with no dataset, stays until a download replaces it or you clear
+  the cache. Registered files never expire (see
+  [Registered files](#registered-files)).
 - **Publication-version invalidation**: a cached file is refetched as soon
   as OECD republishes it, not just after the TTL below expires. See
   [Cache invalidation on republish](#cache-invalidation-on-republish).
 - **Per-entry TTL**: 90 days. A file older than that is refetched even if
   OECD hasn't republished it, which covers the cases where the
   publication-version check above couldn't be made (offline, server error,
-  or the relevant headers absent).
+  or the relevant headers absent). Registered files have no TTL.
 - **Integrity validation**: every freshly downloaded payload is end-to-end
   checked before being trusted (a zip CRC walk, or a footer-magic check for
   the bare-parquet CRS files). Truncation is also caught earlier, during the
   download itself: streamed bytes are compared against the response's
   `Content-Length` and a mismatch is retried before it ever reaches the
-  cache. A corrupt download is removed from the cache and raises
+  cache. A corrupt download is deleted before it replaces anything, so a
+  copy already in the cache stays readable, and raises
   `BulkPayloadCorruptError` so you can simply retry. Cached files are
   trusted on hit (no recheck on every call).
 - **Self-healing**: temp files left behind by interrupted downloads (older
@@ -201,8 +210,70 @@ recently it was fetched. The token comes from the file host:
   still works offline. A blocked request (for example a Cloudflare
   challenge) logs a warning naming the HTTP status or Cloudflare Ray ID.
 
-You don't need to call anything for this. It runs automatically on every
-bulk download.
+A download happens only when the cache has no copy, the token has changed, or the copy is past
+its TTL. You don't need to call anything for this. It runs automatically on every bulk download.
+
+OECD may change bulk file names, for example by adding a timestamp. Each online read looks up the
+current name in the SDMX dataflow annotations, so a renamed file is found without an ODA Reader
+update (see [Stale dataflow metadata](#stale-dataflow-metadata) for how a cached annotation that
+names the old file is handled). An offline read finds the cached copy by dataset, whatever name it was stored under.
+
+#### Registered files
+
+`register_bulk_file()` puts a file you downloaded yourself into the bulk
+file cache, for when OECD's Cloudflare front end blocks the download (see
+[Cloudflare challenge](bulk-downloads.md#cloudflare-challenge-bulkdownloadchallengeerror)):
+
+```python
+import oda_reader
+
+oda_reader.register_bulk_file(
+    "~/Downloads/CRS.parquet", dataset="crs", url="<url from the error>"
+)
+```
+
+A registered file is stored under the same key a download of that URL
+would use. Online reads treat it like any cached file: the reader resolves
+the URL, reads the host's version token, and serves the registered copy
+when the token equals the one recorded at registration or cannot be read.
+A different token means OECD has republished, so the reader downloads the
+new file and it replaces the registration.
+
+The recorded token is the `version` you pass. The challenge error prints
+the token the reader saw just before the blocked download, so copying its
+call records the right one. Without `version`, `register_bulk_file` asks
+the host for the token itself, and records none if that request fails too.
+With no token recorded, the registered copy is served until a token
+request succeeds, which triggers a download.
+
+A registered entry has no TTL, so the 90-day expiry never removes it. It
+is replaced only by another registration, a completed
+download of the same file, or a cache clear such as
+`oda_data.cache.clear('raw')` or `bulk_cache_manager().clear()`.
+`bulk_cache_manager().list_records()` reports `source="registered"` for a
+registered entry and `source="download"` for the rest.
+
+#### Reading offline
+
+`bulk_download_crs`, the DAC1, DAC2a, DAC2b and multisystem bulk readers, and
+`bulk_download_parquet` take `offline=True`. The reader then makes no network
+request, neither the URL lookup nor the version-token request, and reads whatever copy of that dataset the cache holds, registered or
+downloaded, however old:
+
+```python
+from oda_reader import bulk_download_crs
+
+crs = bulk_download_crs(offline=True)
+```
+
+With nothing cached it raises `BulkFileNotCachedError`, which prints the
+`register_bulk_file` call to run. Offline reads find a registered file by
+its `dataset`, so register with `dataset` when you plan to read offline. A
+CRS year file has no `dataset` name. Register it with `url=` and read it
+offline through `bulk_download_parquet(url=..., offline=True)`.
+
+Set `ODA_READER_CACHE_DIR` before registering to put the file in a cache
+other than the default one.
 
 #### Bypassing the bulk file cache
 
@@ -245,17 +316,18 @@ call anything to get this behaviour. It runs automatically for
 #### Handling corrupt downloads
 
 The OECD's bulk endpoint occasionally serves a truncated or malformed file.
-When that happens, a `BulkPayloadCorruptError` is raised and the bad entry is
-already removed from disk by the time you see it, so the next call cleanly
-re-downloads:
+When that happens, a `BulkPayloadCorruptError` is raised and the bad download is
+already removed from disk by the time you see it. Any copy the cache already
+held, including a registered file, stays in place with its record, and the
+next call cleanly re-downloads:
 
 ```python
 from oda_reader import bulk_download_crs, BulkPayloadCorruptError
 
 try:
     crs = bulk_download_crs()
 except BulkPayloadCorruptError:
-    # Bad entry already removed — just retry
+    # Bad download already removed, just retry
     crs = bulk_download_crs()
 ```
 
@@ -377,7 +449,7 @@ from oda_reader import download_dac1
 data = download_dac1(start_year=2022, end_year=2022)  # instant
 ```
 
-Cache persists until you clear it. HTTP responses expire after 7 days and bulk files after 30.
+Cache persists until you clear it. HTTP responses expire after 7 days. Bulk files are refetched after 90 days.
 
 ### Filter aggressively to reduce API load