feat(storage): add split range disk cache - #6781
Conversation
Based on quickwit-oss#6700. Pin Foyer and add the disabled-by-default cache configuration, including clean-block and write-throughput defaults. Co-authored-by: Cursor <cursoragent@cursor.com>
Based on quickwit-oss#6701. Encode cache keys portably while rejecting only I/O failures and non-UTF-8 URIs. Co-authored-by: Cursor <cursoragent@cursor.com>
Based on quickwit-oss#6702. Build and recover the process-wide cache, flush write-on-eviction entries on close, and apply configured reclaim and write throttling. Co-authored-by: Cursor <cursoragent@cursor.com>
Based on quickwit-oss#6707. Add the storage decorator, manage cache lifecycle in searchers, and route footer and body range reads through Foyer without changing whole-split cache behavior. Co-authored-by: Cursor <cursoragent@cursor.com>
Based on quickwit-oss#6709. Export cache metrics, document production sizing, and disable the long-lived fast-field RAM cache by default when Foyer is configured. Co-authored-by: Cursor <cursoragent@cursor.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 053cb917b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let uri_len = read_u64(reader)? as usize; | ||
| let mut uri_bytes = vec![0; uri_len]; |
There was a problem hiding this comment.
When recovering a truncated or corrupted cache image, the first eight key bytes can decode to an arbitrarily large u64; casting that value and immediately allocating vec![0; uri_len] can exhaust memory or abort the process instead of returning a Foyer decode error so recovery can reject the entry. Validate the conversion and bound the URI length against the encoded entry size before allocating.
AGENTS.md reference: AGENTS.md:L21-L22
Useful? React with 👍 / 👎.
| _buckets: Vec<f64>, | ||
| ) -> BoxedHistogramVec { | ||
| self.register_histogram_vec(name, desc, label_names) |
There was a problem hiding this comment.
When Foyer registers a histogram through register_histogram_vec_with_buckets, this implementation discards the supplied boundaries and creates an ordinary recorder histogram instead. The Prometheus exporter will consequently aggregate these observations using its default buckets rather than Foyer's requested buckets, making the resulting latency and operation distributions inaccurate; propagate the boundaries into the recorder configuration rather than ignoring them.
Useful? React with 👍 / 👎.
| # compression: lz4 | ||
| # recover_mode: quiet | ||
| # block_size: 64M | ||
| # max_entry_size: 60M |
There was a problem hiding this comment.
a full float field will use 8byte per element
We target 10M docs, but we overshoot sometimes, so we may have 12M docs.
So the max size is closer to 100MB
what happens if the size is larger than max_entry_size?
| | `partial_request_cache_capacity` | Partial request in memory cache capacity on a Searcher. Cache intermediate state for a request, possibly making subsequent requests faster. It can be disabled by setting the size to `0`. | `64M` | | ||
| | `max_num_concurrent_split_searches` | Maximum number of concurrent split search requests running on a Searcher. | `100` | | ||
| | `split_cache` | Searcher split cache configuration options defined in the section below. Cache disabled if unspecified. | | | ||
| | `split_range_disk_cache` | Process-wide on-disk cache for exact split footer and body byte ranges. Configuration options are defined in the section below. Cache disabled if unspecified. | | |
There was a problem hiding this comment.
| | `split_range_disk_cache` | Process-wide on-disk cache for exact split footer and body byte ranges. Configuration options are defined in the section below. Cache disabled if unspecified. | | | |
| | `split_range_disk_cache` | Process-wide on-disk cache for split byte ranges. Configuration options are defined in the section below. Cache disabled if unspecified. | | |
|
|
||
| ### Searcher split range disk cache configuration | ||
|
|
||
| This section contains the configuration options for the process-wide on-disk cache of exact split footer and body ranges. The cache is disabled when this section is omitted or set to `null`. If it is set and `fast_field_cache_capacity` is omitted, the long-lived fast field RAM cache is disabled; set a capacity explicitly to keep both. |
There was a problem hiding this comment.
| This section contains the configuration options for the process-wide on-disk cache of exact split footer and body ranges. The cache is disabled when this section is omitted or set to `null`. If it is set and `fast_field_cache_capacity` is omitted, the long-lived fast field RAM cache is disabled; set a capacity explicitly to keep both. | |
| This section contains the configuration options for the process-wide on-disk cache of split byte ranges. The cache is disabled when this section is omitted or set to `null`. If it is set and `fast_field_cache_capacity` is omitted, the long-lived fast field RAM cache is disabled; set a capacity explicitly to keep both. |
| /// Disabled-by-default searcher disk cache for exact split byte ranges. | ||
| #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] | ||
| #[serde(deny_unknown_fields)] | ||
| pub struct SplitRangeDiskCacheConfig { |
There was a problem hiding this comment.
I would avoid putting to many foyer specific parameters in the config. We can have defaults and override them via env parameters instead.
| fn default() -> Self { | ||
| SearcherConfig { | ||
| fast_field_cache: CacheConfig::default_with_capacity(ByteSize::gb(1)), | ||
| fast_field_cache: None, |
There was a problem hiding this comment.
why did you change the default here?
| NonZeroU64::new(30).unwrap() | ||
| } | ||
|
|
||
| /// Long-lived `.fast` RAM cache after applying defaults. |
There was a problem hiding this comment.
why long-lived? do we have a short-lived RAM cache?
| split_and_footer_offsets: &SplitIdAndFooterOffsets, | ||
| ) -> anyhow::Result<(OwnedBytes, BundleStorage)> { | ||
| let split_file = PathBuf::from(format!("{}.split", split_and_footer_offsets.split_id)); | ||
| let foyer_storage: Arc<dyn Storage> = match &searcher_context.split_range_disk_cache_opt { |
There was a problem hiding this comment.
wrong variable name, the None case is not foyer
| ) -> anyhow::Result<(OwnedBytes, BundleStorage)> { | ||
| let split_file = PathBuf::from(format!("{}.split", split_and_footer_offsets.split_id)); | ||
| let foyer_storage: Arc<dyn Storage> = match &searcher_context.split_range_disk_cache_opt { | ||
| Some(cache) => wrap_storage_with_split_range_cache(cache.clone(), index_storage.clone()), |
There was a problem hiding this comment.
| Some(cache) => wrap_storage_with_split_range_cache(cache.clone(), index_storage.clone()), | |
| Some(cache) => wrap_storage_with_disk_cache(cache.clone(), index_storage.clone()), |
| pub predicate_cache: Arc<PredicateCacheImpl>, | ||
| /// Search split cache. `None` if no split cache is configured. | ||
| pub split_cache_opt: Option<Arc<SearchSplitCache>>, | ||
| /// Process-wide split range disk cache. `None` if not configured. |
There was a problem hiding this comment.
| /// Process-wide split range disk cache. `None` if not configured. | |
| /// Split range disk cache. `None` if not configured. |
Process-wide implies it's different, but it's not
| let storage_long_term_cache = | ||
| Arc::new(QuickwitCache::new(&searcher_config.fast_field_cache)); | ||
| let fast_field_cache = searcher_config.resolved_fast_field_cache(); | ||
| let storage_long_term_cache = if fast_field_cache.capacity().as_u64() == 0 { |
There was a problem hiding this comment.
this check should be in QuickwitCache::new
| Some(config) => Some(Arc::new( | ||
| FoyerSplitRangeCache::open(config) | ||
| .await | ||
| .context("failed to open searcher split range disk cache")?, |
There was a problem hiding this comment.
when will this happen? data corruption?
we probably want a way to safeguard against corruption or format changes, and clear the cache instead of rendering it unusable
| /// Manual `Code` impl: keep Foyer's serde feature off so bincode is not pulled | ||
| /// in for keys or values. Foyer 0.22.3 already implements `Code` for `Bytes`, | ||
| /// so only this key needs a codec. Enabling serde would encode each value byte | ||
| /// as an integer; for a 15 MiB payload that is millions of serializer visits |
There was a problem hiding this comment.
and that's a real performance problem?
| }; | ||
| pub use storage::{FoyerSplitRangeStorage, wrap_storage_with_split_range_cache}; | ||
|
|
||
| /// Process-wide Foyer hybrid cache for exact split byte-range payloads. |
There was a problem hiding this comment.
| /// Process-wide Foyer hybrid cache for exact split byte-range payloads. | |
| /// Foyer hybrid cache for exact split byte-range payloads. |
2 participants
Summary
Test plan
make fmtcargo test -p quickwit-config --lib node_config::tests::test_split_range_disk_cache -- --nocapturecargo test -p quickwit-storage --all-features --lib split_range_cache -- --nocapturecargo test -p quickwit-search --all-features --lib split_range_cache_layer -- --nocapturecargo clippy -p quickwit-config -p quickwit-storage -p quickwit-search --all-features --tests -- -D warningsMade with Cursor