Skip to content

Add estore stats #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: redesign-expire
Choose a base branch
from

Conversation

sggeorgiev
Copy link

Overview

Adds detailed per-estore statistics to help visualize and monitor the internal structure and distribution of items across ebuckets. These metrics provide insight into how data cascades across different TTL tiers (L1/L2/L3), aiding both debugging and TTL expiry tuning.

Changes

  • estoreGetStats(): Collects and formats structural statistics for a given estore, with support for both standalone and clustered modes
  • estoreCombineStats(): Aggregates statistics from multiple ebuckets instances
  • ebGetStats(): Walks internal structures (stack, rax, list) and computes metrics like total items, segments, and level distribution
  • ebGetStatsMsg(): Formats stats into human-readable text for use in CLI/debug output

Key Metrics Collected

  • total items: Total keys stored
  • total buckets: Total hash buckets in use
  • total segments: Number of storage segments allocated
  • total items in L1/L2/L3: Distribution across TTL cascade levels
  • avg items per bucket: Data distribution efficiency
  • avg items per segment: Segmentation density
  • avg segments per bucket: Storage fragmentation indicator

Test Coverage

  • ✅ estoreCombineStats() correctly aggregates and computes averages
  • ✅ Handles edge case where totalBuckets == 0 (skips avg calculations)
  • ✅ estoreGetStats() works in non-clustered mode, including L1/L2/L3 splits
  • ✅ estoreGetStats() works in clustered mode, combining all bucket stats
  • ✅ CLI output verified to contain all expected fields and values

Use Cases

  • Cascade Monitoring: See how items are distributed across L1 (short TTL), L2 (medium TTL), and L3 (long TTL) tiers
  • Expiration Behavior Analysis: Understand how TTLs shape storage patterns over time
  • Debugging & Profiling: Identify imbalances or fragmentation in bucket and segment usage
  • Capacity Planning: Track how data structures scale and adapt under different workloads

Example Usage (CLI)

127.0.0.1:6379> set test1 test ex 1000
OK
127.0.0.1:6379> set test2 test ex 6000
OK
127.0.0.1:6379> set test3 test ex 2000000000000
OK
127.0.0.1:6379> debug HTSTATS 0
[Dictionary HT]
Hash table 0 stats (main hash table):
 table size: 4
 number of elements: 3
[EBUCKETS]
Ebuckets stats:
 total items: 3
 total buckets: 2
 total segments: 2
 total items in L1: 1
 total items in L2: 1
 total items in L3: 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant