httsps://pivot-test.solidproject.org:3200 a clean v7.2.0 CSS server crashes on a 30mn pattern
The server uses
- 30s lock file
- 10 mn clear lock
The 30 mn comes from the timer
This is the IA analysis
This confirms the relationship you suspected between the 10-minute sweep and the crash. Updated analysis:
Revised sequence of events
12:40:27–28 — The 10-minute WrappedExpiringStorage sweep (removeExpiredEntries, your timeout: 10) runs. It needs a lock on /.internal/accounts/index/owner/ to iterate/prune, but that lock takes >30s to acquire (production disk under load from many pods), so it errors. This one is safe — caught by setSafeInterval's try/catch, just logged as error, no crash. Server keeps serving GETs fine afterward.
12:41:07 — Roughly 30–40s later, four separate /accounts/data/<uuid> locks expire almost simultaneously. This is the real crash trigger, and it lines up with my earlier finding: unrelated createAccountTimeout 30-minute cleanup timers (in BaseLoginAccountStorage, unprotected by try/catch) fire in a burst because many accounts were created in the same time window during seeding. This is the one that's unhandled and kills the process.
The causal link (your instinct was right)
The 10-minute sweep isn't crashing directly, but it contends for the same lock backend (accounts/index/owner, part of the shared AccountStorage/IndexedStorage). While it holds/retries that lock:
- Any other request needing a lock on a related resource (including the four concurrent account-data deletes from the unrelated 30-min timers) queues up behind it in
FileSystemResourceLocker's infinite-retry acquire loop.
- If several of those queued operations don't get their lock within their own 30s
WrappedExpiringReadWriteLocker window, they all expire at once — a small "thundering herd" — and since the account-timeout path has no try/catch, the first one to reject is enough to crash the process.
So: 10-min sweep → lock contention spike on shared index storage → concurrent unrelated 30-min account-cleanup timers also miss their 30s deadline → unhandled rejection → crash. The two features interact through lock contention on shared backend storage, not through calling each other directly.
Practical implications
- Increasing the sweep interval (say, back toward default 60 min, or higher) reduces how often this contention window opens.
- Increasing lock
expiration (30s → 60-120s) gives contended operations more room before they trip.
- The actual crash-causing bug remains the unprotected
createAccountTimeout callback in BaseLoginAccountStorage — that's the one that needs a try/catch to stop being fatal, regardless of tuning the timers.
httsps://pivot-test.solidproject.org:3200 a clean v7.2.0 CSS server crashes on a 30mn pattern
The server uses
The 30 mn comes from the timer
This is the IA analysis