fix(reddit): prevent enrichment timeout from discarding all search results

[thinkun]

Summary

• Bug: When ScrapeCreators comment enrichment is slow, search_and_enrich() exceeds the caller's 90-second ThreadPoolExecutor timeout. The resulting TimeoutError discards all search results — even though the search phase already completed successfully (e.g. 116 posts found and deduped). The user sees ✓ Reddit Found 0 threads despite logs showing 116 posts were found.
• Fix: Run enrich_with_comments() in a dedicated thread with its own 45-second timeout. If enrichment times out or fails, search results are returned without comments instead of being thrown away. Backward-compatible — no caller changes needed.

How to reproduce

1. Run /last30days with a broad topic that returns many Reddit results (e.g. "best local AI models across different use cases")
2. If ScrapeCreators comment-fetch latency is high (network conditions, API load), the enrichment phase for 5 posts can take >45 seconds
3. The search phase (~30s) + enrichment (~60s) exceeds the 90s reddit_future.result(timeout=90) deadline in last30days.py
4. All Reddit results are silently discarded — output shows **ERROR:** Reddit search timed out after 90s and Reddit: 0 threads

Root cause

search_and_enrich() ran enrich_with_comments() synchronously in the same call stack as the search. There was no timeout boundary between the two phases, so slow enrichment consumed the entire budget:

0s   — search starts (ScrapeCreators API)
~30s — search completes: 116 posts ✓
~30s — enrichment starts: fetching comments for top 5 posts
~90s — ⚡ caller's 90s timeout fires DURING enrichment
       └─ TimeoutError raised
       └─ reddit_items stays as initialized [] 
       └─ ALL 116 posts discarded

Changes

scripts/lib/reddit.py — search_and_enrich()

• Enrichment now runs in a ThreadPoolExecutor(max_workers=1) with a 45-second timeout
• On TimeoutError or any exception: search results are preserved, a log message is emitted
• New optional enrich_timeout parameter (default 45s, set to 0 to skip enrichment)
• No changes to any callers required

Test plan

• Run /last30days with a topic that triggers Reddit search — verify Reddit results appear in output
• Simulate slow enrichment (e.g. add time.sleep(60) at top of enrich_with_comments) — verify search results are returned without comments and log shows timeout message
• Verify enrich_timeout=0 skips enrichment entirely
• Verify normal fast enrichment still attaches comments as before

🤖 Generated with Claude Code

[DanRWilloughby]

Hit this exact issue running research in autonomous agent pipelines — logs show 100+ posts found, output says 0. The two-phase approach (return search results even if enrichment times out) is the right fix. Would love to see this merged.

[sudhakrishnan388-art]

Error code has to be synchronized with the Github server

[sudhakrishnan388-art]

.server has to be changed

[mvanhorn]

This is a real problem - losing 100+ search results because enrichment timed out is painful. The two-phase approach (return search results even if enrichment fails) is the right instinct. Can't commit to merging this as-is because the v3.0 rewrite completely rearchitects how source results flow through the pipeline, but I think the pattern is sound and I'll consider incorporating it. Thanks for the contribution.