Skip to content

Stabilize experience latency admission - #65

Merged
CaspianG merged 2 commits into
mainfrom
goal4/experience-quality-ci
Jul 28, 2026
Merged

Stabilize experience latency admission#65
CaspianG merged 2 commits into
mainfrom
goal4/experience-quality-ci

Conversation

@CaspianG

Copy link
Copy Markdown
Owner

Summary

  • measure WaveMind Core and Experience end-to-end latency in paired, alternating order
  • use the median of three samples per held-out task so one shared-runner spike cannot flip the product verdict
  • require the paired latency protocol in experience-quality admission and refresh all linked evidence

Evidence

  • 60 training trajectories and the frozen 30-task held-out split are unchanged
  • experience-quality admission remains 12/12 admitted
  • task success: 100% vs 16.7% Core
  • repeated observed errors: 0% vs 83.3% Core
  • median tool steps: 3 vs 4
  • median context: 36 vs 60 tokens
  • p95: 19.57 ms vs 33.85 ms Core (-42.2%)

Validation

  • python -m ruff check .
  • python -m pytest -q -> 882 passed, 5 skipped
  • six repeated benchmark runs -> 6/6 pass
  • package build and twine check pass
  • benchmark artifact audit passes

This does not weaken the existing 20% p95 regression gate. It makes the measurement protocol resistant to a single unrelated shared-runner scheduling spike while preserving end-to-end retrieval, tool, and trajectory-write cost.

@CaspianG
CaspianG merged commit f9a219e into main Jul 28, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant