Skip to content

Commit 2e9c8d4

Browse files
authored
Merge pull request #44 from CaspianG/release/v2.6.2
Prepare v2.6.2 release
2 parents 0cf9704 + 97c0dc3 commit 2e9c8d4

24 files changed

Lines changed: 53 additions & 43 deletions

.github/workflows/memory-os-remote-soak.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,6 @@ jobs:
2323
remote-soak:
2424
# GitHub-hosted jobs stop after six hours, before a six-hour soak can upload evidence.
2525
runs-on: [self-hosted, linux, wavemind-evidence]
26-
environment: memory-os-production-evidence
2726
timeout-minutes: 480
2827
env:
2928
WAVEMIND_REMOTE_WORKER_URLS: ${{ secrets.WAVEMIND_REMOTE_WORKER_URLS }}

benchmarks/BENCHMARK_LEADERBOARD.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# WaveMind Benchmark Leaderboard
22

33
Generated from `benchmarks/benchmark_matrix_results.json`.
4-
Last refresh: `2026-07-15T00:57:04Z` from `025b8285cbf2`.
4+
Last refresh: `2026-07-17T18:08:53Z` from `f9d073469f74`.
55

66
This is a compact reader-facing view of checked-in benchmark results. It is not a universal vector-database leaderboard: each row uses the primary quality metric for that benchmark, and latency is shown separately so quality wins are not confused with speed wins.
77

@@ -34,7 +34,7 @@ This is a compact reader-facing view of checked-in benchmark results. It is not
3434

3535
| area | current source | claim status | next action |
3636
|---|---|---|---|
37-
| Artifact freshness | local matrix refresh at `2026-07-15T00:57:04Z` | source `025b8285cbf2`; audit gate enforced by `validate_benchmark_artifacts.py` | Keep weekly refresh green before public claims. |
37+
| Artifact freshness | local matrix refresh at `2026-07-17T18:08:53Z` | source `f9d073469f74`; audit gate enforced by `validate_benchmark_artifacts.py` | Keep weekly refresh green before public claims. |
3838
| Serverless telemetry | loopback API pool; `loopback-api-capacity-estimate`; 4 measured replicas | observed SLO `True`; loopback evidence, not a managed-serverless claim | Run `.github/workflows/serverless-observed-telemetry.yml` against deployed API nodes. |
3939
| External HTTP cluster load | kubernetes-kind-non-loopback-ci; `kubernetes-pod-dns-physical-node-drill`; 4 nodes | SLO `True`; non-loopback Kubernetes pod-DNS evidence | Run `.github/workflows/external-http-cluster-load.yml` with a remote node manifest. |
4040
| External HTTP active-active loopback | local-loopback; `loopback-api-regions`; 3 regions | SLO `True`; external URL contract over local API regions | Run `.github/workflows/external-http-active-active.yml` with remote regions for production evidence. |

benchmarks/BENCHMARK_REPORT.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# WaveMind Benchmark Report
22

33
This report is generated from `benchmarks/benchmark_matrix_results.json`.
4-
Last refresh: `2026-07-15T00:57:04Z` from `025b8285cbf2`.
4+
Last refresh: `2026-07-17T18:08:53Z` from `f9d073469f74`.
55
It separates completed local runs from runner-ready public benchmarks and planned external evaluations.
66

77
Planned rows are not claimed wins. They are the public proof path WaveMind must complete before stronger production claims.
@@ -55,7 +55,7 @@ Planned rows are not claimed wins. They are the public proof path WaveMind must
5555
| benchmark | category | status | current result | next step |
5656
|---|---|---|---|---|
5757
| External multimodal evidence runner | production-scale | runner-ready | WaveMind external multimodal evidence: runner ready True, checked in result False, requires wavemind multimodal-external-evidence --manifest external_multimodal_manifest.json --write-artifacts --output benchmarks/multimodal_external_encoder_results.json | Run the external multimodal manifest path against real s3-backed assets and then let wavemind multimodal-admission evaluate the produced artifact. |
58-
| Memory OS remote multi-worker production soak | production-scale | runner-ready | WaveMind remote worker handoff: status action_required, worker count 0, distinct worker count 0, worker https False, redis tls False, min duration seconds 21600, min worker cycles 500, worker commit must match True, allowed error rate 0.00, missing check ids worker-endpoints, non-loopback-workers, worker-transport, remote-redis, redis-transport, admin-auth | Configure the memory-os-production-evidence GitHub Environment and dispatch .github/workflows/memory-os-remote-soak.yml. |
58+
| Memory OS remote multi-worker production soak | production-scale | runner-ready | WaveMind remote worker handoff: status action_required, worker count 0, distinct worker count 0, worker https False, redis tls False, min duration seconds 21600, min worker cycles 500, worker commit must match True, allowed error rate 0.00, missing check ids worker-endpoints, non-loopback-workers, worker-transport, remote-redis, redis-transport, admin-auth | Configure the workflow's repository Actions secrets, register a wavemind-evidence runner, and dispatch .github/workflows/memory-os-remote-soak.yml. |
5959
| [MIRACL Russian](https://miracl.ai/) | multilingual-retrieval | runner-ready | No checked-in result yet. | Extend the NoMIRACL loader to full MIRACL Russian corpus once disk/service capacity is available. |
6060
| [VectorDBBench](https://github.com/zilliztech/VectorDBBench) | vector-db | runner-ready | WaveMind custom dataset export: status ready, vectors 10000, queries 100, dim 128, top k 10 | Run the generated custom dataset through official VectorDBBench targets for Qdrant, Milvus, pgvector, and WaveMind-backed FAISS/Qdrant profiles. |
6161

benchmarks/MEMORY_OS_REMOTE_SOAK_HANDOFF.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Status: `action_required`
1717

1818
## Handoff
1919

20-
- GitHub environment: `memory-os-production-evidence`
20+
- Secret scope: `repository_actions_secrets`
2121
- Workflow: `.github/workflows/memory-os-remote-soak.yml`
2222
- Dispatch: `gh workflow run memory-os-remote-soak.yml --ref main -f cycles=500 -f contenders=4`
2323
- Minimum duration: `21600` seconds

benchmarks/PRODUCTION_READINESS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ verdict, not a marketing claim.
1414

1515
| criterion | status | evidence | next step |
1616
|---|---|---|---|
17-
| Checked-in benchmark artifacts are synchronized | `pass` | audit status pass, generated_at 2026-07-12T17:10:28Z | Keep the benchmark refresh workflow green and block stale artifacts before release. |
17+
| Checked-in benchmark artifacts are synchronized | `pass` | audit status pass, generated_at 2026-07-15T00:57:04Z | Keep the benchmark refresh workflow green and block stale artifacts before release. |
1818
| Agent coherence benchmark proves behavioral lift | `pass` | WaveMind success 0.917, Chroma static 0.333, Static vector 0.333, stale error 0.000, context saved 0.931, coherent turn rate 0.750, avg latency 2.647 ms | Keep agent-behavior quality gated in CI and extend it with LLM answer-quality runs on LoCoMo/LongMemEval. |
1919
| LongMemEval answer generation beats static RAG baselines | `pass` | ollama qwen2.5:1.5b, queries 50, exact 0.240, contains 0.380, token F1 0.333, answered 0.520, grounded 0.520, supported 1.000, unsupported 0.000, faithful 1.000, abstain 0.480, evidence recall 0.920, retrieval 36.586 ms, Chroma F1 0.170, Qdrant F1 0.170 | Scale this from the checked 50-query local run to full LongMemEval-S with stronger local/API models and faithfulness scoring. |
2020
| 100k service-backed load profile passes SLO and cost gate | `pass` | recall 1.0, p99 21.25629998045042 ms, cost $1.39/1M queries | Keep the 100k profile green while adding persisted FAISS and pgvector service runs. |

benchmarks/benchmark_matrix_results.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"schema": "wavemind.benchmark_matrix.v1",
3-
"generated_at": "2026-07-15T00:57:04Z",
4-
"source_ref": "025b8285cbf2",
3+
"generated_at": "2026-07-17T18:08:53Z",
4+
"source_ref": "f9d073469f74",
55
"workflow_run_id": null,
66
"refresh_profile": "local",
77
"note": "Implemented entries are runnable from this repository. Planned entries are public benchmarks that require optional datasets, services, or heavier dependencies.",
@@ -2704,7 +2704,7 @@
27042704
}
27052705
},
27062706
"target": "Pass a fresh six-hour, 500-cycle, exact-commit remote worker soak with zero request errors, lock breaches, duplicate mutations, or state corruption.",
2707-
"next_step": "Configure the memory-os-production-evidence GitHub Environment and dispatch .github/workflows/memory-os-remote-soak.yml."
2707+
"next_step": "Configure the workflow's repository Actions secrets, register a wavemind-evidence runner, and dispatch .github/workflows/memory-os-remote-soak.yml."
27082708
},
27092709
{
27102710
"id": "memory_os_policy_bundle",

benchmarks/benchmark_registry.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2732,7 +2732,7 @@ def _implemented_entries(root: Path) -> list[dict[str, Any]]:
27322732
),
27332733
},
27342734
"target": "Pass a fresh six-hour, 500-cycle, exact-commit remote worker soak with zero request errors, lock breaches, duplicate mutations, or state corruption.",
2735-
"next_step": "Configure the memory-os-production-evidence GitHub Environment and dispatch .github/workflows/memory-os-remote-soak.yml.",
2735+
"next_step": "Configure the workflow's repository Actions secrets, register a wavemind-evidence runner, and dispatch .github/workflows/memory-os-remote-soak.yml.",
27362736
},
27372737
{
27382738
"id": "memory_os_policy_bundle",

benchmarks/memory_os_remote_soak_handoff_results.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"schema": "wavemind.memory_os_remote_worker_preflight.v1",
3-
"generated_at": "2026-07-15T00:57:52Z",
4-
"source_ref": "025b8285cbf28670d3bf712cf25d408ae8080eb3",
3+
"generated_at": "2026-07-17T17:49:54Z",
4+
"source_ref": "0cf97042351185bba80eedfc78cc9cda37967907",
55
"status": "action_required",
66
"claim_boundary": "This preflight only validates a production-like remote topology contract. It does not admit Memory OS until the remote worker soak itself passes.",
77
"topology": {
@@ -74,7 +74,7 @@
7474
],
7575
"handoff": {
7676
"workflow": ".github/workflows/memory-os-remote-soak.yml",
77-
"github_environment": "memory-os-production-evidence",
77+
"github_secret_scope": "repository_actions_secrets",
7878
"workflow_runner": [
7979
"self-hosted",
8080
"linux",

benchmarks/memory_os_remote_worker_soak.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,7 @@ def build_preflight(
193193
"worker_required_environment": ["WAVEMIND_COMMIT_SHA"],
194194
"handoff": {
195195
"workflow": ".github/workflows/memory-os-remote-soak.yml",
196-
"github_environment": "memory-os-production-evidence",
196+
"github_secret_scope": "repository_actions_secrets",
197197
"workflow_runner": ["self-hosted", "linux", "wavemind-evidence"],
198198
"command": "gh workflow run memory-os-remote-soak.yml --ref main -f cycles=500 -f contenders=4",
199199
"contract": {
@@ -687,7 +687,7 @@ def render_markdown(payload: dict[str, Any]) -> str:
687687
"",
688688
"## Handoff",
689689
"",
690-
f"- GitHub environment: `{handoff['github_environment']}`",
690+
f"- Secret scope: `{handoff['github_secret_scope']}`",
691691
f"- Workflow: `{handoff['workflow']}`",
692692
f"- Dispatch: `{handoff['command']}`",
693693
f"- Minimum duration: `{(handoff.get('contract') or {}).get('min_duration_seconds', PRODUCTION_MIN_DURATION_SECONDS)}` seconds",

benchmarks/production_admission_results.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"schema": "wavemind.production_admission.v1",
3-
"generated_at": "2026-07-12T19:58:26Z",
3+
"generated_at": "2026-07-17T18:05:10Z",
44
"status": "plan_only",
55
"admitted": false,
66
"deployment": "production",

0 commit comments

Comments
 (0)