[experiments] Daily Experiment Report — 2026-05-04

[github-actions[bot]]

🧪 Daily Experiment Report — 2026-05-04

2 workflows in github/gh-aw declare experiments: sections. No experiment state branches (experiments/*) have been initialized yet — 0 runs recorded. Experiments are configured but awaiting their first runs to begin data collection.

---

prompt_style · daily-community-attribution

Variants: concise vs verbose · Window: last 30 runs · Analysed: 0 runs
min_samples: 20 per variant (default)

No hypothesis declared.

Experiment : prompt_style
Workflow   : daily-community-attribution
Hypothesis : (not specified)
Window     : last 30 runs  |  Analysed: 0 runs with artifacts
min_samples: 20 per variant

+------------------+------+----------+----------------+--------------------+-----------+---------------+
| Variant          |  n   | Succ %   | Mean dur (s)   | 95% CI (s)         |  p-value  | min_samples   |
+------------------+------+----------+----------------+--------------------+-----------+---------------+
| concise          |   0  |   N/A    |    N/A         | N/A                |  (ref)    |  0/20 (0%)   |
| verbose          |   0  |   N/A    |    N/A         | N/A                |  N/A      |  0/20 (0%)   |
+------------------+------+----------+----------------+--------------------+-----------+---------------+
Significance: * p<0.05   ** p<0.01   *** p<0.001

Guardrails: none declared

Recommendation: EXTEND
Rationale     : No runs recorded yet — experiment state branch not initialized; awaiting first workflow execution.

Recommendation: EXTEND — No runs recorded yet; awaiting first workflow execution to initialize state.

---

prompt_style · daily-astrostylelite-markdown-spellcheck

Variants: concise vs detailed · Window: last 30 runs · Analysed: 0 runs
min_samples: 30 per variant

Hypothesis: Concise prompt reduces token consumption ≥20% without degrading fix precision. H0: no difference in fix rate.

Primary metric: effective_tokens
Secondary metrics: pr_acceptance_rate, output_length_chars

Guardrails:

• empty_output_rate < 0.10
• run_success_rate >= 0.90

Experiment : prompt_style
Workflow   : daily-astrostylelite-markdown-spellcheck
Hypothesis : Concise prompt reduces token consumption ≥20% without degrading fix precision. H0: no difference in fix rate.
Window     : last 30 runs  |  Analysed: 0 runs with artifacts
min_samples: 30 per variant

+------------------+------+----------+----------------+--------------------+-----------+---------------+
| Variant          |  n   | Succ %   | Mean dur (s)   | 95% CI (s)         |  p-value  | min_samples   |
+------------------+------+----------+----------------+--------------------+-----------+---------------+
| concise          |   0  |   N/A    |    N/A         | N/A                |  (ref)    |  0/30 (0%)   |
| detailed         |   0  |   N/A    |    N/A         | N/A                |  N/A      |  0/30 (0%)   |
+------------------+------+----------+----------------+--------------------+-----------+---------------+
Significance: * p<0.05   ** p<0.01   *** p<0.001

Guardrails:
  empty_output_rate <0.10 : PENDING (no data)
  run_success_rate >=0.90 : PENDING (no data)

Recommendation: EXTEND
Rationale     : No runs recorded yet — experiment state branch not initialized; awaiting first workflow execution.

Recommendation: EXTEND — No runs recorded yet; awaiting first workflow execution to initialize state.

---

📊 Summary

Experiment	Workflow	Control	Best variant	p-value	Guardrails	Recommendation
prompt_style	daily-community-attribution	concise	N/A	N/A	N/A	EXTEND
prompt_style	daily-astrostylelite-markdown-spellcheck	concise	N/A	N/A	PENDING	EXTEND

Analysis window: last 30 runs per workflow · Significance threshold: p < 0.05 (two-tailed)
Run: §25309518223

Note: No experiments/ state branches exist in github/gh-aw. Both workflows have experiments: configured but have not yet recorded any variant assignments. The experiments will begin collecting data on the next scheduled workflow run.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• proxy.golang.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"

See Network Configuration for more information.

Generated by daily-experiment-report · ● 3.3M · ◷

• expires on May 7, 2026, 8:51 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by daily-experiment-report.

A newer discussion is available at Discussion #30314.