Skip to content

fix: enforce build verification gate + retry empty OpenRouter responses#3

Open
DeSennd wants to merge 6 commits into
waitdeadai:mainfrom
DeSennd:fix/build-verification-gate
Open

fix: enforce build verification gate + retry empty OpenRouter responses#3
DeSennd wants to merge 6 commits into
waitdeadai:mainfrom
DeSennd:fix/build-verification-gate

Conversation

@DeSennd

@DeSennd DeSennd commented Jul 8, 2026

Copy link
Copy Markdown

Problem

ForgeGod has no enforced build/compile gate for compiled languages (C++/CMake, Rust, Go, etc.):

  1. Closure gate checks execution, not successagent.py tracks verification commands by substring-matching the command string, but never checks the exit code. A failing cmake --build (exit 2) satisfies the gate the same as a passing one.

  2. No build gate in story finalizationloop.py _finalize_story_result runs lint (Python/TS only) → reviewer (LLM text, no subprocess) → taste → status = DONE. Nothing ever invokes a compiler.

  3. C++ tools invisible to closure gateVERIFICATION_COMMAND_MARKERS had no cmake, make, ctest, g++, clang++. Build commands were unrecognized.

  4. OpenRouter null content crashmsg.get("content", "") returns None when JSON has "content": null (key exists with null value). This propagates to agent.py:373 where response_text.strip() crashes with AttributeError: 'NoneType' object has no attribute 'strip'. No retry — the router treats null as success.

Fix

  1. config.py: Add VerifyConfig ([verify] section) with build_command, test_command, timeout_s, max_fail_lines.

  2. loop.py: Add _run_verify_command helper + build/test gate in _finalize_story_result. Runs build_command as subprocess before marking DONE; non-zero exit routes story back to TODO with compiler output in error_log.

  3. agent.py: Track bash exit codes via [exit code: N] regex. Block completion when last verification command failed. Reset on new writes.

  4. agent.py: Add C++ build tools to VERIFICATION_COMMAND_MARKERS.

  5. router.py: Rewrite _call_openrouter with retry loop (3 retries, 4s delay) for empty/null content. Normalize msg.get("content") or "" to fix NoneType crash.

Evidence

Observed on a 65-story C++/CMake project: 16 stories marked done with zero successful builds after P0-01. Every cmake --build returned exit 2 in commands.log but stories still closed as done. The NoneType crash burned 3+ story retry slots across P0-01, P0-04, P2-02.

Backward compatibility

  • VerifyConfig defaults to build_command = "" (empty) — gate is skipped when no command configured, so Python/JS projects are unaffected.
  • FileNotFoundError (toolchain missing) skips the gate rather than blocking.
  • Router retry only adds delay on empty responses; successful responses return immediately.

DeSennd added 6 commits July 8, 2026 16:19
Five fixes for compiled-language projects (C++/CMake):

1. config.py: Add VerifyConfig ([verify] section) with build_command,
   test_command, timeout_s, max_fail_lines. Wired into ForgeGodConfig.

2. loop.py: Add _run_verify_command helper + build/test gate in
   _finalize_story_result. Runs build_command as subprocess before
   marking story DONE; non-zero exit routes story back to TODO with
   compiler output in error_log. FileNotFoundError skips gate.

3. agent.py: Track bash exit codes via [exit code: N] regex parse.
   New _last_verification_exit_code state field. Block completion when
   last verification command failed (non-zero exit). Reset on new writes.

4. agent.py: Add C++ build tools (cmake, make, ninja, ctest, g++,
   clang++, gcc, clang, meson, nasm) to VERIFICATION_COMMAND_MARKERS.

5. router.py: Rewrite _call_openrouter with retry loop (3 retries,
   4s delay) for empty/null content responses. Normalize
   msg.get('content') or '' to fix NoneType crash at agent.py:373.
   Handles: no choices, null content, empty string, tool-call-only.
- agent.py: Skip auto-research when task prompt contains 'Previous
  attempt errors' — indicates a retry where research already ran on
  the first attempt. Saves ~30s of wasted research per retry.
- loop.py: Bump error_log truncation in story prompt from 500 to 2000
  chars so the agent sees actual compiler errors, not just build progress.
- loop.py: Extract error lines (error:, fatal, undefined reference, etc.)
  from build gate output instead of raw tail. The 2000-char budget now
  contains the useful errors, not '[ 3%] Built target...' noise.
Previous change skipped research on every retry. Now only skips when
the last error_log entry is a build/test gate failure ('Build failed'
or 'Test failed'). Other failures (reviewer rejection, timeout, crash)
still trigger research — a wrong approach may benefit from fresh
domain research.

Loop now tags the error section header differently:
- Build/test failure: 'Build/Test failure — fix compiler errors'
- Other failure: 'Previous attempt errors (FIX THESE)'
Agent checks for 'Build/Test failure' to decide whether to skip.
The smoke test runs during the verify gate BEFORE the story is marked
done, so the phase check couldn't see the last story as completed.
Moved the phase completion logic into loop.py's _finalize_story_result
after the story status is set to DONE and the PRD is saved. This runs
after commit, checks if all stories in the phase are done, generates
the XLSX checklist, and drops the killswitch to stop the loop for
manual testing.
When the model returns empty responses repeatedly (no content, no tool
calls), the context gets poisoned with repeated closure-gate blocker
messages, making the model less likely to respond on subsequent turns.
After 4 consecutive empty responses, abort the story so it gets a fresh
agent with clean context on the next retry attempt.

Also added a _consecutive_empty_responses counter to Agent state,
reset to 0 whenever a non-empty response is received.
The 6000-char limit was too small for multi-file changes. The reviewer
was seeing truncated diffs and rejecting valid code because it couldn't
see the full implementation across split files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant