Fix GEPA integration silently falling back to MIPROv2 on DSPy >=3.1

[pradeeps319]

Problem

dspy.GEPA() raises on every run with DSPy >= 3.1, and the surrounding try/except in evolve_skill.py catches it and drops to MIPROv2. Runs print Running GEPA optimization, complete successfully, and write out an "evolved" skill — without GEPA ever being involved. The fallback message scrolls past in the middle of optimizer output, so this is easy to miss entirely.

Reproduced on a clean install (pip install -e ".[dev]", DSPy 3.2.1, gepa 0.0.27):

Running GEPA optimization (2 iterations)...

GEPA not available (GEPA.__init__() got an unexpected keyword argument 'max_steps'),
falling back to MIPROv2

and after fixing that, a second one:

GEPA not available (GEPA metric must accept five arguments:
(gold, pred, trace, pred_name, pred_trace).), falling back to MIPROv2

Causes

1. max_steps is not a DSPy >= 3.1 argument. The budget is now max_metric_calls (or auto / max_full_evals).

2. reflection_lm is required and was never passed — GEPA raises without it. Separately, --optimizer-model (help text: "Model for GEPA reflections") was unused: printed to the console, then dropped. Only --eval-model ever reached DSPy. It now drives the reflection LM, which is what the flag already claimed to do.

3. The metric signature and return type are wrong for GEPA. It needs five arguments and a score plus textual feedback. skill_fitness_metric takes three and returns a bare float from keyword overlap, so even once constructed, GEPA would have had no trace-level signal to reflect on — which is the mechanism the README describes as the reason to use GEPA.

Fix

(1) and (2) are direct argument fixes. For (3), the metric now routes through the existing LLMJudge, whose feedback field is already documented in fitness.py as "Textual feedback for GEPA's reflective analysis" but was never connected to the optimizer. skill_fitness_metric is kept as a fallback when a judge call fails, so a flaky judge degrades the score instead of killing the run.

Verification

GEPA now engages instead of falling through:

INFO dspy.teleprompt.gepa.gepa: Running GEPA for approx 20 metric calls of the
program. This amounts to 1.33 full evals on the train+val set.
INFO dspy.teleprompt.gepa.gepa: Using 5 examples for tracking Pareto scores.

Existing suite passes unchanged (145 passed). Tested against a local Ollama-served model via litellm's ollama_chat/ prefix rather than an OpenAI endpoint, so the fix is not verified against the default openai/gpt-4.1 path — worth a second look from someone with those credentials.

Note

You may want to reconsider the bare except Exception around the optimizer construction. It's what turned three straightforward API mismatches into a silent behavioral downgrade; narrowing it, or at least making the fallback loud, would surface the next DSPy break immediately.

🤖 Generated with Claude Code

[TurtleMcTurtle]

Review: PR #159

This PR provides a thorough fix for GEPA silent fallback to MIPROv2. Key changes:

• ✅ Wraps metric in gepa_skill_metric that returns dspy.Prediction(score=..., feedback=...) — routes through LLMJudge for meaningful GEPA reflections
• ✅ Uses max_metric_calls + reflection_lm with explicit temperature/max_tokens
• ✅ Uses LLMJudge for feedback (GEPA needs textual feedback to reason about mutations, not just a bare float)

However, this PR does NOT fix:

• The SkillModule still passes skill text as an InputField (not Signature instructions), so GEPA can't actually mutate it — fix(evolution): restore GEPA skill evolution on dspy 3.2 #155 and fix: make DSPy skill evolution persist optimized instructions #153 fix this
• The validator bug (evolved_body vs evolved_full) — Validate the reassembled skill, not the bare body #161 fixes this
• The evolved body extraction (optimized_module.skill_text returns original, not evolved) — fix(skills): extract evolved instruction with overfit/collapse guard #146 and fix(evolution): restore GEPA skill evolution on dspy 3.2 #155 fix this

Overlap: The GEPA constructor fix (max_metric_calls + reflection_lm) overlaps with #155 and #167. The LLMJudge integration for feedback is unique and valuable — worth preserving if a combined fix is assembled.

Suggestion: This has the best metric implementation (LLMJudge feedback), but needs to be combined with #155's SkillModule fix to actually work end-to-end. Consider coordinating with #155 author.