Configuration

Configuration

ACE's behavior is tuned through ACEConfig and the arguments you pass to ACE(...). This page collects the knobs you're most likely to reach for.

---

Backends

from ace import ACE, OpenAILLM, SimulatedLLM

# Real model (one backend for all three roles)
ace = ACE(OpenAILLM(model="gpt-4o-mini"))

# Mix backends per role
ace = ACE(generator_llm=OpenAILLM(model="gpt-4o"),
          reflector_llm=OpenAILLM(model="gpt-4o-mini"),
          curator_llm=OpenAILLM(model="gpt-4o-mini"))

# Deterministic, offline, key-free (tests + demos)
ace = ACE(SimulatedLLM(env))

Any object implementing the two-method LLM protocol (complete, complete_json) works as a backend.

OpenAILLM hardening

OpenAILLM requests native JSON via response_format={"type":"json_object"} (with transparent fallback for providers that reject it) and supports built-in max_retries / backoff and a request timeout.

---

Curation mode

By default the Curator calls the LLM to propose ADD/UPDATE/REMOVE edits, with a deterministic ADD-only fallback that never drops a distilled lesson. Force the deterministic path:

from ace import ACE, ACEConfig
ace = ACE(llm, config=ACEConfig(curator_use_llm=False))

---

Grow-and-refine knobs

Knob	Effect
dedup_threshold	similarity above which two bullets are merged
harmful_margin	prune a bullet when harmful_count − helpful_count ≥ margin
refine_every	proactive refine: run grow-and-refine every N steps
lazy_refine_token_budget	lazy refine: run only when the playbook exceeds a token budget

---

Semantic de-duplication

Pass an embedder for embedding-based dedupe (cosine similarity); without one, ACE falls back to dependency-free Jaccard token overlap.

from ace import ACE, make_openai_embedder
ace = ACE(OpenAILLM(model="gpt-4o-mini"), embedder=make_openai_embedder())

When no embedder is passed and a role backend is an OpenAILLM, grow-and-refine de-duplication auto-wires OpenAI embeddings (config auto_embedder), with lexical fallback on any error.

---

Evaluation throughput

evaluate(max_workers=…) runs a concurrent, order-preserving, result-identical inference pass:

result = ace.evaluate(test, max_workers=8)

---

Feedback & labels

• use_labels=True (default) grades against sample.answer via task.evaluate.
• Pass feedback_fn(sample, generation) → Feedback for label-free / custom signals (see Custom Tasks and Feedback).

The defaults are tuned to match the paper's setup. Most users only change the backend and, occasionally, the grow-and-refine thresholds.