[Phase 4.D] Multi-provider LLM support — connect any model to CodeFrame

[frankbria]

Overview

CodeFrame should be a delivery harness, not an Anthropic-specific tool. The agent execution layer should accept any LLM — Anthropic, OpenAI, Ollama, Qwen-Coder, GLM, Deepseek, vLLM, LM Studio, or any OpenAI-compatible endpoint — chosen by the user at runtime.

Current State

The right abstraction already exists:

• LLMProvider ABC in codeframe/adapters/llm/base.py with complete() / stream(), Purpose enum, and env var overrides
• AnthropicProvider implements it correctly
• ReactAgent uses it correctly — never imports anthropic directly

The gap is in the worker agents and CLI plumbing:

• worker_agent.py, backend_worker_agent.py, frontend_worker_agent.py, test_worker_agent.py all hardcode AsyncAnthropic() directly
• streaming_chat.py is wired to anthropic.AsyncAnthropic().messages.stream()
• No --llm-provider / --llm-model CLI flags
• get_provider() factory only supports "anthropic" and "mock"

Architecture

Because OpenAI's API is now an industry standard (Ollama, vLLM, LM Studio, Qwen, GLM, Deepseek all expose OpenAI-compatible endpoints), two adapters cover the entire ecosystem:

codeframe/adapters/llm/
├── base.py          ✅ LLMProvider ABC, Purpose enum, ModelSelector
├── anthropic.py     ✅ Anthropic implementation
├── openai.py        ❌ OpenAI-compatible (covers OpenAI + any base_url override)
└── mock.py          ✅ Test mock

Runtime selection:

cf work start <task> --execute --llm-provider openai --llm-model qwen2.5-coder:7b
cf work start <task> --execute --llm-provider openai --llm-model gpt-4o
cf work start <task> --execute  # default: anthropic, claude-sonnet-4-5

Subissues (implement in order)

• [4.D.1] OpenAI-compatible LLM adapter (adapters/llm/openai.py) #543 — OpenAI-compatible LLM adapter (adapters/llm/openai.py)
• [4.D.2] Provider factory + env var wiring (get_provider, ModelSelector) #544 — Provider factory + env var wiring (get_provider() + ModelSelector)
• [4.D.3] CLI flags: --llm-provider and --llm-model on cf work start / batch run #545 — CLI flags: --llm-provider and --llm-model
• [4.D.4] Refactor worker_agent.py to use LLMProvider abstraction #546 — Refactor worker_agent.py to use LLMProvider abstraction
• [4.D.5] Refactor backend_worker_agent, frontend_worker_agent, test_worker_agent to use LLMProvider #547 — Refactor backend_worker_agent.py, frontend_worker_agent.py, test_worker_agent.py
• [4.D.6] Abstract streaming layer (streaming_chat.py) to use LLMProvider #548 — Abstract streaming layer (streaming_chat.py)
• [4.D.7] Workspace config support for LLM provider (llm: block in config.yaml) #549 — Workspace config support (llm: block in .codeframe/config.yaml)

Acceptance Criteria

• cf work start <task> --execute --llm-provider openai --llm-model gpt-4o works end-to-end
• OPENAI_BASE_URL=http://localhost:11434/v1 routes to local Ollama
• Default behavior (no flags) unchanged — still uses Anthropic
• All existing tests pass
• Worker agents use LLMProvider abstraction, not AsyncAnthropic directly

[traycerai]

Warning

Ticket Assist will be discontinued on May 1, 2026. After that date, the feature will no longer be available. We suggest installing the Traycer IDE Extension for continued AI-powered, spec-driven development. Thank you for choosing Traycer!

Plan

Multi-Provider LLM Support Implementation Plan

Observations

The abstraction layer is already solid: LLMProvider ABC, AnthropicProvider, MockProvider, and get_provider() all exist in file:codeframe/adapters/llm/. The openai package (>=1.12.0) is already a declared dependency in file:pyproject.toml. The critical gaps are:

1. No OpenAI adapter: file:codeframe/adapters/llm/openai.py doesn't exist
2. Limited factory: get_provider() in file:codeframe/adapters/llm/__init__.py only handles "anthropic" and "mock", with no model override support
3. Hardcoded Anthropic in runtime: file:codeframe/core/runtime.py line 649 hardcodes get_provider("anthropic")
4. Worker agents bypass abstraction: file:codeframe/agents/worker_agent.py, file:codeframe/agents/frontend_worker_agent.py, file:codeframe/agents/test_worker_agent.py import and instantiate AsyncAnthropic directly
5. Streaming adapter is Anthropic-specific: file:codeframe/core/adapters/streaming_chat.py uses AsyncAnthropic directly (lazy-loaded at line 185)
6. No CLI flags: work_start and batch_run commands have no --llm-provider / --llm-model options

Approach

Build the OpenAI-compatible adapter first (foundation for all OpenAI-compatible endpoints), then expand get_provider() with model selection, then thread provider selection through the runtime and CLI, then refactor worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

Implementation Steps

1. Create OpenAIProvider in file:codeframe/adapters/llm/openai.py

Model this exactly after AnthropicProvider in file:codeframe/adapters/llm/anthropic.py:

• Constructor accepts api_key: Optional[str], base_url: Optional[str], and model_selector: Optional[ModelSelector]
  • Resolve api_key from: constructor arg → OPENAI_API_KEY env var → raise ValueError
  • Resolve base_url from: constructor arg → OPENAI_BASE_URL env var → None (SDK default)
• Lazy-load the openai.OpenAI sync client (for complete()) and openai.AsyncOpenAI (for stream()) using the same @property pattern as AnthropicProvider.client
• Implement complete():
  • Call self.get_model(purpose) for model selection
  • Convert Tool objects to OpenAI function-calling format: {"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}
  • Convert messages: tool results become {"role": "tool", "tool_call_id": ..., "content": ...}; assistant tool calls become {"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}
  • Parse response: extract message.content for text, message.tool_calls for tool calls (parse arguments with json.loads), map finish_reason to stop_reason
  • Return LLMResponse with content, tool_calls, stop_reason, model, input_tokens (from usage.prompt_tokens), output_tokens (from usage.completion_tokens)
• Implement stream() using openai.AsyncOpenAI with stream=True, yielding text deltas

2. Expand get_provider() in file:codeframe/adapters/llm/__init__.py

• Add OpenAIProvider to the imports and __all__
• Extend get_provider() signature to: get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider
• Add "openai" branch: instantiate OpenAIProvider() — if model is provided, pass a ModelSelector with all purpose fields set to that model
• Read CODEFRAME_LLM_PROVIDER and CODEFRAME_LLM_MODEL env vars as fallback defaults inside get_provider() when the caller passes "anthropic" or "openai" without an explicit model override
• Resolution priority:

  Provider:
    1. provider_type argument
    2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
    3. "anthropic" (hardcoded default)
  
  Model:
    1. model argument
    2. CODEFRAME_LLM_MODEL env var
    3. ModelSelector defaults (per-purpose env vars)
  

3. Thread provider selection through file:codeframe/core/runtime.py

In execute_agent() (line 649), replace the hardcoded:

provider = get_provider("anthropic") if not is_external_engine(engine) else None

• Add llm_provider: str = "anthropic" and llm_model: Optional[str] = None to execute_agent()'s parameter list
• Pass them to get_provider(llm_provider, llm_model) for builtin engines
• Update the API key validation block (around line 642): if llm_provider == "anthropic", check ANTHROPIC_API_KEY; if llm_provider == "openai", check OPENAI_API_KEY

4. Add --llm-provider and --llm-model CLI flags

In work_start command (file:codeframe/cli/app.py, around line 2273):

• Add llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")
• Add llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")
• In the API key validation block (around line 2383), add a branch: if llm_provider == "openai", call require_openai_api_key() instead of require_anthropic_api_key()
• Pass llm_provider and llm_model through to runtime.execute_agent()
• Add env-var fallback: if llm_provider is None, check CODEFRAME_LLM_PROVIDER; if llm_model is None, check CODEFRAME_LLM_MODEL

In batch_run command (around line 3576), apply the same two options and the same forwarding to conductor.start_batch().

In file:codeframe/core/conductor.py (wherever start_batch calls runtime.execute_agent): thread llm_provider and llm_model through.

5. Refactor file:codeframe/agents/worker_agent.py

• Remove the direct from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError import
• Remove SUPPORTED_MODELS and MODEL_PRICING (Anthropic-specific constants) or make them provider-agnostic
• In _call_llm_with_retry() (lines 104–284), replace the direct AsyncAnthropic call with self.llm_provider.complete(...) using the LLMProvider ABC
• Add llm_provider: Optional[LLMProvider] = None to WorkerAgent.__init__() — if None, default to get_provider("anthropic") for backward compatibility
• Replace tenacity retry decorators that catch anthropic-specific exceptions with a generic retry on Exception (or keep tenacity but catch Exception instead of AuthenticationError etc.)

6. Refactor file:codeframe/agents/frontend_worker_agent.py

• Remove from anthropic import AsyncAnthropic
• Remove self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None
• Add llm_provider: Optional[LLMProvider] = None to __init__(), defaulting to get_provider("anthropic")
• In _generate_react_component() (line 286), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000) and read response.content

7. Refactor file:codeframe/agents/test_worker_agent.py

• Same pattern as FrontendWorkerAgent: remove AsyncAnthropic import, add llm_provider parameter
• In _generate_pytest_tests() (line 311) and _correct_failing_tests() (line 655), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, ...) and read response.content

8. Abstract file:codeframe/core/adapters/streaming_chat.py

StreamingChatAdapter currently lazy-loads AsyncAnthropic directly. Refactor it to accept an LLMProvider:

• Add llm_provider: Optional[LLMProvider] = None to StreamingChatAdapter.__init__(); if None, instantiate AnthropicProvider(api_key=resolved_key) for backward compatibility
• Remove the _client property that imports AsyncAnthropic
• In _stream_turn(), replace the async with self._client.messages.stream(...) block with a call to self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096) — since LLMProvider.stream() is a sync Iterator[str], wrap it with asyncio.to_thread and yield ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk) for each chunk
• Tool-call handling in _stream_turn() is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is AnthropicProvider; for other providers, fall back to the non-streaming complete() path with tool support

9. Update file:codeframe/adapters/llm/__init__.py exports

Add OpenAIProvider to __all__ and the module-level imports so callers can do from codeframe.adapters.llm import OpenAIProvider.

---

Data Flow After Changes

sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model<br/>(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)

Loading

Testing Strategy

• Unit tests for OpenAIProvider.complete() and stream() with mock OpenAI responses
• Integration tests for get_provider() with all provider types and model overrides
• CLI tests for --llm-provider and --llm-model flags
• End-to-end test: cf work start <task> --execute --llm-provider openai --llm-model gpt-4o (requires valid OpenAI key)
• Verify default behavior (no flags) still uses Anthropic
• Verify all existing tests pass

Import In IDE

🤖 Prompt for AI Agents

# Multi-Provider LLM Support Implementation Plan

## Observations

The abstraction layer is already solid: `LLMProvider` ABC, `AnthropicProvider`, `MockProvider`, and `get_provider()` all exist in `file:codeframe/adapters/llm/`. The `openai` package (`>=1.12.0`) is already a declared dependency in `file:pyproject.toml`. The critical gaps are:

1. **No OpenAI adapter**: `file:codeframe/adapters/llm/openai.py` doesn't exist
2. **Limited factory**: `get_provider()` in `file:codeframe/adapters/llm/__init__.py` only handles `"anthropic"` and `"mock"`, with no model override support
3. **Hardcoded Anthropic in runtime**: `file:codeframe/core/runtime.py` line 649 hardcodes `get_provider("anthropic")`
4. **Worker agents bypass abstraction**: `file:codeframe/agents/worker_agent.py`, `file:codeframe/agents/frontend_worker_agent.py`, `file:codeframe/agents/test_worker_agent.py` import and instantiate `AsyncAnthropic` directly
5. **Streaming adapter is Anthropic-specific**: `file:codeframe/core/adapters/streaming_chat.py` uses `AsyncAnthropic` directly (lazy-loaded at line 185)
6. **No CLI flags**: `work_start` and `batch_run` commands have no `--llm-provider` / `--llm-model` options

## Approach

Build the OpenAI-compatible adapter first (foundation for all OpenAI-compatible endpoints), then expand `get_provider()` with model selection, then thread provider selection through the runtime and CLI, then refactor worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

## Implementation Steps

### 1. Create `OpenAIProvider` in `file:codeframe/adapters/llm/openai.py`

Model this exactly after `AnthropicProvider` in `file:codeframe/adapters/llm/anthropic.py`:

- Constructor accepts `api_key: Optional[str]`, `base_url: Optional[str]`, and `model_selector: Optional[ModelSelector]`
  - Resolve `api_key` from: constructor arg → `OPENAI_API_KEY` env var → raise `ValueError`
  - Resolve `base_url` from: constructor arg → `OPENAI_BASE_URL` env var → `None` (SDK default)
- Lazy-load the `openai.OpenAI` sync client (for `complete()`) and `openai.AsyncOpenAI` (for `stream()`) using the same `@property` pattern as `AnthropicProvider.client`
- Implement `complete()`:
  - Call `self.get_model(purpose)` for model selection
  - Convert `Tool` objects to OpenAI function-calling format: `{"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}`
  - Convert messages: tool results become `{"role": "tool", "tool_call_id": ..., "content": ...}`; assistant tool calls become `{"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}`
  - Parse response: extract `message.content` for text, `message.tool_calls` for tool calls (parse `arguments` with `json.loads`), map `finish_reason` to `stop_reason`
  - Return `LLMResponse` with `content`, `tool_calls`, `stop_reason`, `model`, `input_tokens` (from `usage.prompt_tokens`), `output_tokens` (from `usage.completion_tokens`)
- Implement `stream()` using `openai.AsyncOpenAI` with `stream=True`, yielding text deltas

### 2. Expand `get_provider()` in `file:codeframe/adapters/llm/__init__.py`

- Add `OpenAIProvider` to the imports and `__all__`
- Extend `get_provider()` signature to: `get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider`
- Add `"openai"` branch: instantiate `OpenAIProvider()` — if `model` is provided, pass a `ModelSelector` with all purpose fields set to that model
- Read `CODEFRAME_LLM_PROVIDER` and `CODEFRAME_LLM_MODEL` env vars as fallback defaults inside `get_provider()` when the caller passes `"anthropic"` or `"openai"` without an explicit model override
- Resolution priority:
  ```
  Provider:
    1. provider_type argument
    2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
    3. "anthropic" (hardcoded default)
  
  Model:
    1. model argument
    2. CODEFRAME_LLM_MODEL env var
    3. ModelSelector defaults (per-purpose env vars)
  ```

### 3. Thread provider selection through `file:codeframe/core/runtime.py`

In `execute_agent()` (line 649), replace the hardcoded:
```python
provider = get_provider("anthropic") if not is_external_engine(engine) else None
```

- Add `llm_provider: str = "anthropic"` and `llm_model: Optional[str] = None` to `execute_agent()`'s parameter list
- Pass them to `get_provider(llm_provider, llm_model)` for builtin engines
- Update the API key validation block (around line 642): if `llm_provider == "anthropic"`, check `ANTHROPIC_API_KEY`; if `llm_provider == "openai"`, check `OPENAI_API_KEY`

### 4. Add `--llm-provider` and `--llm-model` CLI flags

**In `work_start` command** (`file:codeframe/cli/app.py`, around line 2273):
- Add `llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")`
- Add `llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")`
- In the API key validation block (around line 2383), add a branch: if `llm_provider == "openai"`, call `require_openai_api_key()` instead of `require_anthropic_api_key()`
- Pass `llm_provider` and `llm_model` through to `runtime.execute_agent()`
- Add env-var fallback: if `llm_provider` is `None`, check `CODEFRAME_LLM_PROVIDER`; if `llm_model` is `None`, check `CODEFRAME_LLM_MODEL`

**In `batch_run` command** (around line 3576), apply the same two options and the same forwarding to `conductor.start_batch()`.

**In `file:codeframe/core/conductor.py`** (wherever `start_batch` calls `runtime.execute_agent`): thread `llm_provider` and `llm_model` through.

### 5. Refactor `file:codeframe/agents/worker_agent.py`

- Remove the direct `from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError` import
- Remove `SUPPORTED_MODELS` and `MODEL_PRICING` (Anthropic-specific constants) or make them provider-agnostic
- In `_call_llm_with_retry()` (lines 104–284), replace the direct `AsyncAnthropic` call with `self.llm_provider.complete(...)` using the `LLMProvider` ABC
- Add `llm_provider: Optional[LLMProvider] = None` to `WorkerAgent.__init__()` — if `None`, default to `get_provider("anthropic")` for backward compatibility
- Replace `tenacity` retry decorators that catch `anthropic`-specific exceptions with a generic retry on `Exception` (or keep tenacity but catch `Exception` instead of `AuthenticationError` etc.)

### 6. Refactor `file:codeframe/agents/frontend_worker_agent.py`

- Remove `from anthropic import AsyncAnthropic`
- Remove `self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None`
- Add `llm_provider: Optional[LLMProvider] = None` to `__init__()`, defaulting to `get_provider("anthropic")`
- In `_generate_react_component()` (line 286), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000)` and read `response.content`

### 7. Refactor `file:codeframe/agents/test_worker_agent.py`

- Same pattern as `FrontendWorkerAgent`: remove `AsyncAnthropic` import, add `llm_provider` parameter
- In `_generate_pytest_tests()` (line 311) and `_correct_failing_tests()` (line 655), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, ...)` and read `response.content`

### 8. Abstract `file:codeframe/core/adapters/streaming_chat.py`

`StreamingChatAdapter` currently lazy-loads `AsyncAnthropic` directly. Refactor it to accept an `LLMProvider`:

- Add `llm_provider: Optional[LLMProvider] = None` to `StreamingChatAdapter.__init__()`; if `None`, instantiate `AnthropicProvider(api_key=resolved_key)` for backward compatibility
- Remove the `_client` property that imports `AsyncAnthropic`
- In `_stream_turn()`, replace the `async with self._client.messages.stream(...)` block with a call to `self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096)` — since `LLMProvider.stream()` is a sync `Iterator[str]`, wrap it with `asyncio.to_thread` and yield `ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk)` for each chunk
- Tool-call handling in `_stream_turn()` is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is `AnthropicProvider`; for other providers, fall back to the non-streaming `complete()` path with tool support

### 9. Update `file:codeframe/adapters/llm/__init__.py` exports

Add `OpenAIProvider` to `__all__` and the module-level imports so callers can do `from codeframe.adapters.llm import OpenAIProvider`.

---

## Data Flow After Changes

```mermaid
sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model<br/>(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)
```

## Testing Strategy

- Unit tests for `OpenAIProvider.complete()` and `stream()` with mock OpenAI responses
- Integration tests for `get_provider()` with all provider types and model overrides
- CLI tests for `--llm-provider` and `--llm-model` flags
- End-to-end test: `cf work start <task> --execute --llm-provider openai --llm-model gpt-4o` (requires valid OpenAI key)
- Verify default behavior (no flags) still uses Anthropic
- Verify all existing tests pass

---

Execution Information

Branch: main
Commit: f61db8d

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

🔗 Related PRs

#11 - feat: Convert worker agents to async/await pattern (cf-48) [merged]
#125 - Implement WorkerAgent execute_task method [merged]
#126 - Add token tracking to WorkerAgent execute_task method [merged]
#429 - feat(core): Agent Adapter protocol definition [merged]
#512 - feat(core): StreamingChatAdapter for token-by-token Anthropic relay (#503) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[traycerai]

Warning

Ticket Assist will be discontinued on May 1, 2026. After that date, the feature will no longer be available. We suggest installing the Traycer IDE Extension for continued AI-powered, spec-driven development. Thank you for choosing Traycer!

I have updated the Implementation Plan to reflect the recent changes in the ticket.

Previous Plan

Observations

The abstraction layer is already solid: LLMProvider ABC, AnthropicProvider, MockProvider, and get_provider() all exist in file:codeframe/adapters/llm/. The openai package (>=1.12.0) is already a declared dependency in file:pyproject.toml. The critical gaps are:

• get_provider() in file:codeframe/adapters/llm/__init__.py only handles "anthropic" and "mock"
• file:codeframe/core/runtime.py hardcodes get_provider("anthropic") at line 649
• file:codeframe/agents/worker_agent.py, file:codeframe/agents/frontend_worker_agent.py, and file:codeframe/agents/test_worker_agent.py import and instantiate AsyncAnthropic directly
• file:codeframe/core/adapters/streaming_chat.py uses AsyncAnthropic directly (lazy-loaded)
• No --llm-provider / --llm-model CLI flags exist on work start or work batch run

Approach

Build the OpenAI-compatible adapter first (it's the foundation), then expand get_provider(), then thread provider selection through the runtime and CLI, then refactor the worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

Implementation Steps

1. Create OpenAIProvider in file:codeframe/adapters/llm/openai.py

Model this exactly after AnthropicProvider in file:codeframe/adapters/llm/anthropic.py:

• Constructor accepts api_key: Optional[str], base_url: Optional[str], and model_selector: Optional[ModelSelector]
  • Resolve api_key from: constructor arg → OPENAI_API_KEY env var → raise ValueError
  • Resolve base_url from: constructor arg → OPENAI_BASE_URL env var → None (SDK default)
• Lazy-load the openai.OpenAI sync client (for complete()) and openai.AsyncOpenAI (for stream()) using the same @property pattern as AnthropicProvider.client
• Implement complete():
  • Call self.get_model(purpose) for model selection
  • Convert Tool objects to OpenAI function-calling format: {"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}
  • Convert messages: tool results become {"role": "tool", "tool_call_id": ..., "content": ...}; assistant tool calls become {"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}
  • Parse response: extract message.content for text, message.tool_calls for tool calls (parse arguments with json.loads), map finish_reason to stop_reason
  • Return LLMResponse with content, tool_calls, stop_reason, model, input_tokens (from usage.prompt_tokens), output_tokens (from usage.completion_tokens)
• Implement stream() using openai.AsyncOpenAI with stream=True, yielding text deltas

2. Expand get_provider() in file:codeframe/adapters/llm/__init__.py

• Add OpenAIProvider to the imports and __all__
• Extend get_provider() to accept a second optional model: Optional[str] = None parameter
• Add "openai" branch: instantiate OpenAIProvider() — if model is provided, pass a ModelSelector with all purpose fields set to that model
• Read CODEFRAME_LLM_PROVIDER and CODEFRAME_LLM_MODEL env vars as fallback defaults inside get_provider() when the caller passes "anthropic" or "openai" without an explicit model override
• Signature becomes: get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider

Priority for provider resolution inside get_provider():
  1. provider_type argument
  2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
  3. "anthropic" (hardcoded default)

Priority for model resolution:
  1. model argument
  2. CODEFRAME_LLM_MODEL env var
  3. ModelSelector defaults (per-purpose env vars)

3. Thread provider selection through file:codeframe/core/runtime.py

In execute_agent() (line 649), replace the hardcoded:

provider = get_provider("anthropic") if not is_external_engine(engine) else None

with a call that reads the new llm_provider and llm_model parameters that will be added to execute_agent()'s signature:

• Add llm_provider: str = "anthropic" and llm_model: Optional[str] = None to execute_agent()'s parameter list
• Pass them to get_provider(llm_provider, llm_model) for builtin engines
• Also remove the hardcoded ANTHROPIC_API_KEY check for non-external engines — replace it with a provider-aware check: if llm_provider == "anthropic", check ANTHROPIC_API_KEY; if llm_provider == "openai", check OPENAI_API_KEY

4. Add --llm-provider and --llm-model CLI flags

In work_start command (file:codeframe/cli/app.py, around line 2270):

• Add llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")
• Add llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")
• In the API key validation block (around line 2383), add a branch: if llm_provider == "openai", call require_openai_api_key() instead of require_anthropic_api_key()
• Pass llm_provider and llm_model through to runtime.execute_agent()

In batch_run command (around line 3574), apply the same two options and the same forwarding to conductor.start_batch().

In file:codeframe/core/conductor.py (wherever start_batch calls runtime.execute_agent): thread llm_provider and llm_model through.

Env-var fallback (in work_start and batch_run): if llm_provider is None, check CODEFRAME_LLM_PROVIDER; if llm_model is None, check CODEFRAME_LLM_MODEL. This mirrors the existing engine resolution pattern already in both commands.

5. Refactor file:codeframe/agents/worker_agent.py

• Remove the direct from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError import
• Remove SUPPORTED_MODELS and MODEL_PRICING (Anthropic-specific constants) or make them provider-agnostic
• In _call_llm_with_retry() (lines 104–284), replace the direct AsyncAnthropic call with self.llm_provider.complete(...) using the LLMProvider ABC
• Add llm_provider: Optional[LLMProvider] = None to WorkerAgent.__init__() — if None, default to get_provider("anthropic") for backward compatibility
• Replace tenacity retry decorators that catch anthropic-specific exceptions with a generic retry on Exception (or keep tenacity but catch Exception instead of AuthenticationError etc.)

6. Refactor file:codeframe/agents/frontend_worker_agent.py

• Remove from anthropic import AsyncAnthropic
• Remove self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None
• Add llm_provider: Optional[LLMProvider] = None to __init__(), defaulting to get_provider("anthropic")
• In _generate_react_component() (line 286), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000) and read response.content

7. Refactor file:codeframe/agents/test_worker_agent.py

• Same pattern as FrontendWorkerAgent: remove AsyncAnthropic import, add llm_provider parameter
• In _generate_pytest_tests() (line 311) and _correct_failing_tests() (line 655), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, ...) and read response.content

8. Abstract file:codeframe/core/adapters/streaming_chat.py

StreamingChatAdapter currently lazy-loads AsyncAnthropic directly. Refactor it to accept an LLMProvider:

• Add llm_provider: Optional[LLMProvider] = None to StreamingChatAdapter.__init__(); if None, instantiate AnthropicProvider(api_key=resolved_key) for backward compatibility
• Remove the _client property that imports AsyncAnthropic
• In _stream_turn(), replace the async with self._client.messages.stream(...) block with a call to self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096) — since LLMProvider.stream() is a sync Iterator[str], wrap it with asyncio.to_thread and yield ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk) for each chunk
• Tool-call handling in _stream_turn() is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is AnthropicProvider; for other providers, fall back to the non-streaming complete() path with tool support

9. Update file:codeframe/adapters/llm/__init__.py exports

Add OpenAIProvider to __all__ and the module-level imports so callers can do from codeframe.adapters.llm import OpenAIProvider.

---

Data Flow After Changes

sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model\n(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)

Loading

Import In IDE

🤖 Prompt for AI Agents

## Observations

The abstraction layer is already solid: `LLMProvider` ABC, `AnthropicProvider`, `MockProvider`, and `get_provider()` all exist in `file:codeframe/adapters/llm/`. The `openai` package (`>=1.12.0`) is already a declared dependency in `file:pyproject.toml`. The critical gaps are:
- `get_provider()` in `file:codeframe/adapters/llm/__init__.py` only handles `"anthropic"` and `"mock"`
- `file:codeframe/core/runtime.py` hardcodes `get_provider("anthropic")` at line 649
- `file:codeframe/agents/worker_agent.py`, `file:codeframe/agents/frontend_worker_agent.py`, and `file:codeframe/agents/test_worker_agent.py` import and instantiate `AsyncAnthropic` directly
- `file:codeframe/core/adapters/streaming_chat.py` uses `AsyncAnthropic` directly (lazy-loaded)
- No `--llm-provider` / `--llm-model` CLI flags exist on `work start` or `work batch run`

## Approach

Build the OpenAI-compatible adapter first (it's the foundation), then expand `get_provider()`, then thread provider selection through the runtime and CLI, then refactor the worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

## Implementation Steps

### 1. Create `OpenAIProvider` in `file:codeframe/adapters/llm/openai.py`

Model this exactly after `AnthropicProvider` in `file:codeframe/adapters/llm/anthropic.py`:

- Constructor accepts `api_key: Optional[str]`, `base_url: Optional[str]`, and `model_selector: Optional[ModelSelector]`
  - Resolve `api_key` from: constructor arg → `OPENAI_API_KEY` env var → raise `ValueError`
  - Resolve `base_url` from: constructor arg → `OPENAI_BASE_URL` env var → `None` (SDK default)
- Lazy-load the `openai.OpenAI` sync client (for `complete()`) and `openai.AsyncOpenAI` (for `stream()`) using the same `@property` pattern as `AnthropicProvider.client`
- Implement `complete()`:
  - Call `self.get_model(purpose)` for model selection
  - Convert `Tool` objects to OpenAI function-calling format: `{"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}`
  - Convert messages: tool results become `{"role": "tool", "tool_call_id": ..., "content": ...}`; assistant tool calls become `{"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}`
  - Parse response: extract `message.content` for text, `message.tool_calls` for tool calls (parse `arguments` with `json.loads`), map `finish_reason` to `stop_reason`
  - Return `LLMResponse` with `content`, `tool_calls`, `stop_reason`, `model`, `input_tokens` (from `usage.prompt_tokens`), `output_tokens` (from `usage.completion_tokens`)
- Implement `stream()` using `openai.AsyncOpenAI` with `stream=True`, yielding text deltas

### 2. Expand `get_provider()` in `file:codeframe/adapters/llm/__init__.py`

- Add `OpenAIProvider` to the imports and `__all__`
- Extend `get_provider()` to accept a second optional `model: Optional[str] = None` parameter
- Add `"openai"` branch: instantiate `OpenAIProvider()` — if `model` is provided, pass a `ModelSelector` with all purpose fields set to that model
- Read `CODEFRAME_LLM_PROVIDER` and `CODEFRAME_LLM_MODEL` env vars as fallback defaults inside `get_provider()` when the caller passes `"anthropic"` or `"openai"` without an explicit model override
- Signature becomes: `get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider`

```
Priority for provider resolution inside get_provider():
  1. provider_type argument
  2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
  3. "anthropic" (hardcoded default)

Priority for model resolution:
  1. model argument
  2. CODEFRAME_LLM_MODEL env var
  3. ModelSelector defaults (per-purpose env vars)
```

### 3. Thread provider selection through `file:codeframe/core/runtime.py`

In `execute_agent()` (line 649), replace the hardcoded:
```
provider = get_provider("anthropic") if not is_external_engine(engine) else None
```
with a call that reads the new `llm_provider` and `llm_model` parameters that will be added to `execute_agent()`'s signature:
- Add `llm_provider: str = "anthropic"` and `llm_model: Optional[str] = None` to `execute_agent()`'s parameter list
- Pass them to `get_provider(llm_provider, llm_model)` for builtin engines
- Also remove the hardcoded `ANTHROPIC_API_KEY` check for non-external engines — replace it with a provider-aware check: if `llm_provider == "anthropic"`, check `ANTHROPIC_API_KEY`; if `llm_provider == "openai"`, check `OPENAI_API_KEY`

### 4. Add `--llm-provider` and `--llm-model` CLI flags

**In `work_start` command** (`file:codeframe/cli/app.py`, around line 2270):
- Add `llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")`
- Add `llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")`
- In the API key validation block (around line 2383), add a branch: if `llm_provider == "openai"`, call `require_openai_api_key()` instead of `require_anthropic_api_key()`
- Pass `llm_provider` and `llm_model` through to `runtime.execute_agent()`

**In `batch_run` command** (around line 3574), apply the same two options and the same forwarding to `conductor.start_batch()`.

**In `file:codeframe/core/conductor.py`** (wherever `start_batch` calls `runtime.execute_agent`): thread `llm_provider` and `llm_model` through.

**Env-var fallback** (in `work_start` and `batch_run`): if `llm_provider` is `None`, check `CODEFRAME_LLM_PROVIDER`; if `llm_model` is `None`, check `CODEFRAME_LLM_MODEL`. This mirrors the existing `engine` resolution pattern already in both commands.

### 5. Refactor `file:codeframe/agents/worker_agent.py`

- Remove the direct `from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError` import
- Remove `SUPPORTED_MODELS` and `MODEL_PRICING` (Anthropic-specific constants) or make them provider-agnostic
- In `_call_llm_with_retry()` (lines 104–284), replace the direct `AsyncAnthropic` call with `self.llm_provider.complete(...)` using the `LLMProvider` ABC
- Add `llm_provider: Optional[LLMProvider] = None` to `WorkerAgent.__init__()` — if `None`, default to `get_provider("anthropic")` for backward compatibility
- Replace `tenacity` retry decorators that catch `anthropic`-specific exceptions with a generic retry on `Exception` (or keep tenacity but catch `Exception` instead of `AuthenticationError` etc.)

### 6. Refactor `file:codeframe/agents/frontend_worker_agent.py`

- Remove `from anthropic import AsyncAnthropic`
- Remove `self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None`
- Add `llm_provider: Optional[LLMProvider] = None` to `__init__()`, defaulting to `get_provider("anthropic")`
- In `_generate_react_component()` (line 286), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000)` and read `response.content`

### 7. Refactor `file:codeframe/agents/test_worker_agent.py`

- Same pattern as `FrontendWorkerAgent`: remove `AsyncAnthropic` import, add `llm_provider` parameter
- In `_generate_pytest_tests()` (line 311) and `_correct_failing_tests()` (line 655), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, ...)` and read `response.content`

### 8. Abstract `file:codeframe/core/adapters/streaming_chat.py`

`StreamingChatAdapter` currently lazy-loads `AsyncAnthropic` directly. Refactor it to accept an `LLMProvider`:

- Add `llm_provider: Optional[LLMProvider] = None` to `StreamingChatAdapter.__init__()`; if `None`, instantiate `AnthropicProvider(api_key=resolved_key)` for backward compatibility
- Remove the `_client` property that imports `AsyncAnthropic`
- In `_stream_turn()`, replace the `async with self._client.messages.stream(...)` block with a call to `self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096)` — since `LLMProvider.stream()` is a sync `Iterator[str]`, wrap it with `asyncio.to_thread` and yield `ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk)` for each chunk
- Tool-call handling in `_stream_turn()` is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is `AnthropicProvider`; for other providers, fall back to the non-streaming `complete()` path with tool support

### 9. Update `file:codeframe/adapters/llm/__init__.py` exports

Add `OpenAIProvider` to `__all__` and the module-level imports so callers can do `from codeframe.adapters.llm import OpenAIProvider`.

---

## Data Flow After Changes

```mermaid
sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model\n(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)
```

---

Execution Information

Branch: main
Commit: f61db8d

New Plan

Multi-Provider LLM Support Implementation Plan

Observations

The abstraction layer is already solid: LLMProvider ABC, AnthropicProvider, MockProvider, and get_provider() all exist in file:codeframe/adapters/llm/. The openai package (>=1.12.0) is already a declared dependency in file:pyproject.toml. The critical gaps are:

1. No OpenAI adapter: file:codeframe/adapters/llm/openai.py doesn't exist
2. Limited factory: get_provider() in file:codeframe/adapters/llm/__init__.py only handles "anthropic" and "mock", with no model override support
3. Hardcoded Anthropic in runtime: file:codeframe/core/runtime.py line 649 hardcodes get_provider("anthropic")
4. Worker agents bypass abstraction: file:codeframe/agents/worker_agent.py, file:codeframe/agents/frontend_worker_agent.py, file:codeframe/agents/test_worker_agent.py import and instantiate AsyncAnthropic directly
5. Streaming adapter is Anthropic-specific: file:codeframe/core/adapters/streaming_chat.py uses AsyncAnthropic directly (lazy-loaded at line 185)
6. No CLI flags: work_start and batch_run commands have no --llm-provider / --llm-model options

Approach

Build the OpenAI-compatible adapter first (foundation for all OpenAI-compatible endpoints), then expand get_provider() with model selection, then thread provider selection through the runtime and CLI, then refactor worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

Implementation Steps

1. Create OpenAIProvider in file:codeframe/adapters/llm/openai.py

Model this exactly after AnthropicProvider in file:codeframe/adapters/llm/anthropic.py:

• Constructor accepts api_key: Optional[str], base_url: Optional[str], and model_selector: Optional[ModelSelector]
  • Resolve api_key from: constructor arg → OPENAI_API_KEY env var → raise ValueError
  • Resolve base_url from: constructor arg → OPENAI_BASE_URL env var → None (SDK default)
• Lazy-load the openai.OpenAI sync client (for complete()) and openai.AsyncOpenAI (for stream()) using the same @property pattern as AnthropicProvider.client
• Implement complete():
  • Call self.get_model(purpose) for model selection
  • Convert Tool objects to OpenAI function-calling format: {"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}
  • Convert messages: tool results become {"role": "tool", "tool_call_id": ..., "content": ...}; assistant tool calls become {"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}
  • Parse response: extract message.content for text, message.tool_calls for tool calls (parse arguments with json.loads), map finish_reason to stop_reason
  • Return LLMResponse with content, tool_calls, stop_reason, model, input_tokens (from usage.prompt_tokens), output_tokens (from usage.completion_tokens)
• Implement stream() using openai.AsyncOpenAI with stream=True, yielding text deltas

2. Expand get_provider() in file:codeframe/adapters/llm/__init__.py

• Add OpenAIProvider to the imports and __all__
• Extend get_provider() signature to: get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider
• Add "openai" branch: instantiate OpenAIProvider() — if model is provided, pass a ModelSelector with all purpose fields set to that model
• Read CODEFRAME_LLM_PROVIDER and CODEFRAME_LLM_MODEL env vars as fallback defaults inside get_provider() when the caller passes "anthropic" or "openai" without an explicit model override
• Resolution priority:

  Provider:
    1. provider_type argument
    2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
    3. "anthropic" (hardcoded default)
  
  Model:
    1. model argument
    2. CODEFRAME_LLM_MODEL env var
    3. ModelSelector defaults (per-purpose env vars)
  

3. Thread provider selection through file:codeframe/core/runtime.py

In execute_agent() (line 649), replace the hardcoded:

provider = get_provider("anthropic") if not is_external_engine(engine) else None

• Add llm_provider: str = "anthropic" and llm_model: Optional[str] = None to execute_agent()'s parameter list
• Pass them to get_provider(llm_provider, llm_model) for builtin engines
• Update the API key validation block (around line 642): if llm_provider == "anthropic", check ANTHROPIC_API_KEY; if llm_provider == "openai", check OPENAI_API_KEY

4. Add --llm-provider and --llm-model CLI flags

In work_start command (file:codeframe/cli/app.py, around line 2273):

• Add llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")
• Add llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")
• In the API key validation block (around line 2383), add a branch: if llm_provider == "openai", call require_openai_api_key() instead of require_anthropic_api_key()
• Pass llm_provider and llm_model through to runtime.execute_agent()
• Add env-var fallback: if llm_provider is None, check CODEFRAME_LLM_PROVIDER; if llm_model is None, check CODEFRAME_LLM_MODEL

In batch_run command (around line 3576), apply the same two options and the same forwarding to conductor.start_batch().

In file:codeframe/core/conductor.py (wherever start_batch calls runtime.execute_agent): thread llm_provider and llm_model through.

5. Refactor file:codeframe/agents/worker_agent.py

• Remove the direct from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError import
• Remove SUPPORTED_MODELS and MODEL_PRICING (Anthropic-specific constants) or make them provider-agnostic
• In _call_llm_with_retry() (lines 104–284), replace the direct AsyncAnthropic call with self.llm_provider.complete(...) using the LLMProvider ABC
• Add llm_provider: Optional[LLMProvider] = None to WorkerAgent.__init__() — if None, default to get_provider("anthropic") for backward compatibility
• Replace tenacity retry decorators that catch anthropic-specific exceptions with a generic retry on Exception (or keep tenacity but catch Exception instead of AuthenticationError etc.)

6. Refactor file:codeframe/agents/frontend_worker_agent.py

• Remove from anthropic import AsyncAnthropic
• Remove self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None
• Add llm_provider: Optional[LLMProvider] = None to __init__(), defaulting to get_provider("anthropic")
• In _generate_react_component() (line 286), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000) and read response.content

7. Refactor file:codeframe/agents/test_worker_agent.py

• Same pattern as FrontendWorkerAgent: remove AsyncAnthropic import, add llm_provider parameter
• In _generate_pytest_tests() (line 311) and _correct_failing_tests() (line 655), replace await self.client.messages.create(...) with await asyncio.to_thread(self.llm_provider.complete, ...) and read response.content

8. Abstract file:codeframe/core/adapters/streaming_chat.py

StreamingChatAdapter currently lazy-loads AsyncAnthropic directly. Refactor it to accept an LLMProvider:

• Add llm_provider: Optional[LLMProvider] = None to StreamingChatAdapter.__init__(); if None, instantiate AnthropicProvider(api_key=resolved_key) for backward compatibility
• Remove the _client property that imports AsyncAnthropic
• In _stream_turn(), replace the async with self._client.messages.stream(...) block with a call to self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096) — since LLMProvider.stream() is a sync Iterator[str], wrap it with asyncio.to_thread and yield ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk) for each chunk
• Tool-call handling in _stream_turn() is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is AnthropicProvider; for other providers, fall back to the non-streaming complete() path with tool support

9. Update file:codeframe/adapters/llm/__init__.py exports

Add OpenAIProvider to __all__ and the module-level imports so callers can do from codeframe.adapters.llm import OpenAIProvider.

---

Data Flow After Changes

sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model<br/>(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)

Loading

Testing Strategy

• Unit tests for OpenAIProvider.complete() and stream() with mock OpenAI responses
• Integration tests for get_provider() with all provider types and model overrides
• CLI tests for --llm-provider and --llm-model flags
• End-to-end test: cf work start <task> --execute --llm-provider openai --llm-model gpt-4o (requires valid OpenAI key)
• Verify default behavior (no flags) still uses Anthropic
• Verify all existing tests pass

Import In IDE

🤖 Prompt for AI Agents

# Multi-Provider LLM Support Implementation Plan

## Observations

The abstraction layer is already solid: `LLMProvider` ABC, `AnthropicProvider`, `MockProvider`, and `get_provider()` all exist in `file:codeframe/adapters/llm/`. The `openai` package (`>=1.12.0`) is already a declared dependency in `file:pyproject.toml`. The critical gaps are:

1. **No OpenAI adapter**: `file:codeframe/adapters/llm/openai.py` doesn't exist
2. **Limited factory**: `get_provider()` in `file:codeframe/adapters/llm/__init__.py` only handles `"anthropic"` and `"mock"`, with no model override support
3. **Hardcoded Anthropic in runtime**: `file:codeframe/core/runtime.py` line 649 hardcodes `get_provider("anthropic")`
4. **Worker agents bypass abstraction**: `file:codeframe/agents/worker_agent.py`, `file:codeframe/agents/frontend_worker_agent.py`, `file:codeframe/agents/test_worker_agent.py` import and instantiate `AsyncAnthropic` directly
5. **Streaming adapter is Anthropic-specific**: `file:codeframe/core/adapters/streaming_chat.py` uses `AsyncAnthropic` directly (lazy-loaded at line 185)
6. **No CLI flags**: `work_start` and `batch_run` commands have no `--llm-provider` / `--llm-model` options

## Approach

Build the OpenAI-compatible adapter first (foundation for all OpenAI-compatible endpoints), then expand `get_provider()` with model selection, then thread provider selection through the runtime and CLI, then refactor worker agents and streaming adapter. This ordering ensures each layer is ready before the next depends on it.

---

## Implementation Steps

### 1. Create `OpenAIProvider` in `file:codeframe/adapters/llm/openai.py`

Model this exactly after `AnthropicProvider` in `file:codeframe/adapters/llm/anthropic.py`:

- Constructor accepts `api_key: Optional[str]`, `base_url: Optional[str]`, and `model_selector: Optional[ModelSelector]`
  - Resolve `api_key` from: constructor arg → `OPENAI_API_KEY` env var → raise `ValueError`
  - Resolve `base_url` from: constructor arg → `OPENAI_BASE_URL` env var → `None` (SDK default)
- Lazy-load the `openai.OpenAI` sync client (for `complete()`) and `openai.AsyncOpenAI` (for `stream()`) using the same `@property` pattern as `AnthropicProvider.client`
- Implement `complete()`:
  - Call `self.get_model(purpose)` for model selection
  - Convert `Tool` objects to OpenAI function-calling format: `{"type": "function", "function": {"name": ..., "description": ..., "parameters": tool.input_schema}}`
  - Convert messages: tool results become `{"role": "tool", "tool_call_id": ..., "content": ...}`; assistant tool calls become `{"role": "assistant", "tool_calls": [{"id": ..., "type": "function", "function": {"name": ..., "arguments": json.dumps(input)}}]}`
  - Parse response: extract `message.content` for text, `message.tool_calls` for tool calls (parse `arguments` with `json.loads`), map `finish_reason` to `stop_reason`
  - Return `LLMResponse` with `content`, `tool_calls`, `stop_reason`, `model`, `input_tokens` (from `usage.prompt_tokens`), `output_tokens` (from `usage.completion_tokens`)
- Implement `stream()` using `openai.AsyncOpenAI` with `stream=True`, yielding text deltas

### 2. Expand `get_provider()` in `file:codeframe/adapters/llm/__init__.py`

- Add `OpenAIProvider` to the imports and `__all__`
- Extend `get_provider()` signature to: `get_provider(provider_type: str = "anthropic", model: Optional[str] = None) -> LLMProvider`
- Add `"openai"` branch: instantiate `OpenAIProvider()` — if `model` is provided, pass a `ModelSelector` with all purpose fields set to that model
- Read `CODEFRAME_LLM_PROVIDER` and `CODEFRAME_LLM_MODEL` env vars as fallback defaults inside `get_provider()` when the caller passes `"anthropic"` or `"openai"` without an explicit model override
- Resolution priority:
  ```
  Provider:
    1. provider_type argument
    2. CODEFRAME_LLM_PROVIDER env var (when provider_type is the default)
    3. "anthropic" (hardcoded default)
  
  Model:
    1. model argument
    2. CODEFRAME_LLM_MODEL env var
    3. ModelSelector defaults (per-purpose env vars)
  ```

### 3. Thread provider selection through `file:codeframe/core/runtime.py`

In `execute_agent()` (line 649), replace the hardcoded:
```python
provider = get_provider("anthropic") if not is_external_engine(engine) else None
```

- Add `llm_provider: str = "anthropic"` and `llm_model: Optional[str] = None` to `execute_agent()`'s parameter list
- Pass them to `get_provider(llm_provider, llm_model)` for builtin engines
- Update the API key validation block (around line 642): if `llm_provider == "anthropic"`, check `ANTHROPIC_API_KEY`; if `llm_provider == "openai"`, check `OPENAI_API_KEY`

### 4. Add `--llm-provider` and `--llm-model` CLI flags

**In `work_start` command** (`file:codeframe/cli/app.py`, around line 2273):
- Add `llm_provider: Optional[str] = typer.Option(None, "--llm-provider", help="LLM provider: anthropic (default) or openai")`
- Add `llm_model: Optional[str] = typer.Option(None, "--llm-model", help="Model name override (e.g. gpt-4o, qwen2.5-coder:7b)")`
- In the API key validation block (around line 2383), add a branch: if `llm_provider == "openai"`, call `require_openai_api_key()` instead of `require_anthropic_api_key()`
- Pass `llm_provider` and `llm_model` through to `runtime.execute_agent()`
- Add env-var fallback: if `llm_provider` is `None`, check `CODEFRAME_LLM_PROVIDER`; if `llm_model` is `None`, check `CODEFRAME_LLM_MODEL`

**In `batch_run` command** (around line 3576), apply the same two options and the same forwarding to `conductor.start_batch()`.

**In `file:codeframe/core/conductor.py`** (wherever `start_batch` calls `runtime.execute_agent`): thread `llm_provider` and `llm_model` through.

### 5. Refactor `file:codeframe/agents/worker_agent.py`

- Remove the direct `from anthropic import AsyncAnthropic, AuthenticationError, RateLimitError, APIConnectionError` import
- Remove `SUPPORTED_MODELS` and `MODEL_PRICING` (Anthropic-specific constants) or make them provider-agnostic
- In `_call_llm_with_retry()` (lines 104–284), replace the direct `AsyncAnthropic` call with `self.llm_provider.complete(...)` using the `LLMProvider` ABC
- Add `llm_provider: Optional[LLMProvider] = None` to `WorkerAgent.__init__()` — if `None`, default to `get_provider("anthropic")` for backward compatibility
- Replace `tenacity` retry decorators that catch `anthropic`-specific exceptions with a generic retry on `Exception` (or keep tenacity but catch `Exception` instead of `AuthenticationError` etc.)

### 6. Refactor `file:codeframe/agents/frontend_worker_agent.py`

- Remove `from anthropic import AsyncAnthropic`
- Remove `self.client = AsyncAnthropic(api_key=self.api_key) if self.api_key else None`
- Add `llm_provider: Optional[LLMProvider] = None` to `__init__()`, defaulting to `get_provider("anthropic")`
- In `_generate_react_component()` (line 286), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, messages=[{"role": "user", "content": prompt}], purpose=Purpose.EXECUTION, max_tokens=2000)` and read `response.content`

### 7. Refactor `file:codeframe/agents/test_worker_agent.py`

- Same pattern as `FrontendWorkerAgent`: remove `AsyncAnthropic` import, add `llm_provider` parameter
- In `_generate_pytest_tests()` (line 311) and `_correct_failing_tests()` (line 655), replace `await self.client.messages.create(...)` with `await asyncio.to_thread(self.llm_provider.complete, ...)` and read `response.content`

### 8. Abstract `file:codeframe/core/adapters/streaming_chat.py`

`StreamingChatAdapter` currently lazy-loads `AsyncAnthropic` directly. Refactor it to accept an `LLMProvider`:

- Add `llm_provider: Optional[LLMProvider] = None` to `StreamingChatAdapter.__init__()`; if `None`, instantiate `AnthropicProvider(api_key=resolved_key)` for backward compatibility
- Remove the `_client` property that imports `AsyncAnthropic`
- In `_stream_turn()`, replace the `async with self._client.messages.stream(...)` block with a call to `self.llm_provider.stream(messages=current_messages, purpose=Purpose.GENERATION, max_tokens=4096)` — since `LLMProvider.stream()` is a sync `Iterator[str]`, wrap it with `asyncio.to_thread` and yield `ChatEvent(type=ChatEventType.TEXT_DELTA, content=chunk)` for each chunk
- Tool-call handling in `_stream_turn()` is Anthropic-specific (it reads raw SDK events). For now, keep the Anthropic-specific tool loop only when the provider is `AnthropicProvider`; for other providers, fall back to the non-streaming `complete()` path with tool support

### 9. Update `file:codeframe/adapters/llm/__init__.py` exports

Add `OpenAIProvider` to `__all__` and the module-level imports so callers can do `from codeframe.adapters.llm import OpenAIProvider`.

---

## Data Flow After Changes

```mermaid
sequenceDiagram
    participant CLI as cf work start
    participant Runtime as runtime.execute_agent()
    participant Factory as get_provider()
    participant Provider as LLMProvider (Anthropic/OpenAI)
    participant Agent as ReactAgent / WorkerAgent

    CLI->>CLI: Resolve --llm-provider, --llm-model<br/>(CLI flag → CODEFRAME_LLM_PROVIDER → "anthropic")
    CLI->>Runtime: execute_agent(..., llm_provider="openai", llm_model="qwen2.5-coder:7b")
    Runtime->>Factory: get_provider("openai", "qwen2.5-coder:7b")
    Factory->>Provider: OpenAIProvider(base_url=OPENAI_BASE_URL)
    Runtime->>Agent: BuiltinReactAdapter(workspace, provider)
    Agent->>Provider: provider.complete(messages, purpose=EXECUTION)
    Provider-->>Agent: LLMResponse(content, tool_calls)
```

## Testing Strategy

- Unit tests for `OpenAIProvider.complete()` and `stream()` with mock OpenAI responses
- Integration tests for `get_provider()` with all provider types and model overrides
- CLI tests for `--llm-provider` and `--llm-model` flags
- End-to-end test: `cf work start <task> --execute --llm-provider openai --llm-model gpt-4o` (requires valid OpenAI key)
- Verify default behavior (no flags) still uses Anthropic
- Verify all existing tests pass

---

Execution Information

Branch: main
Commit: f61db8d