ACP agent cannot use LLM for auxiliary tasks (title generation, vision check, sub-agents, etc.)

[simonrosenberg]

Problem

The ACPAgent uses a sentinel LLM(model="acp-managed") (acp_agent.py:117-119) that cannot make any direct LLM calls. This is by design — the ACP server manages the main conversational LLM — but several auxiliary features in the SDK and tools rely on agent.llm being a real, callable LLM. These features silently degrade or break when using ACP.

All places where agent.llm is used and ACP can't fulfill it

1. Title generation — degraded (fallback to truncation)

• File: openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py:918-924
• generate_title() checks if llm_to_use.model == "acp-managed" and sets llm_to_use = None, falling back to generate_fallback_title() which just truncates the first user message
• Impact: Conversations get low-quality titles like "Hey can you hel..." instead of LLM-generated ones with emoji categorization
• Also: title_utils.py:119 — llm.completion(messages) is the actual call that never happens

2. Vision capability check — incorrect result

• File: openhands-tools/openhands/tools/file_editor/definition.py:219
• conv_state.agent.llm.vision_is_active() returns False for the sentinel LLM (since "acp-managed" is not a known model in litellm)
• Impact: The file editor tool never advertises image viewing support, even when the ACP server's underlying model (e.g., Claude Sonnet) fully supports vision

3. Delegate tool sub-agent creation — broken

• File: openhands-tools/openhands/tools/delegate/impl.py:155-170
• parent_llm = parent_conversation.agent.llm then parent_llm.model_copy() to create sub-agent LLMs
• The factory function receives the sentinel LLM copy, and the sub-agent will fail if it tries to call completion() or responses() on it
• Impact: Delegation to non-ACP sub-agents is broken

4. Task manager sub-agent creation — broken

• File: openhands-tools/openhands/tools/task/manager.py:309-317
• Same pattern as delegate: parent_llm = parent.agent.llm → parent_llm.model_copy(update={"stream": False})
• Sub-agents receive the sentinel LLM and can't make calls
• Impact: Task sub-agents that need their own LLM are broken

5. TOM consult message formatting — degraded

• File: openhands-tools/openhands/tools/tom_consult/executor.py:143
• conversation.state.agent.llm.format_messages_for_llm(messages) uses the sentinel LLM's default formatting
• Impact: Messages are formatted without model-specific optimizations (no caching, no vision formatting, etc.)

6. Context condensation — explicitly unsupported

• File: openhands-sdk/openhands/sdk/context/condenser/llm_summarizing_condenser.py:180
• ACPAgent.init_state() raises NotImplementedError if a condenser is configured
• Impact: No context condensation available for ACP agents (expected, but worth noting)

7. Model name in system prompts — incorrect

• File: openhands-sdk/openhands/sdk/agent/base.py:222-228
• self.llm.model returns "acp-managed" instead of the actual model name
• Used in static_system_message property for template rendering and get_model_prompt_spec()
• Impact: Model-specific prompt optimizations are skipped (though for ACP this is handled by the ACP server, so less critical)

Can we instantiate a real OpenHands LLM alongside ACP?

Yes, technically. The LLM class only requires model to be non-empty. A minimal instantiation:

from openhands.sdk import LLM
from pydantic import SecretStr

auxiliary_llm = LLM(
    model="claude-haiku-4-5-20251001",  # cheap model for auxiliary tasks
    api_key=SecretStr(os.environ["ANTHROPIC_API_KEY"]),
)

Key considerations:

1. API keys are often already available — The ACP agent's environment typically has ANTHROPIC_API_KEY or OPENAI_API_KEY set (they're needed for ACP server authentication). These same keys could be used for an auxiliary LLM.

2. Cost control — Auxiliary tasks (title generation, vision check) are lightweight. Using a small/cheap model (e.g., Haiku) would add negligible cost.

3. The sentinel LLM already has metrics tracking — ACPAgent._record_usage() uses self.llm.metrics for ACP usage stats. An auxiliary LLM would need separate metrics to avoid conflating ACP costs with auxiliary LLM costs.

4. Not all auxiliary uses need a full LLM — Some just need model metadata (vision check, model name). These could potentially be solved by passing the ACP server's model info without instantiating a real LLM.

Proposed solutions

Option A: Auxiliary LLM field on ACPAgent

Add an optional auxiliary_llm (or utility_llm) field to ACPAgent. When set, auxiliary features use it instead of the sentinel. When not set, current fallback behavior is preserved.

class ACPAgent(AgentBase):
    auxiliary_llm: LLM | None = Field(
        default=None,
        description="Optional real LLM for auxiliary tasks (title generation, etc.). "
        "If None, auxiliary tasks use fallback behavior.",
    )

Pros: Clean separation, opt-in, doesn't change ACP semantics
Cons: Every callsite needs to check auxiliary_llm or self.llm, adds complexity

Option B: Make the sentinel LLM real but cheap

Instead of LLM(model="acp-managed"), create a real LLM from env vars with a cheap model:

def _make_auxiliary_llm() -> LLM:
    api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("OPENAI_API_KEY")
    if api_key:
        return LLM(model="claude-haiku-4-5-20251001", api_key=SecretStr(api_key))
    return LLM(model="acp-managed")  # fallback sentinel

Pros: Transparent — all existing agent.llm callsites just work
Cons: Mixes ACP and direct LLM costs/metrics, may surprise users with unexpected API calls

Option C: Expose model metadata from ACP server

For features that only need model info (vision support, model name), expose metadata from the ACP InitializeResponse or session info. Only use a real LLM for features that actually need completion().

Pros: Minimal API calls, solves vision/model-name issues cleanly
Cons: Doesn't solve title generation or sub-agent creation

Option D: Use ACP fork_session() for auxiliary tasks

Similar to how ask_agent() already uses fork_session(), title generation could fork the session and ask the ACP server to generate a title.

Pros: Uses the same billing/model as the main conversation
Cons: Heavy for simple tasks, depends on ACP server supporting arbitrary prompts in forks

Recommendation

A combination of Option A + Option C seems best:

• Expose ACP server model metadata to solve vision/model-name checks without any LLM calls
• Add an optional auxiliary_llm for tasks that genuinely need completion() (title generation)
• For sub-agent creation (delegate/task), consider using fork_session() or requiring the factory to support ACP sub-agents

🤖 Generated with Claude Code

[enyst]

Claude Code missed the LLM profiles option:

• a variant of Option A: use profile ID, not model name

[enyst]

@OpenHands Analyze the ACP code and understand LLM Profiles. What other options do we have, using profiles? You are allowed to expand the profiles feature in the direction you need to enumerate options. Answer directly as a comment to this issue.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[simonrosenberg]

@OpenHands Analyze the ACP code and understand LLM Profiles. What other options do we have, using profiles? You are allowed to expand the profiles feature in the direction you need to enumerate options. Answer directly as a comment to this issue.

[openhands-ai]

I'm on it! simonrosenberg can track my progress at all-hands.dev

[simonrosenberg]

Analysis: Using LLM Profiles for Auxiliary Tasks in ACP

I've analyzed the codebase and the existing LLM Profiles feature. Here's a comprehensive exploration of profile-based solutions.

Current LLM Profile Infrastructure

The SDK already has a robust LLMProfileStore (in openhands-sdk/openhands/sdk/llm/llm_profile_store.py) that:

• Stores LLM configurations as JSON files in ~/.openhands/profiles/
• Supports save/load/list/delete operations with file locking
• Can serialize/deserialize full LLM instances including credentials

The FallbackStrategy (in openhands-sdk/openhands/sdk/llm/fallback_strategy.py) already uses profiles for fallback LLMs:

class FallbackStrategy(BaseModel):
    fallback_llms: list[str] = Field(
        description="Ordered list of LLM profile names to try on transient failure."
    )
    profile_store_dir: str | Path | None = Field(...)

Option E: Profile-Based Auxiliary LLM (Recommended)

Instead of creating an auxiliary_llm: LLM | None field with direct credentials, use a profile reference:

class ACPAgent(AgentBase):
    auxiliary_llm_profile: str | None = Field(
        default=None,
        description=(
            "Name of an LLM profile to load for auxiliary tasks (title generation, "
            "sub-agent creation, etc.). Profile is loaded from ~/.openhands/profiles/. "
            "If None, auxiliary tasks use fallback behavior."
        ),
    )
    auxiliary_profile_store_dir: str | Path | None = Field(
        default=None,
        description="Custom profile store directory. Defaults to ~/.openhands/profiles/."
    )
    
    _auxiliary_llm: LLM | None = PrivateAttr(default=None)
    
    def get_auxiliary_llm(self) -> LLM | None:
        """Lazily load the auxiliary LLM from profile."""
        if self._auxiliary_llm is None and self.auxiliary_llm_profile:
            store = LLMProfileStore(self.auxiliary_profile_store_dir)
            self._auxiliary_llm = store.load(self.auxiliary_llm_profile)
        return self._auxiliary_llm

Advantages over Option A (direct LLM field):

1. No credentials in agent config: Profile names are just strings, so agent configs can be safely serialized without exposing API keys
2. Reusable across agents: Users configure profiles once, reference them everywhere
3. Consistent with FallbackStrategy: Same pattern already proven in the codebase
4. Easy to configure: openhands profile save aux-llm --model claude-haiku-4-5-20251001

Option E.1: Automatic Profile Discovery

Extend the profile approach with automatic discovery of well-known profile names:

# Well-known profile names that ACPAgent will auto-load if present
_AUX_PROFILE_NAMES = ["auxiliary", "aux", "utility", "cheap"]

class ACPAgent(AgentBase):
    auxiliary_llm_profile: str | None = Field(
        default=None,
        description=(
            "Profile name for auxiliary LLM. If not set, auto-discovers from "
            "well-known names: 'auxiliary', 'aux', 'utility', 'cheap'."
        ),
    )
    
    def get_auxiliary_llm(self) -> LLM | None:
        if self._auxiliary_llm is not None:
            return self._auxiliary_llm
            
        store = LLMProfileStore(self.auxiliary_profile_store_dir)
        
        # Try explicit profile first
        if self.auxiliary_llm_profile:
            self._auxiliary_llm = store.load(self.auxiliary_llm_profile)
            return self._auxiliary_llm
        
        # Auto-discover from well-known names
        existing = set(store.list())
        for name in _AUX_PROFILE_NAMES:
            if f"{name}.json" in existing:
                try:
                    self._auxiliary_llm = store.load(name)
                    logger.info(f"Auto-loaded auxiliary LLM from profile '{name}'")
                    return self._auxiliary_llm
                except Exception:
                    continue
        
        return None

Advantage: Zero-configuration for users who just want "it to work" — create ~/.openhands/profiles/auxiliary.json once and all ACP agents pick it up.

Option E.2: Profile with Model Metadata Hints

Extend profiles to include model metadata (vision support, canonical name) that can be used without loading a full LLM:

# In LLM class or separate ProfileMetadata
class LLMProfileMetadata(BaseModel):
    """Lightweight metadata extracted from LLM profiles."""
    model: str
    supports_vision: bool = False
    model_canonical_name: str | None = None

class LLMProfileStore:
    def get_metadata(self, name: str) -> LLMProfileMetadata:
        """Load just the metadata without instantiating the LLM."""
        profile_path = self._get_profile_path(name)
        with open(profile_path) as f:
            data = json.load(f)
        return LLMProfileMetadata(
            model=data["model"],
            supports_vision=data.get("supports_vision", False),
            model_canonical_name=data.get("model_canonical_name"),
        )

This allows solving the vision capability check without any API calls — just read the profile JSON.

Option E.3: ACP Server Profile Reference

For cases where the ACP server's model should be used (sub-agent creation), add a special "acp-server" profile reference:

class ACPAgent(AgentBase):
    subagent_llm_profile: str | None = Field(
        default=None,
        description=(
            "Profile for sub-agent LLMs. If 'acp-server', sub-agents use fork_session(). "
            "If another profile name, sub-agents use that LLM directly."
        ),
    )

This distinguishes between:

• Auxiliary tasks (title generation): Use cheap profile
• Sub-agents (delegate/task): Use main model via fork or explicit profile

Summary: Profile-Based Solution Matrix

Use Case	Current Behavior	Profile Solution
Title generation	Falls back to truncation	Load auxiliary_llm_profile → call completion()
Vision check	Returns False	Read profile metadata or call vision_is_active() on loaded LLM
Sub-agent creation	Broken (sentinel LLM)	Use subagent_llm_profile or fork_session()
TOM consult formatting	Default formatting	Use auxiliary LLM's format_messages_for_llm()
Model name in prompts	Returns "acp-managed"	Read profile's model field

Recommended Implementation

1. Add profile fields to ACPAgent:

   auxiliary_llm_profile: str | None = None
   auxiliary_profile_store_dir: str | Path | None = None

2. Add lazy-loading helper:

   def get_auxiliary_llm(self) -> LLM | None

3. Update callsites to use agent.get_auxiliary_llm() or agent.llm:

   • local_conversation.py:generate_title() — already has fallback logic
   • file_editor/definition.py — use auxiliary LLM for vision check
   • delegate/impl.py and task/manager.py — use auxiliary LLM for sub-agents

4. Optional: Profile metadata API for zero-cost vision/model-name checks

This approach:

• ✅ Keeps credentials out of agent configs
• ✅ Reuses existing profile infrastructure
• ✅ Is consistent with FallbackStrategy pattern
• ✅ Allows gradual opt-in (profiles are optional)
• ✅ Separates auxiliary costs from main ACP costs (different LLM instance)

[openhands-ai]

Summary

I successfully completed the requested task to analyze the ACP code and LLM Profiles feature, then post the analysis as a comment to GitHub issue #2471.

What was done:

✅ Analyzed the ACP agent code (acp_agent.py) to understand how the sentinel LLM(model="acp-managed") works

✅ Analyzed the LLM Profiles infrastructure including:

• LLMProfileStore - stores LLM configs as JSON files
• FallbackStrategy - already uses profiles for fallback LLMs

✅ Enumerated profile-based options to solve the auxiliary LLM problem:

• Option E: Profile-based auxiliary LLM with auxiliary_llm_profile: str | None field
• Option E.1: Automatic profile discovery from well-known names
• Option E.2: Profile metadata API for zero-cost vision/model checks
• Option E.3: ACP server profile reference for sub-agents

✅ Posted analysis as GitHub comment: #2471 (comment)

Key insight from the analysis:

The SDK already has profile infrastructure (LLMProfileStore) that's used by FallbackStrategy. The recommended solution extends this pattern by adding an auxiliary_llm_profile field to ACPAgent that references a profile by name (not embedded credentials), keeping configurations clean and reusable while enabling auxiliary LLM functionality for title generation, vision checks, and sub-agent creation.

[enyst]

I think that’s roughly what I had in mind… at least as long as all must be passed via ACPAgent, but maybe I should double-check: must all be in ACPAgent?

title generation, in the SDK at the moment, is not linked to Agent, and its LLM seems like it could be independent too.

On the other hand, the opposite is also true: these auxiliary tasks, specially sub-agents, might need to have their own profiles? That is, do we need more than one extra-profile, for sub-agents?

[simonrosenberg]

Corrected analysis: actual scope is much smaller than originally reported

Delegation/sub-agents are NOT affected

Items 3 and 4 from the original issue (delegate tool and task manager copying parent_llm) are not a real problem in ACP mode. When using ACP, the ACP server (e.g., Claude Code) handles its own sub-agent/delegation internally — the SDK's delegate and task tools are never invoked for an ACP agent. Same for context condensation (#6) — the ACP server manages its own context window.

What's actually broken/degraded:

#	Feature	Severity	Root cause
1	Title generation	Degraded (truncation fallback)	Sentinel LLM can't call completion()
2	Vision capability check	Incorrect (False even when ACP model supports vision)	"acp-managed" is unknown to litellm
3	Model name in system prompts	Minor (reports "acp-managed")	Sentinel has no real model name
4	TOM consult formatting	Minor (default formatting)	Sentinel has no model-specific config

Proposed fixes

Title generation — This is independent of the agent. The generate_title(llm=...) parameter already flows through ConversationService → EventService → LocalConversation → title_utils. The caller (e.g., AutoTitleSubscriber) could load an LLM from a profile and pass it in. No changes to ACPAgent needed — this is a conversation-level concern, not an agent-level one.

To answer @enyst's question: no, this does NOT need to go through ACPAgent. A title_llm_profile config at the conversation service / app level would be the cleanest approach.

Vision check / model name — These only need model metadata, not a callable LLM. The acp_model field is already passed to ACPAgent (e.g., "claude-opus-4-6"). We could use it to set the sentinel's model field so vision_is_active() and self.llm.model return correct values:

def _make_dummy_llm(model_name: str = "acp-managed") -> LLM:
    """Create a dummy LLM that should never be called directly."""
    return LLM(model=model_name)

Then in init_state(), pass self.acp_model or "acp-managed" to _make_dummy_llm(). This fixes vision detection and model name without making the LLM callable.

On sub-agent profiles — to answer @enyst's second question: we do NOT need extra profiles for sub-agents in ACP mode, since ACP handles delegation internally. The only profile needed (if any) is one for title generation, which lives at the conversation/service level, not on ACPAgent.

🤖 Generated with Claude Code