Python: Specialist-to-specialist handoff orchestration with tracing not rendering correctly on the AI Foundry portal

[tsuting]

Description

When using setup_observability to enable tracing for Specialist-to-specialist handoff orchestration, the Azure AI Foundry portal does not correctly display the tracing information. Issues found are:

1. The agents show the full conversation from the beginning including all the upcoming user inputs
2. The triage agent shows the information from other agents
3. The traces for some agents do not appear (linked with (2))

However, from the console log, it looks okay.

Expected Behavior

The traces should show

1. triage agent gets the inputs from users (only shows input from user and the output from triage agent)
2. logs for replacement, delivery and billing agents and the correct input / output of each agent
3. triage agents gets new inputs from users (which is not showing from the portal neither, only one user input)

and so on.

Actual Behavior

(1) The agents show the full conversation from the begining
(2) The triage agent shows the information from other agents
(3) The traces for some agents do not appear (linked with (2))

Scripts to Reproduce

Almost the same as https://github.com/microsoft/agent-framework/blob/main/python/samples/getting_started/workflows/orchestration/handoff_specialist_to_specialist.py. I only added

from dotenv import load_dotenv
import os
load_dotenv()

# setup observability for application insights
setup_observability(
    applicationinsights_connection_string=os.getenv(
        "APPLICATIONINSIGHTS_CONNECTION_STRING"
    ),
    enable_sensitive_data=os.getenv("ENABLE_SENSITIVE_DATA"),
)

# Copyright (c) Microsoft. All rights reserved.

"""Sample: Multi-tier handoff workflow with specialist-to-specialist routing.

This sample demonstrates advanced handoff routing where specialist agents can hand off
to other specialists, enabling complex multi-tier workflows. Unlike the simple handoff
pattern (see handoff_simple.py), specialists here can delegate to other specialists
without returning control to the user until the specialist chain completes.

Routing Pattern:
    User → Triage → Specialist A → Specialist B → Back to User

This pattern is useful for complex support scenarios where different specialists need
to collaborate or escalate to each other before returning to the user. For example:
    - Replacement agent needs shipping info → hands off to delivery agent
    - Technical support needs billing info → hands off to billing agent
    - Level 1 support escalates to Level 2 → hands off to escalation agent

Configuration uses `.add_handoff()` to explicitly define the routing graph.

Prerequisites:
    - `az login` (Azure CLI authentication)
    - Environment variables configured for AzureOpenAIChatClient
"""

import asyncio
from collections.abc import AsyncIterable
from typing import cast

from agent_framework import (
    ChatMessage,
    HandoffBuilder,
    HandoffUserInputRequest,
    RequestInfoEvent,
    WorkflowEvent,
    WorkflowOutputEvent,
    WorkflowRunState,
    WorkflowStatusEvent,
)
from agent_framework.azure import AzureOpenAIChatClient
from azure.identity import AzureCliCredential
from agent_framework.observability import setup_observability
from dotenv import load_dotenv
import os

load_dotenv()

# setup observability for application insights
setup_observability(
    applicationinsights_connection_string=os.getenv(
        "APPLICATIONINSIGHTS_CONNECTION_STRING"
    ),
    enable_sensitive_data=os.getenv("ENABLE_SENSITIVE_DATA"),
)


def create_agents(chat_client: AzureOpenAIChatClient):
    """Create triage and specialist agents with multi-tier handoff capabilities.

    Returns:
        Tuple of (triage_agent, replacement_agent, delivery_agent, billing_agent)
    """
    triage = chat_client.create_agent(
        instructions=(
            "You are a customer support triage agent. Assess the user's issue and route appropriately:\n"
            "- For product replacement issues: call handoff_to_replacement_agent\n"
            "- For delivery/shipping inquiries: call handoff_to_delivery_agent\n"
            "- For billing/payment issues: call handoff_to_billing_agent\n"
            "Be concise and friendly."
        ),
        name="triage_agent",
    )

    replacement = chat_client.create_agent(
        instructions=(
            "You handle product replacement requests. Ask for order number and reason for replacement.\n"
            "If the user also needs shipping/delivery information, call handoff_to_delivery_agent to "
            "get tracking details. Otherwise, process the replacement and confirm with the user.\n"
            "Be concise and helpful."
        ),
        name="replacement_agent",
    )

    delivery = chat_client.create_agent(
        instructions=(
            "You handle shipping and delivery inquiries. Provide tracking information, estimated "
            "delivery dates, and address any delivery concerns.\n"
            "If billing issues come up, call handoff_to_billing_agent.\n"
            "Be concise and clear."
        ),
        name="delivery_agent",
    )

    billing = chat_client.create_agent(
        instructions=(
            "You handle billing and payment questions. Help with refunds, payment methods, "
            "and invoice inquiries. Be concise."
        ),
        name="billing_agent",
    )

    return triage, replacement, delivery, billing


async def _drain(stream: AsyncIterable[WorkflowEvent]) -> list[WorkflowEvent]:
    """Collect all events from an async stream into a list."""
    return [event async for event in stream]


def _handle_events(events: list[WorkflowEvent]) -> list[RequestInfoEvent]:
    """Process workflow events and extract pending user input requests."""
    requests: list[RequestInfoEvent] = []

    for event in events:
        if isinstance(event, WorkflowStatusEvent) and event.state in {
            WorkflowRunState.IDLE,
            WorkflowRunState.IDLE_WITH_PENDING_REQUESTS,
        }:
            print(f"[status] {event.state.name}")

        elif isinstance(event, WorkflowOutputEvent):
            conversation = cast(list[ChatMessage], event.data)
            if isinstance(conversation, list):
                print("\n=== Final Conversation ===")
                for message in conversation:
                    # Filter out messages with no text (tool calls)
                    if not message.text.strip():
                        continue
                    speaker = message.author_name or message.role.value
                    print(f"- {speaker}: {message.text}")
                print("==========================")

        elif isinstance(event, RequestInfoEvent):
            if isinstance(event.data, HandoffUserInputRequest):
                _print_handoff_request(event.data)
            requests.append(event)

    return requests


def _print_handoff_request(request: HandoffUserInputRequest) -> None:
    """Display a user input request with conversation context."""
    print("\n=== User Input Requested ===")
    # Filter out messages with no text for cleaner display
    messages_with_text = [msg for msg in request.conversation if msg.text.strip()]
    print(f"Last {len(messages_with_text)} messages in conversation:")
    for message in messages_with_text[-5:]:  # Show last 5 for brevity
        speaker = message.author_name or message.role.value
        text = message.text[:100] + "..." if len(message.text) > 100 else message.text
        print(f"  {speaker}: {text}")
    print("============================")


async def main() -> None:
    """Demonstrate specialist-to-specialist handoffs in a multi-tier support scenario.

    This sample shows:
    1. Triage agent routes to replacement specialist
    2. Replacement specialist hands off to delivery specialist
    3. Delivery specialist can hand off to billing if needed
    4. All transitions are seamless without returning to user until complete

    The workflow configuration explicitly defines which agents can hand off to which others:
    - triage_agent → replacement_agent, delivery_agent, billing_agent
    - replacement_agent → delivery_agent, billing_agent
    - delivery_agent → billing_agent
    """
    chat_client = AzureOpenAIChatClient(credential=AzureCliCredential())
    triage, replacement, delivery, billing = create_agents(chat_client)

    # Configure multi-tier handoffs using fluent add_handoff() API
    # This allows specialists to hand off to other specialists
    workflow = (
        HandoffBuilder(
            name="multi_tier_support",
            participants=[triage, replacement, delivery, billing],
        )
        .set_coordinator(triage)
        .add_handoff(
            triage, [replacement, delivery, billing]
        )  # Triage can route to any specialist
        .add_handoff(
            replacement, [delivery, billing]
        )  # Replacement can delegate to delivery or billing
        .add_handoff(delivery, billing)  # Delivery can escalate to billing
        # Termination condition: Stop when more than 4 user messages exist.
        # This allows agents to respond to the 4th user message before the 5th triggers termination.
        # In this sample: initial message + 3 scripted responses = 4 messages, then 5th message ends workflow.
        .with_termination_condition(
            lambda conv: sum(1 for msg in conv if msg.role.value == "user") > 4
        )
        .build()
    )

    # Scripted user responses simulating a multi-tier handoff scenario
    # Note: The initial run_stream() call sends the first user message,
    # then these scripted responses are sent in sequence (total: 4 user messages).
    # A 5th response triggers termination after agents respond to the 4th message.
    scripted_responses = [
        "I need help with order 12345. I want a replacement and need to know when it will arrive.",
        "The item arrived damaged. I'd like a replacement shipped to the same address.",
        "Great! Can you confirm the shipping cost won't be charged again?",
        "Thank you!",  # Final response to trigger termination after billing agent answers
    ]

    print("\n" + "=" * 80)
    print("SPECIALIST-TO-SPECIALIST HANDOFF DEMONSTRATION")
    print("=" * 80)
    print(
        "\nScenario: Customer needs replacement + shipping info + billing confirmation"
    )
    print("Expected flow: User → Triage → Replacement → Delivery → Billing → User")
    print("=" * 80 + "\n")

    # Start workflow with initial message
    print(
        "[User]: I need help with order 12345. I want a replacement and need to know when it will arrive.\n"
    )
    events = await _drain(
        workflow.run_stream(
            "I need help with order 12345. I want a replacement and need to know when it will arrive."
        )
    )
    pending_requests = _handle_events(events)

    # Process scripted responses
    response_index = 0
    while pending_requests and response_index < len(scripted_responses):
        user_response = scripted_responses[response_index]
        print(f"\n[User]: {user_response}\n")

        responses = {req.request_id: user_response for req in pending_requests}
        events = await _drain(workflow.send_responses_streaming(responses))
        pending_requests = _handle_events(events)

        response_index += 1

if __name__ == "__main__":
    asyncio.run(main())

Version

• agent-framework-azure-ai 1.0.0b251114
• agent-framework-core 1.0.0b251114
• agent-framework-devui 1.0.0b251114

[alliscode]

@TaoChenOSU could you triage this and see if it's an issue in AF or the portal.

[TaoChenOSU]

Hi @tsuting,

This is me running the same sample:

The UI is not rendering the tool call and tool result but the data is there:

This all looks correct to me.

I suspect you are launching multiple instances of script with just one trace exporter. I don't if that's possible and how you achieve that, but it looks like the spans got messed up based on your screenshot.

Would you mind running the sample again in a clean environment/terminal and see if you can reproduce? If so, could you give us more details on how you run the sample?

[tsuting]

Hi,

• I only ran the script once and the only change I made is to add the following two lines at the beginning of the script.

from agent_framework.observability import setup_observability

setup_observability()

• I cleaned up the uv environment and used the devcontainer in the main and ran again it's the same.

• As you said, even I ran the script once, there are 5 records generated from the portal. They're different but one of them is correct.

  

• I tried handoff_simple.py as well. The issue is the same as this one.

• I tried other scripts such as concurrent_agents.py and sequential_agents.py. They all look correct.

Could I ask which version of azure-monitor-opentelemetry-exporter you're using? I found this warning when I ran the script. I am using 1.0.0b45.

/home/tsuting/Projects/agent-framework/python/.venv/lib/python3.13/site-packages/azure/monitor/opentelemetry/exporter/_configuration/__init__.py:236 - WARNING] Version mismatch between change and config responses.No configurations updated.

[TaoChenOSU]

Hi @tsuting

You are seeing 5 spans because the workflow is run 5 times. If you put a root span in the sample, you will see the following:

This is how to put a root span in the sample:

from agent_framework.observability import get_tracer

if __name__ == "__main__":
    with get_tracer().start_as_current_span("Handoff Specialist") as current_span:
        asyncio.run(main())

If you expand each run and find the first invoke_agent span for the triage_agent, you will see that the input grows each run. That's because the context of the triage_agent accumulates. Outputs from other agents will accumulate in the triage_agent's cache and the next time the triage_agent is invoked, the whole cache will be passed to the agent. This is how Handoff works such that all the agents are in sync.

[tsuting]

Thanks for the explanation. Do you think it would be helpful to include a code example, such as this one, or add documentation on how to use it for specific workflow orchestration scenarios? They behave differently, so some guidance might be useful.

[TaoChenOSU]

Yeah, I think it'd be helpful to have a more detailed document explaining how each orchestration works. We will work on that.

As far as observability is concerned, it will be the same for all applications. You will need to know the mechanisms of the orchestrations to make sense of the traces.