Skip to content

user-guide.md

Latest commit

 

History

History
494 lines (335 loc) · 15.7 KB

user-guide.md

File metadata and controls

494 lines (335 loc) · 15.7 KB
Raw

Telemetry User Guide

How it works

Telemetry has two layers that work together:

  • Auto-instrumentation handles the what: LLM calls, token counts, latency, model names — automatically, by calling auto_instrument() once.
  • Custom spans handle the who and why: which agent, which user, which operation — context that autoinstrumentation can't infer.

The primary pattern is to wrap autoinstrumented calls in a parent span that carries your business context:

invoke_agent span  ← you create this (agent name, tenant, session, operation type)
  └─ chat span     ← autoinstrumentation creates this (model, tokens, latency)

Quick start

1. Enable auto-instrumentation

Call before importing AI libraries:

from sap_cloud_sdk.core.telemetry import auto_instrument

auto_instrument()

from litellm import completion
# LLM calls are now automatically traced

2. Add business context with a parent span

Wrap your LLM calls to add the context autoinstrumentation can't provide:

from sap_cloud_sdk.core.telemetry import invoke_agent_span

with invoke_agent_span(
    provider="openai", agent_name="SupportBot", conversation_id="conv-123"
):
    # autoinstrumented LLM call is a child of this span
    response = client.chat.completions.create(...)

Library instrumentation

auto_instrument() automatically instruments any supported library that is already installed in the service — no extra configuration needed. If a library is not installed, it is silently skipped.

Supported libraries:

Library What is traced
httpx Outbound HTTP requests (sync and async)
requests Outbound HTTP requests
grpcio gRPC client and server calls
starlette Inbound HTTP requests
fastapi Inbound HTTP requests with route details
aiohttp Outbound async HTTP requests
django Inbound HTTP requests
flask Inbound HTTP requests
sqlalchemy Database queries
logging Injects trace_id and span_id into every log record for log-trace correlation

Instrumentation activates based on what is installed in the service, not on what extras were used to install the SDK. If your service has django in its own requirements, the SDK will instrument it automatically.

The SDK ships opentelemetry-instrumentation-* packages for all of the above as hard dependencies. The target frameworks themselves are optional — install them via your service's own requirements or via the SDK's convenience extras (e.g. sap-cloud-sdk[django]).

Introspection

Use get_instrumented_libraries() to query which libraries were actually patched at runtime:

from sap_cloud_sdk.core.telemetry import Library, get_instrumented_libraries

get_instrumented_libraries()  # -> [Library.HTTPX, Library.SQLALCHEMY, ...] after auto_instrument(), [] before

Only libraries that were installed and successfully instrumented appear in the list. Libraries skipped because they are not installed do not appear. Returns an empty list if auto_instrument() has not been called yet.

Span functions

For operations following OpenTelemetry GenAI conventions:

from sap_cloud_sdk.core.telemetry import chat_span, execute_tool_span, invoke_agent_span

# Agent invocation — top-level parent span for an agent turn
with invoke_agent_span(
    provider="openai", agent_name="SupportBot", conversation_id="cid"
):
    response = client.beta.threads.runs.create(...)

# LLM chat call — use when autoinstrumentation is not available
with chat_span(model="gpt-4", provider="openai", conversation_id="cid") as span:
    response = client.chat.completions.create(...)

# Tool execution
with execute_tool_span(
    tool_name="get_weather", tool_type="mcp", tool_description="weather mcp server"
):
    result = call_weather_api(location)

Generic spans

Use context_overlay for operations without a dedicated function:

from sap_cloud_sdk.core.telemetry import context_overlay, GenAIOperation

with context_overlay(GenAIOperation.RETRIEVAL, attributes={"index": "knowledge-base"}):
    documents = retrieve_documents(query)

Thread-safe and async-safe. Automatic Propagation.

Propagate extension context

When calling extension tools (e.g., MCP servers), wrap the call in extension_context() to propagate extension metadata via OTel baggage:

from sap_cloud_sdk.core.telemetry import (
    extension_context,
    ExtensionType,
)

# When calling an extension tool
with extension_context(
    capability_id="default",
    extension_name="ServiceNow Extension",
):
    result = await mcp_client.call_tool("generate_offer_letter", args)
    # HTTP request includes baggage header with extension metadata

Available extension types:

ExtensionType.TOOL  # MCP tool call (default)
ExtensionType.INSTRUCTION  # Instruction/prompt injection

In downstream services, read the propagated context:

from sap_cloud_sdk.core.telemetry import get_extension_context

ext_ctx = get_extension_context()
if ext_ctx:
    print(ext_ctx["capability_id"])  # "default"
    print(ext_ctx["extension_name"])  # "ServiceNow Extension"
    print(ext_ctx["extension_type"])  # "tool"

The extension baggage span processor (registered automatically by auto_instrument()) stamps sap.extension.* attributes on all spans created inside an extension_context() block, including spans from third-party instrumentation. It uses a built-in BaggageSpanProcessor under the hood to stamp baggage keys.

Available operations

GenAIOperation.CHAT
GenAIOperation.TEXT_COMPLETION
GenAIOperation.EMBEDDINGS
GenAIOperation.GENERATE_CONTENT
GenAIOperation.RETRIEVAL
GenAIOperation.EXECUTE_TOOL
GenAIOperation.CREATE_AGENT
GenAIOperation.INVOKE_AGENT

Logging

auto_instrument() sets up OTel logs alongside traces and metrics. It installs a handler on the root stdlib logger so all existing logging.getLogger(...) calls in your app automatically ship log records to the OTel backend with the same resource attributes (service name, region, subaccount, etc.).

No changes to your logging code are needed:

import logging

logger = logging.getLogger(__name__)

logger.info("Destination fetched")
logger.warning("Retrying request, attempt %d", attempt)
logger.error("Failed to connect", exc_info=True)

Identity attributes

sap.tenancy.tenant_id and user.id are automatically stamped on every log record when a request context is active — no extra code needed. The same identity used for traces is used for logs.

Resolution priority:

  1. Runtime context populated by bootstrap() (GLOBAL_TENANT_ID, USER_ID from IASContextProvider)
  2. IAS auth context set by StarletteIASTelemetryMiddleware (sap_gtid, user_uuid claims)
  3. Omitted when no identity is available (e.g. log lines emitted at startup)

Structured fields

Use extra={} to attach structured attributes to a log record:

logger.info("Request completed", extra={"tenant_id": tid, "duration_ms": 120})

Log level filtering

By default all levels (DEBUG and above) flow through OTel. To restrict what gets exported, set the level on the root logger or any specific logger:

# Only WARNING and above to OTel
logging.getLogger().setLevel(logging.WARNING)

# Or scope it to your app's logger tree
logging.getLogger("my_app").setLevel(logging.INFO)

Correlation with traces

OTel logs emitted inside an active span are automatically correlated — the trace_id and span_id are injected into the log record. No extra work needed.

Third-party logging libraries

The OTel handler is installed on the root stdlib logging logger. Any library that propagates to stdlib works automatically.

Libraries that bypass stdlib entirely need a custom sink that forwards records to logging.getLogger(...).log(...). The OTel handler then picks them up from there.

Adding attributes

To the current span

Add attributes to whichever span is currently active — including autoinstrumented ones:

from sap_cloud_sdk.core.telemetry import add_span_attribute

with invoke_agent_span(provider="openai", agent_name="SupportBot"):
    response = client.chat.completions.create(...)
    add_span_attribute("response.length", len(response.choices[0].message.content))

To a specific span

Every span function yields the span for direct access:

with invoke_agent_span(provider="openai", agent_name="SupportBot") as span:
    span.add_event("tool_selected", attributes={"tool": "search"})
    response = client.chat.completions.create(...)

Propagating parent attributes to child spans

By default, attributes set on a parent span stay on that span. If you need attributes to also appear on child spans — for example, to filter by user.id at the LLM span level in your observability backend — use propagate=True:

with invoke_agent_span(
    provider="openai",
    agent_name="SupportBot",
    attributes={"user.id": "u-456"},
    propagate=True,
):
    # child spans automatically receive user.id
    with execute_tool_span("search"):
        ...
    with chat_span("gpt-4", "openai"):
        ...

Note: propagate=True is specific for backends that require attributes to appear on every span individually. In most cases, querying by the parent span is sufficient and preferred.

Priority rules — child span values always win (highest to lowest):

  1. Required semantic keys set by the span function (e.g. gen_ai.operation.name)
  2. User-provided attributes on the child span
  3. Propagated attributes from ancestors

Propagation is scoped: once the parent span exits, its attributes stop propagating to subsequent spans.

Complete example

import logging
from sap_cloud_sdk.core.telemetry import (
    auto_instrument,
    invoke_agent_span,
    execute_tool_span,
    set_tenant_id,
    add_span_attribute,
)

auto_instrument()

from litellm import completion

logger = logging.getLogger(__name__)


async def handle_request(query: str, user_id: str):
    set_tenant_id("bh7sjh...")

    logger.info("Handling request", extra={"user_id": user_id})

    # Parent span carries business context for the whole agent turn.
    # Autoinstrumentation creates the child LLM span automatically.
    with invoke_agent_span(
        provider="openai", agent_name="SupportBot", attributes={"user.id": user_id}
    ):
        documents = await retrieve_knowledge_base(query)
        add_span_attribute("documents.retrieved", len(documents))
        logger.debug("Retrieved %d documents", len(documents))

        response = completion(
            model="gpt-4",
            messages=[
                {"role": "system", "content": f"Context: {documents}"},
                {"role": "user", "content": query},
            ],
        )

        return response

Request-scoped telemetry with middlewares

auto_instrument accepts a middlewares list for injecting per-request attributes into spans — things like tenant ID and user ID that live in the incoming request but aren't visible to autoinstrumentation.

Each middleware implements two methods:

  • register() — called once at startup to hook into the web framework
  • get_attributes() — called on every span to retrieve the current request's attributes

You can write your own by subclassing TelemetryMiddleware:

from sap_cloud_sdk.core.telemetry.middleware.base import TelemetryMiddleware


class MyMiddleware(TelemetryMiddleware):
    def register(self) -> None:
        # hook into your framework here
        ...

    def get_attributes(self) -> dict:
        # return attributes for the current request
        return {"my.attribute": ...}

Pass it to auto_instrument:

auto_instrument(middlewares=[MyMiddleware(app=app)])

Built-in: StarletteIASTelemetryMiddleware

For Starlette/FastAPI apps with IAS authentication, the SDK ships a ready-to-use middleware that reads the Authorization: Bearer <token> header on each request, parses it as an IAS JWT, and injects:

  • sap.tenancy.tenant_id from the sap_gtid claim
  • user.id from the user_uuid claim

If the header is absent or the token cannot be parsed, no attributes are set and the request continues normally.

from starlette.applications import Starlette
from sap_cloud_sdk.core.telemetry import auto_instrument
from sap_cloud_sdk.core.telemetry.middleware import StarletteIASTelemetryMiddleware

app = Starlette(...)
auto_instrument(middlewares=[StarletteIASTelemetryMiddleware(app=app)])

Configuration

Production

Ensure OTEL_EXPORTER_OTLP_ENDPOINT points to your OTLP endpoint.

Local development

Print traces to console:

export OTEL_TRACES_EXPORTER=console

Use an OTLP collector:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel-collector.example.com"

Transport protocol

Traces, metrics, and logs all use gRPC by default. Switch to HTTP/protobuf by setting:

export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"

Supported values: grpc (default), http/protobuf.

Span processor

By default, auto_instrument uses BatchSpanProcessor, which exports spans asynchronously in a background thread and is recommended for production workloads. If you need synchronous span processing (e.g. in short-lived scripts or tests where the process may exit before the batch is flushed), pass disable_batch=True:

auto_instrument(disable_batch=True)

System role

export APPFND_CONHOS_SYSTEM_ROLE="S4HC"

Solution area

export SAP_SOLUTION_AREA="AFND"

Instrumenting SDK modules with record_metrics

The record_metrics decorator records request and error counters for any SDK module operation. It is the standard way to add usage telemetry to a client method.

from sap_cloud_sdk.core.telemetry import record_metrics


class MyClient:
    @record_metrics("my_module", "my_operation")
    def my_method(self): ...

Each call to the decorated method increments sap.cloud_sdk.capability.requests. On exception it increments sap.cloud_sdk.capability.errors and re-raises. Metrics are emitted only when OTEL_EXPORTER_OTLP_ENDPOINT is set — no-op otherwise.

Using the built-in enums

For modules that live inside this package, use the Module and Operation enums:

from sap_cloud_sdk.core.telemetry import record_metrics, Module, Operation


class DestinationClient:
    @record_metrics(Module.DESTINATION, Operation.DESTINATION_GET_DESTINATION)
    def get_destination(self, name: str): ...

Using plain strings (external packages)

External packages that depend on sap-cloud-sdk can pass plain strings directly without contributing to the enums in this repo:

from sap_cloud_sdk.core.telemetry import record_metrics


class MyExternalClient:
    @record_metrics("my_module", "my_operation")
    def my_method(self): ...

The Module enum values are still the canonical form for OSS modules. Plain strings are the extension point for packages that have their own release lifecycle.

Source attribution

When one SDK module creates a client from another internally, set _telemetry_source so the metric reflects the originating module:

auditlog_client = AuditLogClient(_telemetry_source=Module.OBJECTSTORE)

The decorator reads _telemetry_source from self (or from __init__ kwargs) and passes it as the source attribute on the metric.

ORD document ID

export ORD_DOCUMENT_ID="sap.foo:ord-doc:v1"