[Feature]: Integrate Vertex AI SDK for observing on cloud

[dot-agi]

🎯 Goal

To enable comprehensive observability for applications deployed on Google Cloud Vertex AI Agent Engine by integrating them with AgentOps using OpenTelemetry. This will allow for detailed tracing, monitoring, and cost analysis of agent performance within the AgentOps platform.

---

📖 Background

• AgentOps (https://github.com/AgentOps-AI/agentops) is an observability platform for AI agents, utilizing OpenTelemetry for tracing and monitoring. It provides an SDK with decorators for easy instrumentation.
• Vertex AI Agent Engine (Cloud Docs) is a managed service for deploying and scaling AI agents. It natively supports OpenTelemetry for Google Cloud Trace.
• Google ADK (https://github.com/google/adk-python) is a toolkit for building agents, often deployed to Vertex AI Agent Engine.
• Vertex AI SDK (https://github.com/googleapis/python-aiplatform) provides the interface for interacting with Vertex AI services, including Agent Engine.

The core idea is to configure agent applications running on Agent Engine to export OpenTelemetry data to both Google Cloud Trace (for native Vertex AI observability) and AgentOps (for specialized AI agent observability).

---

🛠️ Proposed Integration Plan

1. Agent Application Instrumentation:

   • Develop or modify agent applications (e.g., those built with Google ADK, LangChain, or custom Python code) to incorporate OpenTelemetry.
   • Utilize AgentOps decorators (@session, @agent, @operation, etc.) or manual OpenTelemetry span creation for detailed tracing of agent logic, tool calls, and model invocations.
   • Adhere to OpenTelemetry semantic conventions for GenAI where applicable.

2. Configure OpenTelemetry Exporters:

   • Within the agent's Python application code, configure the OpenTelemetry SDK.
   • Ensure the Google Cloud Trace Exporter is active (often auto-configured in Google Cloud environments). The Agent Engine service account must have permissions to write traces.
   • Initialize the AgentOps SDK (agentops.init(api_key="YOUR_AGENTOPS_API_KEY")). This typically sets up an OTLP exporter pointing to AgentOps' ingestion endpoint.
   • If manual OTel configuration is used, explicitly add an OTLP exporter for AgentOps.

3. Instrument Agent Engine Interactions:

   • Wrap key parts of the agent's logic running within the Agent Engine environment with OpenTelemetry spans.
   • For agents built with frameworks like ADK, ensure operations defined within the ADK constructs are instrumented.

4. Deployment to Vertex AI Agent Engine:

   • Package the instrumented agent application for deployment on Agent Engine (e.g., as a container).
   • Include AgentOps SDK and OpenTelemetry libraries as dependencies in the container.
   • Securely pass the AGENTOPS_API_KEY as an environment variable to the deployed container.

5. Verification and Monitoring:

   • Invoke the deployed agent.
   • Verify that traces appear in both Google Cloud Trace (for Agent Engine infrastructure) and the AgentOps dashboard (for application-level agent behavior and metrics).
   • Confirm correlation of trace data where possible.

---

✅ Key Tasks

• Research: Investigate best practices for dual OpenTelemetry exporter configuration (Google Cloud Trace + OTLP).
• Develop Sample Agent: Create or adapt a simple agent (e.g., using Google ADK) suitable for deployment on Agent Engine.
• Implement OpenTelemetry: Instrument the sample agent using AgentOps decorators and/or manual OpenTelemetry spans.
• Configure Dual Export: Set up the OpenTelemetry SDK in the sample agent to export to both Google Cloud Trace and AgentOps.
• Containerize Agent: Package the instrumented agent into a Docker container.
• Deploy to Agent Engine: Deploy the containerized agent to Vertex AI Agent Engine, ensuring the AGENTOPS_API_KEY is configured.
• Test & Verify:
  • Trigger agent execution.
  • Confirm traces are visible in Google Cloud Trace.
  • Confirm traces, events, and metrics (e.g., cost) are visible in the AgentOps dashboard.
• Documentation: Create guidelines and examples for users wanting to integrate their Agent Engine applications with AgentOps.

---

💡 Considerations & Best Practices

• Context Propagation: Ensure W3C Trace Context is correctly propagated across all components and service calls.
• Sampling Strategy: Define an appropriate OpenTelemetry sampling strategy to manage trace volume and costs.
• Custom Attributes: Encourage the use of custom attributes on spans for richer data in AgentOps (model names, tool usage, token counts, user IDs).
• Error Reporting: Ensure exceptions are captured by OpenTelemetry and correctly reported in AgentOps.
• Security: Emphasize secure management of the AGENTOPS_API_KEY within the Vertex AI environment (e.g., using Secret Manager).
• Performance Overhead: Monitor for any performance impact due to instrumentation and optimize if necessary.
• Framework Compatibility: Leverage existing AgentOps integrations for frameworks like LangChain if used within the Agent Engine.

---

✔️ Acceptance Criteria

• Successfully deployed agent on Vertex AI Agent Engine sends telemetry data to AgentOps.
• Key agent operations (e.g., LLM calls, tool usage) are visible as distinct spans/events in the AgentOps dashboard.
• Basic metrics (e.g., latency, token counts, estimated costs) for agent interactions are reported in AgentOps.
• Traces are also visible in Google Cloud Trace for the underlying Agent Engine infrastructure.
• Clear documentation or a working example is available demonstrating the integration.

🤔 Related Problem

Doing this in the name of love for @AtomSilverman and @areibman

🤝 Contribution

• Yes, I'd be happy to submit a pull request with these changes.
• I need some guidance on how to contribute.
• I'd prefer the AgentOps team to handle this update.

[saeedmithra]

AgentOps is the first solution that google-adk introduces for observibility. Really cool UI and easy to understand.

I was searching about how I can integrate it with vertex agent engine and came accross this issue. So for now I think there is no integration.

[johnny-ftt]

Can I check if we have a timeline on this feature?