feat: enable GenAI event capture for all agents #487
Conversation
Add environment variables to capture GenAI events (user messages, system prompts, model completions) for all agent types across both deployment targets. Changes: - Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true for all agents - Add service.namespace and service.version to OTEL_RESOURCE_ATTRIBUTES - Update telemetry log filter to capture gen_ai.* logs by namespace - Configure Agent Engine telemetry via deploy.py environment variables This enables automatic telemetry collection for: - ADK agents (with span content capture disabled) - LangGraph agents - CrewAI agents - Both Agent Engine and Cloud Run deployments Version tracking uses COMMIT_SHA environment variable (defaults to "dev").
Summary of ChangesHello @eliasecchig, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a standardized approach to enable and configure GenAI event telemetry across all agent types within the system. By setting essential environment variables and updating log filters, it ensures that new projects automatically capture detailed GenAI interaction events, along with critical service context like namespace and version. This enhancement significantly improves the system's observability and provides a clearer understanding of AI-powered application behavior. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request enables GenAI event telemetry by default for all agent types, which is a great step towards better observability. The changes include updating the telemetry log filter in Terraform to capture gen_ai.* events and setting several environment variables in the agent applications to enable and configure the telemetry capture, including service namespace and version tracking. My review focuses on improving code maintainability by reducing duplication and adhering to Python's standard coding style. I've left a few comments suggesting how to refactor the duplicated telemetry setup logic and correct the placement of an import statement.
| # Enable GenAI event capture for telemetry | ||
| os.environ.setdefault("GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY", "true") | ||
| {%- if cookiecutter.is_adk %} | ||
| os.environ.setdefault("ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS", "false") | ||
| {%- endif %} | ||
| os.environ.setdefault("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", "true") | ||
| # Set resource attributes with service namespace and commit SHA for version tracking | ||
| commit_sha = os.environ.get("COMMIT_SHA", "dev") | ||
| os.environ.setdefault( | ||
| "OTEL_RESOURCE_ATTRIBUTES", | ||
| f"service.namespace={{cookiecutter.project_name}},service.version={commit_sha}", | ||
| ) |
There was a problem hiding this comment.
This block of code for setting up telemetry environment variables is duplicated later in this file (lines 206-214). To improve maintainability and avoid duplication, consider extracting this logic into a shared utility function. This function could be called from both places where this setup is needed.
| # Enable GenAI event capture for telemetry | ||
| import os | ||
|
|
||
| os.environ.setdefault("GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY", "true") | ||
| os.environ.setdefault("ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS", "false") | ||
| os.environ.setdefault("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", "true") | ||
| # Set resource attributes with service namespace and commit SHA for version tracking | ||
| commit_sha = os.environ.get("COMMIT_SHA", "dev") | ||
| os.environ.setdefault( | ||
| "OTEL_RESOURCE_ATTRIBUTES", | ||
| f"service.namespace={{cookiecutter.project_name}},service.version={commit_sha}", | ||
| ) |
There was a problem hiding this comment.
This block of code for setting up telemetry is duplicated multiple times in this file (e.g., lines 346-357 and 473-481). To adhere to the DRY (Don't Repeat Yourself) principle and improve maintainability, I recommend extracting this logic into a single, reusable function in a utility module. This function can then be called from all the places where this setup is required.
| from .app_utils.typing import Feedback | ||
|
|
||
| # Enable GenAI event capture for telemetry | ||
| import os |
There was a problem hiding this comment.
According to PEP 8, imports should be grouped at the top of the file. import os is a standard library import and should be placed with other standard library imports at the beginning of the file (or at the beginning of the template block, in this case, around line 15), before third-party and local application imports.1
Style Guide References
Footnotes
-
PEP 8: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. Imports should be grouped in the following order: 1. Standard library imports. 2. Related third party imports. 3. Local application/library specific imports. ↩
Add Terraform resources to control access to GenAI telemetry logs: - Log view on _Default bucket filtered to GenAI events - Optional log exclusion to hide GenAI logs from default bucket views - IAM binding example for granting view-specific access - Documentation explaining access control approach New variable `exclude_genai_logs_from_default_bucket` (default: false): - When true: GenAI logs excluded from _Default bucket, only in BigQuery - When false: GenAI logs visible via log view to authorized users This enables role-based access control for sensitive GenAI telemetry data.
Summary
Changes
Environment variables added:
GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY=true- enables telemetry collectionOTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true- captures GenAI eventsOTEL_RESOURCE_ATTRIBUTES- includesservice.namespace(project name) andservice.version(commit SHA)ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=false- ADK-specific (disables span content)Files modified:
agent_engine_app.py- runtime env vars for Agent Enginefast_api_app.py- runtime env vars for Cloud Rundeploy.py- UI-visible env vars for Agent Engine deploymentvariables.tf- updated telemetry log filter tologName=~"gen_ai\\." AND resource.labels.namespace=PROJECT_NAMEImpact