Skip to content

Conversation

@ASCE-D
Copy link
Contributor

@ASCE-D ASCE-D commented Nov 10, 2025

Summary by CodeRabbit

  • New Features

    • Added LLM tracing and monitoring capabilities for local development environments
    • Added configurable code provider support with environment variables
  • Documentation

    • Updated setup instructions with local development tracing configuration
  • Chores

    • Added new dependencies for tracing instrumentation and support libraries

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 10, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

Introduces comprehensive Phoenix tracing infrastructure for LLM monitoring across the application. Adds environment configuration variables, a centralized tracing module with initialization functions, setup calls in main app and Celery workers, agent instrumentation, documentation, and required dependencies.

Changes

Cohort / File(s) Summary
Environment & Configuration
.env.template
Adds Phoenix tracing config block with PHOENIX_ENABLED, PHOENIX_COLLECTOR_ENDPOINT, PHOENIX_PROJECT_NAME, and code provider settings (CODE_PROVIDER, CODE_PROVIDER_BASE_URL, CODE_PROVIDER_TOKEN).
Documentation
README.md
Adds optional Phoenix Tracing Setup section for local development with commands to run phoenix serve and access tracing dashboard at http://localhost:6006.
Core App Initialization
app/main.py, app/celery/celery_app.py
Adds setup_phoenix_tracing() method/function to initialize Phoenix tracing at startup in both the FastAPI app and Celery worker contexts.
Tracing Module
app/modules/intelligence/tracing/phoenix_tracer.py
New module providing initialize_phoenix_tracing(), get_tracer(), is_phoenix_enabled(), and shutdown_phoenix_tracing() functions. Configures tracer provider, OTLP exporter, span processors, and instrumentation for Pydantic AI and LiteLLM.
Agent Instrumentation
app/modules/intelligence/agents/chat_agents/pydantic_agent.py
Adds "instrument": True to agent configuration to enable telemetry/metrics collection.
Dependencies
requirements.txt
Adds Arize Phoenix packages (arize-phoenix, arize-phoenix-client, arize-phoenix-evals, arize-phoenix-otel), OpenInference instrumentation packages (openinference-instrumentation-litellm, openinference-instrumentation-openai, openinference-instrumentation-pydantic-ai), and anthropic.
Minor Formatting
app/modules/integrations/integrations_router.py
Line-wrapping adjustment in linear_oauth_callback with no functional behavior change.

Sequence Diagram(s)

sequenceDiagram
    participant App as MainApp/Celery
    participant Setup as setup_phoenix_tracing()
    participant Phoenix as initialize_phoenix_tracing()
    participant Provider as TraceProvider
    participant Exporter as OTLP Exporter
    participant Agent as Pydantic Agent
    participant Collector as Phoenix Collector<br/>(localhost:6006)

    rect rgb(230, 245, 230)
    Note over App,Setup: Initialization Phase (Startup)
    end
    
    App->>Setup: Call setup_phoenix_tracing()
    activate Setup
    
    Setup->>Phoenix: initialize_phoenix_tracing()<br/>(from env config)
    activate Phoenix
    
    alt PHOENIX_ENABLED == true
        Phoenix->>Provider: Create TracerProvider
        Phoenix->>Exporter: Configure OTLP Exporter<br/>(endpoint from env)
        Phoenix->>Provider: Add span processors
        Phoenix->>Provider: Instrument LiteLLM
        Phoenix-->>Setup: return True
    else PHOENIX_ENABLED == false or ImportError
        Phoenix-->>Setup: return False/warn
    end
    
    deactivate Phoenix
    deactivate Setup

    rect rgb(240, 245, 255)
    Note over Agent,Collector: Runtime Phase (Agent Execution)
    end
    
    Agent->>Agent: Execute with<br/>instrument=True
    Agent->>Provider: Create spans
    Provider->>Exporter: Send trace data
    Exporter->>Collector: POST traces
    Collector-->>Collector: Store & display traces
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • New module complexity: phoenix_tracer.py introduces integration with multiple OpenTelemetry/Arize libraries; verify proper error handling, initialization order, and environment variable defaults.
  • Initialization points: Review the setup calls in both main.py and celery_app.py to ensure graceful degradation if Phoenix is disabled or dependencies are missing.
  • Dependencies: Confirm all added packages in requirements.txt are compatible with current Python version and existing dependencies.
  • Agent instrumentation: Verify that "instrument": True doesn't introduce unexpected side effects or performance implications in the Pydantic agent.

Possibly related PRs

  • cleanup up version #482: Implements the same Phoenix tracing additions (environment variables, README documentation, tracing module, initialization in main and Celery, requirements, and agent instrumentation).

Poem

🐰 Traces now dance through Phoenix's flame,
LLM whispers, captured without shame,
From Celery's heart to Pydantic's mind,
At localhost six-oh-oh-six, truths aligned!

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Title check ⚠️ Warning The title 'cleanup up version' does not accurately describe the main changes in this PR, which add Phoenix tracing integration across multiple files and dependencies. Consider revising the title to reflect the primary change, such as 'Add Phoenix tracing integration for LLM monitoring' or 'Integrate Arize Phoenix for distributed tracing'.
Docstring Coverage ⚠️ Warning Docstring coverage is 68.18% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1fe4b66 and b787a45.

📒 Files selected for processing (4)
  • .env.template (1 hunks)
  • README.md (1 hunks)
  • app/main.py (2 hunks)
  • app/modules/integrations/integrations_router.py (1 hunks)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
README.md (1)

227-233: Use a proper markdown heading instead of bold text.

The Phoenix tracing setup section uses bold text for the title. Per markdown best practices, use a proper heading level (e.g., ###) instead.

Apply this diff:

-**Optional: Phoenix Tracing Setup (Local Only)**
-
+### Optional: Phoenix Tracing Setup (Local Only)
+

Based on static analysis hints.

app/main.py (1)

67-75: Avoid swallowing unexpected Phoenix init failures

initialize_phoenix_tracing() already guards against missing deps and logs rich errors, but this extra except Exception will also swallow programming/configuration bugs (e.g., bad env wiring), making debugging painful. Import the helper and call it directly; rely on its return value/logging instead of catching everything.

Apply this diff:

     def setup_phoenix_tracing(self):
-        try:
-            from app.modules.intelligence.tracing.phoenix_tracer import (
-                initialize_phoenix_tracing,
-            )
-
-            initialize_phoenix_tracing()
-        except Exception as e:
-            logging.warning(f"Phoenix tracing initialization failed (non-fatal): {e}")
+        from app.modules.intelligence.tracing.phoenix_tracer import (
+            initialize_phoenix_tracing,
+        )
+
+        initialize_phoenix_tracing()
app/celery/celery_app.py (1)

84-95: Let the Phoenix initializer report failures itself

As in the web app, the helper already logs and soft-fails when tracing can’t start. Catching Exception here hides stack traces for unexpected bugs. Import and invoke it directly to preserve diagnostics.

Apply this diff:

 def setup_phoenix_tracing():
     """Initialize Phoenix tracing for LLM monitoring in Celery workers."""
-    try:
-        from app.modules.intelligence.tracing.phoenix_tracer import (
-            initialize_phoenix_tracing,
-        )
-
-        initialize_phoenix_tracing()
-    except Exception as e:
-        logger.warning(
-            f"Phoenix tracing initialization failed in Celery worker (non-fatal): {e}"
-        )
+    from app.modules.intelligence.tracing.phoenix_tracer import (
+        initialize_phoenix_tracing,
+    )
+
+    initialize_phoenix_tracing()
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2614c14 and a25da69.

📒 Files selected for processing (19)
  • .env.template (1 hunks)
  • README.md (1 hunks)
  • app/celery/celery_app.py (1 hunks)
  • app/main.py (2 hunks)
  • app/modules/intelligence/agents/chat_agents/pydantic_agent.py (1 hunks)
  • app/modules/intelligence/tools/jira_tools/README.md (1 hunks)
  • app/modules/intelligence/tools/jira_tools/add_jira_comment_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/create_jira_issue_tool.py (4 hunks)
  • app/modules/intelligence/tools/jira_tools/get_jira_issue_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/get_jira_project_details_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/get_jira_project_users_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/get_jira_projects_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/link_jira_issues_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/search_jira_issues_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/transition_jira_issue_tool.py (2 hunks)
  • app/modules/intelligence/tools/jira_tools/update_jira_issue_tool.py (2 hunks)
  • app/modules/intelligence/tools/tool_service.py (1 hunks)
  • app/modules/intelligence/tracing/phoenix_tracer.py (1 hunks)
  • requirements.txt (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
app/celery/celery_app.py (2)
app/main.py (1)
  • setup_phoenix_tracing (67-75)
app/modules/intelligence/tracing/phoenix_tracer.py (1)
  • initialize_phoenix_tracing (44-170)
app/modules/intelligence/tools/tool_service.py (2)
app/modules/intelligence/tools/jira_tools/get_jira_project_details_tool.py (1)
  • get_jira_project_details_tool (89-120)
app/modules/intelligence/tools/jira_tools/get_jira_project_users_tool.py (1)
  • get_jira_project_users_tool (107-135)
app/main.py (2)
app/celery/celery_app.py (1)
  • setup_phoenix_tracing (84-95)
app/modules/intelligence/tracing/phoenix_tracer.py (1)
  • initialize_phoenix_tracing (44-170)
app/modules/intelligence/tracing/phoenix_tracer.py (1)
app/modules/utils/logger.py (1)
  • setup_logger (4-15)
🪛 dotenv-linter (4.0.0)
.env.template

[warning] 78-78: [UnorderedKey] The PHOENIX_COLLECTOR_ENDPOINT key should go before the PHOENIX_ENABLED key

(UnorderedKey)

🪛 markdownlint-cli2 (0.18.1)
README.md

227-227: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

🪛 Ruff (0.14.3)
app/celery/celery_app.py

92-92: Do not catch blind exception: Exception

(BLE001)

app/main.py

74-74: Do not catch blind exception: Exception

(BLE001)

app/modules/intelligence/tracing/phoenix_tracer.py

158-158: Consider moving this statement to an else block

(TRY300)


229-229: Do not catch blind exception: Exception

(BLE001)


230-230: Use logging.exception instead of logging.error

Replace with exception

(TRY400)

🔇 Additional comments (14)
app/modules/intelligence/tools/jira_tools/search_jira_issues_tool.py (1)

34-56: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 125-140

app/modules/intelligence/tools/jira_tools/add_jira_comment_tool.py (1)

28-42: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 101-114

app/modules/intelligence/tools/jira_tools/create_jira_issue_tool.py (1)

45-75: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 176-196

app/modules/intelligence/tools/jira_tools/link_jira_issues_tool.py (1)

31-52: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 114-130

app/modules/intelligence/tools/jira_tools/get_jira_projects_tool.py (1)

28-43: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 106-117

app/modules/intelligence/tools/jira_tools/update_jira_issue_tool.py (1)

41-57: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 151-167

app/modules/intelligence/tools/jira_tools/transition_jira_issue_tool.py (1)

30-46: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 127-140

app/modules/intelligence/tools/jira_tools/get_jira_project_users_tool.py (1)

37-48: LGTM! Formatting improvements enhance readability.

The added blank lines improve the structure and readability of the tool descriptions without any functional changes.

Also applies to: 123-133

app/modules/intelligence/tools/jira_tools/README.md (1)

24-24: LGTM!

Formatting cleanup looks good.

app/modules/intelligence/tools/jira_tools/get_jira_issue_tool.py (1)

28-34: LGTM!

Docstring formatting improvements enhance readability.

Also applies to: 94-103

app/modules/intelligence/tools/jira_tools/get_jira_project_details_tool.py (1)

28-39: LGTM!

Consistent docstring formatting improvements.

Also applies to: 106-117

app/modules/intelligence/tools/tool_service.py (1)

122-127: LGTM!

Multi-line formatting improves consistency with the rest of the file.

.env.template (1)

74-80: Phoenix tracer implementation files are present in the codebase—no action needed.

Verification confirms that app/modules/intelligence/tracing/phoenix_tracer.py, app/main.py (with setup_phoenix_tracing() and initialize_phoenix_tracing() calls), and app/celery/celery_app.py (with setup_phoenix_tracing() implementation) all exist. The environment variables in .env.template correctly reference implementation that is already in place.

app/modules/intelligence/agents/chat_agents/pydantic_agent.py (1)

142-142: Instrumentation parameter is supported—no issues found.

The pydantic-ai Agent initializer accepts an instrument parameter (InstrumentationSettings | bool | None), confirming that "instrument": True is a valid and supported configuration for version 0.4.3.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
app/modules/intelligence/tracing/phoenix_tracer.py (3)

41-41: Consider thread-safety for concurrent initialization.

The global _PHOENIX_INITIALIZED flag is not thread-safe. If multiple workers or threads call initialize_phoenix_tracing() concurrently during startup, race conditions could occur. While the documented usage pattern ("initialize once at application startup") mitigates this, consider using a threading.Lock() for defensive programming in distributed environments like Celery.

Example:

import threading

_PHOENIX_INITIALIZED = False
_INIT_LOCK = threading.Lock()

Then in initialize_phoenix_tracing():

with _INIT_LOCK:
    if _PHOENIX_INITIALIZED:
        logger.info("Phoenix tracing already initialized")
        return True
    # ... rest of initialization

192-198: Clarify the behavior when Phoenix is not initialized.

The log message says "returning default tracer" when Phoenix is not initialized, but the function always returns trace.get_tracer(name) regardless. This is likely correct (OpenTelemetry returns a no-op tracer when not configured), but the comment could be clearer about this behavior.

Consider updating the log message:

-            logger.debug("Phoenix not initialized, returning default tracer")
+            logger.debug("Phoenix not initialized, returning no-op tracer")

232-233: Use logging.exception for cleaner exception logging.

When logging an exception in an except block, logging.exception is more idiomatic than logging.error with exc_info=True as it automatically includes the traceback.

Apply this diff:

     except Exception as e:
-        logger.error("Error shutting down Phoenix tracing: %s", e)
+        logger.exception("Error shutting down Phoenix tracing")

Based on static analysis hints.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a25da69 and 8ae45df.

📒 Files selected for processing (2)
  • app/modules/intelligence/tracing/phoenix_tracer.py (1 hunks)
  • requirements.txt (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
app/modules/intelligence/tracing/phoenix_tracer.py (1)
app/modules/utils/logger.py (1)
  • setup_logger (4-15)
🪛 Ruff (0.14.3)
app/modules/intelligence/tracing/phoenix_tracer.py

161-161: Consider moving this statement to an else block

(TRY300)


232-232: Do not catch blind exception: Exception

(BLE001)


233-233: Use logging.exception instead of logging.error

Replace with exception

(TRY400)

🔇 Additional comments (3)
app/modules/intelligence/tracing/phoenix_tracer.py (3)

146-152: Previous review feedback addressed correctly!

The auto_instrument flag is now properly respected. LiteLLM instrumentation is correctly gated by the flag, and there's appropriate logging for both branches.


100-105: Parameter precedence is well-implemented.

The configuration precedence (function parameter > environment variable > default) is clearly documented and correctly implemented.


205-207: LGTM!

Clean implementation of the status check.

@nndn nndn self-requested a review November 17, 2025 10:42
@nndn nndn merged commit c9a2acb into main Nov 17, 2025
@nndn nndn deleted the feat-phoenix branch November 17, 2025 10:43
@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
3 Security Hotspots

See analysis details on SonarQube Cloud

@coderabbitai coderabbitai bot mentioned this pull request Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants