Update local llama stack config

[carbonin]

With the latest images this doesn't work anymore, make the configuration more like what we use in the SaaS template

Summary by CodeRabbit

Chores

• Configuration system updated to use environment variables for database credentials, improving security and deployment flexibility.
• Telemetry and logging settings revised for enhanced system monitoring and observability.
• Storage paths standardized across components for consistency.
• Service endpoint configuration adjusted.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: carbonin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [carbonin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Walkthrough

Configuration file updated to migrate database connection strings from hardcoded localhost values to environment variables, update telemetry service configuration with new sink settings and paths, reorganize model definitions with new host/port/db/user/password fields, and change MCP endpoint URI from service hostname to localhost.

Changes

Cohort / File(s)	Summary
Database Connection Configuration config/llama_stack_client_config.yaml	metadata_store, responses_store, and inference_store connection details updated to use ASSISTED_CHAT_* environment variables instead of hardcoded localhost credentials; ssl_mode disabled and ca_cert_path added to all three stores.
Telemetry and Storage config/llama_stack_client_config.yaml	service_name set to "assisted-chat"; sinks changed to console and sqlite; sqlite_db_path updated to STORAGE_MOUNT_PATH/sqlite/trace_store.db; metadata_store db_path updated to STORAGE_MOUNT_PATH/sqlite/registry.db.
API and Model Configuration config/llama_stack_client_config.yaml	api_key changed from env.GEMINI_API_KEY to dummy string; model definitions replaced with empty list; new host/port/db/user/password fields added for model management; MCP endpoint URI changed from http://assisted-service-mcp:8000/mcp to http://localhost:8000/mcp.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Primary focus: configuration value migrations with repetitive patterns (environment variable replacements, connection string updates)
• Special attention: MCP endpoint hostname change and telemetry service name change to verify these are intentional environment-specific modifications

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "Update local llama stack config" is directly related to the changeset, which exclusively modifies the llama_stack_client_config.yaml file. The title accurately summarizes the primary action being performed: updating the local llama stack configuration. It is concise and readable, and a teammate reviewing the commit history would understand that this involves a configuration update. While the title could be more specific about which configuration aspects are being updated or the reason for the change (compatibility with latest images), it clearly and accurately conveys the main change in the changeset.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (2)

config/llama_stack_client_config.yaml (2)

29-35: Verify all required environment variables are properly configured in the runtime environment.

The configuration depends on multiple environment variables (ASSISTED_CHAT_POSTGRES_HOST, ASSISTED_CHAT_POSTGRES_PORT, ASSISTED_CHAT_POSTGRES_NAME, ASSISTED_CHAT_POSTGRES_USER, ASSISTED_CHAT_POSTGRES_PASSWORD, STORAGE_MOUNT_PATH). Ensure these are:

1. Documented in setup/deployment guides
2. Set in local development, CI/CD, and production environments
3. Not accidentally leaked in logs or configs

Consider adding a validation step (e.g., startup check) to catch missing environment variables early.

Also applies to: 38-44, 70-76

---

34-34: Confirm SSL is intentionally disabled for local development.

All PostgreSQL connections have ssl_mode: disable and ca_cert_path: /etc/tls/ca-bundle.pem. While acceptable for local development, confirm this is not used in staging/production environments. Consider:

1. Adding environment-specific configs or conditional SSL settings
2. Documenting that SSL must be enabled for production deployments
3. Defaulting to ssl_mode: require for safety

Also applies to: 43-43, 75-75

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9dc8780 and 77450eb.

📒 Files selected for processing (1)

• config/llama_stack_client_config.yaml (4 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: Red Hat Konflux / assisted-chat-saas-main-on-pull-request
• GitHub Check: Red Hat Konflux / assisted-chat-test-image-saas-main-on-pull-request

🔇 Additional comments (4)

config/llama_stack_client_config.yaml (4)

19-19: Verify API key placeholder is appropriate for Vertex integration.

The api_key is set to a placeholder string with a comment indicating Vertex is used instead of Gemini directly. Confirm this placeholder prevents errors during Llama stack initialization and that the actual Vertex authentication is handled elsewhere (e.g., via GOOGLE_APPLICATION_CREDENTIALS or similar).

---

67-67: Verify STORAGE_MOUNT_PATH environment variable is consistently used.

The metadata_store db_path references ${STORAGE_MOUNT_PATH} which also appears on line 51 (telemetry sqlite_db_path). Ensure this path is writable and persistent in your local/production environment, and document its purpose and expected location.

---

77-77: Clarify why the models list is now empty.

The models configuration has been changed from populated entries to an empty list (models: []). Confirm this is intentional and does not represent an incomplete migration. If models are defined elsewhere (e.g., environment variables, separate config file), document that reference.

---

87-87: Verify MCP endpoint localhost change works in your local setup.

The MCP endpoint URI changed from http://assisted-service-mcp:8000/mcp (service hostname) to http://localhost:8000/mcp (localhost). This assumes:

1. The MCP service is accessible on localhost:8000 in local development
2. No cross-container networking is required (or port forwarding is configured)

Confirm this works in your containerized local environment and document the setup if there are special networking requirements.

[openshift-ci]

@carbonin: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/eval-test	77450eb	link	false	/test eval-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[omertuc]

@luis5tb is that OK? You mentioned you're still using Gemini without vertex

[luis5tb]

that means it requires to be added later on. My question here is that it seems a changed not mentioned in the PR. It it was because it was failing, perhaps just need to move from gemini/gemini-XXX to models/gemini-XXX

[omertuc]

Same here