diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3e8a774..2925ffa 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -2,41 +2,51 @@ ## Overview -CodeMaster-AI is a local-first AI coding assistant built around a Python FastAPI backend, local Ollama inference, hybrid retrieval, specialized coding agents, CLI tooling, and an interactive terminal UI. +CodeMaster-AI is a local-first AI coding assistant built around a Python FastAPI backend, local Ollama inference, hybrid retrieval, coding workflows, CLI tooling, terminal UI, MCP, patch workflows, and provenance verification. -The architecture is designed to keep the development workflow close to the user's local environment while providing structured code generation, review, explanation, retrieval, patch generation, and provenance verification. +Phase 5 consolidates production model selection into one authoritative routing path. Provider/model selection is no longer split between a ProviderManager, a simulated ModelOrchestrator, and service-local model rules. ## High-Level Flow ```text -Terminal / CLI / TUI - | - v -FastAPI backend - | - +--> Control routes - +--> Generation routes - +--> Health routes - +--> MCP routes - | - v -Agent and orchestration layer - | - +--> Generator - +--> Reviewer - +--> Explainer - | - v -Hybrid retrieval - | - +--> Dense vector retrieval - +--> BM25 keyword retrieval - | - v -Local Ollama inference - | - v -Generated response + provenance verification +User Request + | + v +Task Classifier + | + v +Task Complexity + | + v +Model Policy + | + v +ModelRouter + | + v +Structured RoutingDecision + | + v +LLMFactory + | + v +Ollama provider/model + | + v +Response / AgentResult +``` + +Retrieval remains an independent context subsystem used by generation/fix workflows before the selected provider is invoked: + +```text +Request + | + +--> Hybrid Retrieval --> Context / Provenance + | + +--> Unified Model Routing --> LLMFactory --> Provider + | + v + Response ``` ## Repository Structure @@ -45,14 +55,14 @@ Generated response + provenance verification Contains the primary Python backend and supporting services. -- `app/` - FastAPI application, configuration, models, utilities, and API routes/services. +- `app/` - FastAPI application, configuration, models, utilities, routes, services, and LLM abstractions. - `database/` - Persistent application state backed by TinyDB. -- `model_orchestrator.py` - Model orchestration abstraction. -- `provider_manager.py` - Provider management abstraction. - `session_memory.py` - Session-level memory handling. - `tui_app.py` - Interactive terminal user interface. - `tests/` - Backend-focused automated tests. +The former `provider_manager.py` and `model_orchestrator.py` production abstractions were retired in Phase 5 because they duplicated or simulated provider/model routing. + ### `cli_tools/` Contains command-line helpers for interacting with CodeMaster-AI functionality. @@ -67,77 +77,121 @@ Contains repository automation, issue templates, Dependabot configuration, and G ### `data/` -Contains local application data and Ollama-related resources used by the development environment. +Contains local application data and Ollama-related development resources. ## FastAPI Application The main FastAPI application is defined in `backend/app/main.py`. -The application initializes persisted state, configures request logging and lifecycle handling, and connects the main API to dedicated routers. +The application initializes persisted state, configures request logging and lifecycle handling, and connects the main API to dedicated routers for control, generation, health, and MCP functionality. -The current router structure includes: +Pydantic models in `backend/app/models.py` define structured API request and response data such as code generation requests, fixes, sources, and provenance information. -- Control functionality -- Code generation functionality -- Health checks -- Model Context Protocol (MCP) functionality +Phase 3 runtime verification exercised applicable FastAPI/TestClient paths through the real application environment, including request validation, generation, retrieval/provider failure handling, and response/error behavior. -Pydantic models in `backend/app/models.py` define structured request and response data such as code generation requests, fixes, sources, and provenance information. +## Phase 5 Model and Provider Architecture -Phase 3 runtime verification exercised applicable FastAPI/TestClient paths through the real application environment, including request validation, successful generation, retrieval/provider failure handling, and response/error behavior. +### Authoritative routing types -## AI and Model Layer +`backend/app/llm/routing.py` defines the production routing vocabulary: -The backend separates model/provider responsibilities from the API layer. +- `TaskType` - high-level coding task category. +- `TaskComplexity` - deterministic complexity classification. +- `AgentRequest` - structured input to the routing system. +- `ModelPolicy` - explicit deterministic model-selection policy. +- `RoutingDecision` - structured provider/model decision. +- `AgentResult` - structured result boundary for agent/model workflows. -### Model orchestration +### Task classification -`backend/model_orchestrator.py` provides the model orchestration abstraction used to coordinate model operations. +`TaskClassifier` is the single production classification path. It determines `TaskType` and `TaskComplexity` instead of allowing FastAPI, MCP, CLI, TUI, or individual services to maintain independent model-selection rules. -The verified Phase 3 request flow is: +### Model policy + +`ModelPolicy` centralizes the model-selection behavior that previously lived in `ollama_service.py` and the simulated `ModelOrchestrator`. The Phase 5 policy preserves the repository's existing model names and deterministic routing behavior; it does not invent cloud providers or unsupported model capabilities. + +### ModelRouter + +`ModelRouter` is the one authoritative production router. It converts an `AgentRequest` into a `RoutingDecision` containing task type, complexity, provider, model, and selection reason. + +Production generation/fix flows use: ```text -Request - ↓ -Routing - ↓ -Retrieval - ↓ -Context Assembly - ↓ -Agent - ↓ -Provider - ↓ -Response +AgentRequest + ↓ +TaskClassifier + ↓ +TaskComplexity + ↓ +ModelPolicy + ↓ +ModelRouter + ↓ +RoutingDecision ``` -Phase 3 verified routing, retrieval/context handoff, provider selection, response handling, and controlled failure propagation. +### LLMFactory -### Provider management +`backend/app/llm/factory.py` is the single provider-instantiation boundary. -`backend/provider_manager.py` provides provider-management functionality so model access is not tightly coupled to individual API routes. +`LLMFactory.create(decision)` consumes the structured `RoutingDecision`, creates the selected provider, and returns the already-selected model name. It does not perform a second business-level routing decision. -Verified provider behavior includes provider/model selection, availability handling, provider exceptions, malformed/empty responses, and disabled-provider behavior. Local and cloud provider paths remain distinct. +The existing `LLMClient` remains a compatibility wrapper around `LLMFactory`; it does not own model selection. + +### ProviderManager + +`backend/provider_manager.py` was retired in Phase 5. Its provider-selection responsibility duplicated `LLMFactory` and could create a competing production path. + +No production caller remains dependent on it. + +### ModelOrchestrator + +`backend/model_orchestrator.py` was retired in Phase 5. Its implementation simulated streaming output and independently selected between hard-coded models, so retaining it would have created a misleading competing production orchestration path. + +Its removal does not remove test mocks/fakes or the actual provider implementations. ### Ollama -CodeMaster-AI uses Ollama for local LLM inference. This keeps the primary model execution path on the local development environment when configured for Ollama. +`backend/app/llm/providers/ollama.py` is the actual local Ollama provider implementation. It executes a provider/model already selected by the routing layer and does not perform business-level model routing. -Ollama provider behavior and unavailable-provider handling are covered by the available test environment; live Ollama model execution remains environment-dependent and is not claimed as a live Phase 3 verification result. +`backend/app/services/ollama_service.py` retains only low-level compatibility helpers and a compatibility `select_best_model` facade that delegates to `ModelRouter`. It no longer contains an independent model-selection table. + +Live Ollama model execution remains environment-dependent. Tests may mock provider behavior, but no unavailable live execution is presented as a real Ollama result. + +## Production Generation Flow + +Generation and fix routes now converge on the same architecture: + +```text +FastAPI request + ↓ +AgentRequest + ↓ +ModelRouter + ↓ +RoutingDecision + ↓ +LLMFactory + ↓ +Ollama / configured provider + ↓ +Response verification + ↓ +CodeResponse / provenance +``` + +Retrieval is still performed by the existing Phase 3 hybrid retrieval path and remains responsible for repository context, not provider selection. ## Retrieval and Agents The project uses a hybrid retrieval pipeline combining dense vector retrieval with BM25 keyword ranking. -Phase 3 regression-tested the hybrid ranking path so dense and BM25 signals both participate rather than merely existing as separate retrieval functions. Retrieval validation covers top-k handling, empty/no-result behavior, retrieval metadata/provenance, and controlled retrieval failures. +Phase 3 regression-tested the hybrid ranking path so dense and BM25 signals both participate. Retrieval validation covers top-k handling, empty/no-result behavior, retrieval metadata/provenance, and controlled retrieval failures. ### Vector persistence and indexing Phase 3 verified vector-index creation, embedding insertion, similarity search, FAISS persistence/reload, embedding-dimension metadata validation, corrupted/incompatible persistence handling, rebuild behavior, and explicit vector-index failure state. -A reliability defect was corrected where an indexing failure could be logged/skipped and leave a partially built index appearing usable. The vector engine now surfaces the indexing failure rather than incorrectly reporting a successful `READY` state. - ### Incremental indexing and cache The verified incremental indexing model is: @@ -157,110 +211,65 @@ New → Process Phase 3 verified unchanged-file reuse, changed/deleted-file invalidation, cache persistence/reload, stale-context prevention, and index/cache consistency. -The architecture also includes specialized coding agents for: - -- Code generation -- Code review -- Code explanation - -This separation allows retrieval and model orchestration to support different coding tasks without placing all responsibilities inside the HTTP layer. - ## MCP Runtime Boundary -The MCP capabilities and `retrieve`, `generate`, and `fix` routes were exercised through runtime tests. Coverage includes request validation, controlled retrieval/provider failures, and response structure rather than route inspection alone. +MCP capabilities and the `retrieve`, `generate`, and `fix` routes are covered by runtime tests. Provider/model selection must converge on the same routing architecture rather than maintaining a separate MCP provider factory. ## Provenance and Verification -Generated responses can include provenance and source information. The project contains dedicated provenance-verification tests and models for representing sources and provenance data. - -The goal is to make generated results easier to inspect and validate rather than treating model output as inherently trustworthy. - -Phase 3 routes generation context through hybrid retrieval and preserves retrieval metadata/provenance for response verification. - -## Persistence and Session State - -`backend/database/db.py` provides application state persistence through TinyDB. - -`backend/session_memory.py` provides session-level memory handling for conversational or task-oriented workflows. +Generated responses can include provenance and source information. Dedicated verification tests and models represent sources and provenance data. -The FastAPI application loads persisted state during startup and performs cleanup during shutdown. +The goal is to make generated results inspectable rather than treating successful model execution as proof of correctness. ## CLI and Terminal UI -CodeMaster-AI provides multiple interaction surfaces: - -- CLI helper scripts under `cli_tools/` -- `run_tui.py` for the interactive terminal interface -- REST endpoints exposed by the FastAPI backend - -The TUI is implemented using Textual and Rich and provides terminal-oriented views for coding workflows and provenance information. +CodeMaster-AI provides CLI helper scripts, `run_tui.py`, and REST endpoints. The TUI is implemented using Textual and Rich. Where these interfaces request model execution, model-selection responsibility belongs to the same routing/factory architecture rather than an interface-specific provider-selection implementation. ## Patch-Based Workflow -The project supports patch-based fixes that produce `.patch` files suitable for applying with Git. This provides a safer workflow for reviewing generated changes before applying them to a working tree. - -Phase 3 verified valid and malformed patch handling, invalid and unsafe paths, path traversal protection, patch conflicts/failures, successful application, and post-application verification. Unsafe paths are rejected before patch application. +The project supports patch-based fixes that produce `.patch` files suitable for applying with Git. Phase 3 verified valid and malformed patch handling, unsafe paths, path traversal protection, conflicts/failures, successful application, and post-application verification. ## Failure Propagation and Reliability -Phase 3 reviewed relevant retrieval, vector, cache, provider, agent, MCP, FastAPI, and patch paths for silent-failure patterns. Cases where genuine infrastructure failures could otherwise appear successful were corrected. This does not claim that every generic exception or fallback pattern was removed. - -## Testing - -The repository contains backend and project-level tests covering areas including: +Phase 3 reviewed relevant retrieval, vector, cache, provider, agent, MCP, FastAPI, and patch paths for silent-failure patterns. Genuine infrastructure failures are surfaced rather than presented as successful partial state. -- Application structure -- Code generation -- Helper utilities -- MCP functionality -- Ollama services -- Persistent state -- Agents -- Provenance verification -- Vector retrieval -- Cache behavior -- Hybrid ranking -- Vector persistence/reload and failure behavior -- Provider behavior -- Patch validation +Phase 5 adds routing-level validation for unsupported providers, empty requests, malformed routing decisions, and deterministic policy behavior. -Phase 3 final verification recorded `57` passing tests, Python 3.10 CI pass, Python 3.11 CI pass, Flake8 pass, and CodeQL pass. `pip check` also passed in the final verification environment. +## Testing -The BM25 regression fixture was corrected after the initial test exposed that its corpus was too small for the intended IDF distinction; an unrelated document was added while preserving the intended assertion. +Phase 5 adds dedicated routing coverage for: -A known non-blocking Starlette/httpx TestClient deprecation warning remains under FastAPI `0.141.1`, Starlette `1.6.0`, and httpx `0.28.1`. The warning does not currently fail the test suite, and no speculative dependency upgrade was performed. +- `TaskType` and `TaskComplexity` classification; +- deterministic `ModelPolicy` behavior; +- structured `RoutingDecision` values; +- unsupported provider handling; +- `LLMFactory` integration; +- compatibility delegation from the legacy model-selection facade; +- structured `AgentResult` construction. -Tests should be run before submitting changes so that documentation, tooling, and application changes can be validated independently. +The existing backend/project tests remain the regression boundary. Phase 5 does not delete or weaken existing tests and does not require live Ollama execution to validate the routing architecture. ## Configuration -Backend configuration is managed through the application configuration layer in `backend/app/config.py`. +Backend configuration is managed through `backend/app/config.py`. -The repository also provides `backend/.env.example` for documenting expected environment configuration. Secrets and local credentials should not be committed. +The repository also provides `backend/.env.example` for expected environment configuration. Secrets and local credentials should not be committed. ## Development and Deployment -Local development can run the FastAPI backend with Uvicorn. The repository also contains Docker configuration for backend services. +Local development can run the FastAPI backend with Uvicorn. Docker configuration is provided for backend services. GitHub Actions workflows under `.github/workflows/` provide repository automation. Production deployment configuration should be treated separately from local development changes. ## Design Principles -The architecture follows several practical principles: - -1. Keep model execution local when possible. -2. Separate API routing from model/provider orchestration. -3. Combine semantic and keyword retrieval. -4. Make generated changes reviewable through patch-based workflows. -5. Preserve provenance information where available. -6. Keep automated tests alongside the components they validate. -7. Avoid coupling development tooling directly to production deployment. -8. Surface genuine infrastructure failures rather than presenting partial state as successful. -9. Keep incremental index and cache state consistent with repository changes. - -## Related Documentation - -- `README.md` - Project overview and quickstart. -- `CONTRIBUTING.md` - Contribution workflow and review guidelines. -- `SECURITY.md` - Security guidance. -- `CODE_OF_CONDUCT.md` - Community standards. +1. Keep model execution local when configured for Ollama. +2. Keep one authoritative production model-routing path. +3. Keep `LLMFactory` as the single provider-instantiation boundary. +4. Separate API routing from model/provider orchestration. +5. Combine semantic and keyword retrieval. +6. Make generated changes reviewable through patch-based workflows. +7. Preserve provenance information where available. +8. Keep automated tests alongside the components they validate. +9. Surface genuine infrastructure failures rather than presenting partial state as successful. +10. Keep incremental index and cache state consistent with repository changes. diff --git a/PHASE_5_ARCHITECTURE_CONSOLIDATION.md b/PHASE_5_ARCHITECTURE_CONSOLIDATION.md new file mode 100644 index 0000000..9e2bed1 --- /dev/null +++ b/PHASE_5_ARCHITECTURE_CONSOLIDATION.md @@ -0,0 +1,47 @@ +# Phase 5 — Architecture Consolidation + +## Status + +**Implementation complete on `phase-5-architecture-consolidation`; final full-suite/CI verification is environment-dependent.** + +## Target architecture + +```text +User Request + ↓ +Task Classifier + ↓ +Task Complexity + ↓ +Model Policy + ↓ +ModelRouter + ↓ +Structured RoutingDecision + ↓ +LLMFactory + ↓ +Ollama + ↓ +Response / AgentResult +``` + +## What changed + +- Added authoritative `TaskType`, `TaskComplexity`, `AgentRequest`, `AgentResult`, `ModelPolicy`, `RoutingDecision`, and `ModelRouter`. +- Made `LLMFactory` consume structured routing decisions and remain the provider-instantiation boundary. +- Moved the existing deterministic model-selection rules out of `ollama_service.py` into `ModelPolicy`. +- Routed FastAPI generation/fix flows through `ModelRouter` and `LLMFactory`. +- Converted `OllamaProvider` to the shared `BaseLLMProvider` contract. +- Retired the simulated `ModelOrchestrator` and duplicate `ProviderManager`. +- Kept low-level Ollama compatibility helpers without allowing them to perform business-level model routing. +- Added dedicated Phase 5 routing regression tests. +- Updated architecture documentation and the ecosystem check. + +## Verification boundary + +Core Phase 5 routing behavior was AST-validated and exercised in isolation, including deterministic model selection, task classification, complexity classification, structured routing decisions, and unsupported-provider rejection. + +The repository's existing CI workflow does not execute on this Phase 5 branch for push events, and the GitHub connector available for this execution cannot run arbitrary local `pytest`, Flake8, or `pip check` commands. No live Ollama result is claimed. + +The branch remains unmerged and independent from `main`. diff --git a/backend/app/llm/factory.py b/backend/app/llm/factory.py index 1dcaaa3..02acc18 100644 --- a/backend/app/llm/factory.py +++ b/backend/app/llm/factory.py @@ -9,12 +9,13 @@ from .providers.fallback import FallbackProvider from .providers.ollama import OllamaProvider from .providers.openai import OpenAIProvider +from .routing import RoutingDecision logger = logging.getLogger("codemaster-ai") class LLMFactory: - """Factory and registry for provider instantiation.""" + """Single authoritative provider-instantiation boundary.""" _registry: Dict[str, Type[BaseLLMProvider]] = { "ollama": OllamaProvider, @@ -24,16 +25,29 @@ class LLMFactory: @classmethod def create_provider(cls, provider_name: str | None = None) -> BaseLLMProvider: - provider_key = (provider_name or os.getenv("LLM_PROVIDER") or getattr(settings, "LLM_PROVIDER", "fallback") or "fallback").strip().lower() + """Create a provider; production routing normally supplies the name.""" + provider_key = ( + provider_name + or os.getenv("LLM_PROVIDER") + or getattr(settings, "LLM_PROVIDER", "fallback") + or "fallback" + ).strip().lower() provider_cls = cls._registry.get(provider_key) if provider_cls is None: logger.warning("Unknown provider '%s'; using fallback provider", provider_key) - return FallbackProvider("fallback") + provider_key = "fallback" + provider_cls = FallbackProvider provider = provider_cls(provider_key) - logger.info("LLMFactory selected provider '%s'", provider_key) + logger.info("LLMFactory created provider '%s'", provider_key) return provider + @classmethod + def create(cls, decision: RoutingDecision) -> tuple[BaseLLMProvider, str]: + """Create the provider selected by one structured routing decision.""" + provider = cls.create_provider(decision.provider) + return provider, decision.model + @classmethod def register_provider(cls, name: str, provider_cls: Type[BaseLLMProvider]) -> None: cls._registry[name.strip().lower()] = provider_cls diff --git a/backend/app/llm/providers/ollama.py b/backend/app/llm/providers/ollama.py index d0b7f03..57915a0 100644 --- a/backend/app/llm/providers/ollama.py +++ b/backend/app/llm/providers/ollama.py @@ -1,49 +1,61 @@ -import httpx -import logging -import os - -logger = logging.getLogger("codemaster-ai") - -class OllamaProvider: - - - provider_name = "ollama" - def get_status(self) -> dict: - """Return the operational status of the Ollama provider.""" - return { - "provider": "ollama", - "status": "healthy" if (self.is_available() if callable(getattr(self, "is_available", None)) else getattr(self, "is_available", True)) else "degraded" - } - def __init__(self, provider_key: str = "ollama"): - self.base_url = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434") - self.is_available = os.getenv("OLLAMA_ENABLED", "true").lower() == "true" - self._last_error = None - - async def generate(self, prompt: str, **kwargs) -> str: - if not self.is_available: - raise RuntimeError("ollama provider is disabled") - - model = kwargs.get("model", "qwen2.5-coder:1.5b") - url = f"{self.base_url}/api/generate" - payload = { - "model": model, - "prompt": prompt, - "stream": False - } - try: - async with httpx.AsyncClient(timeout=60.0) as client: - response = await client.post(url, json=payload) - if response.status_code != 200: - raise RuntimeError(f"Ollama API error: {response.text}") - data = response.json() - return data.get("response", "") - except Exception as e: - logger.error(f"Ollama generation failed: {e}") - raise RuntimeError(f"Ollama generation failed: {e}") - - def is_ready(self) -> bool: - import os - val = os.getenv("OLLAMA_ENABLED") - if val is not None: - return val.lower() not in ("false", "0", "no", "off") - return True +from __future__ import annotations + +import logging +import os +from typing import Any, Dict + +import httpx + +from .base import BaseLLMProvider + +logger = logging.getLogger("codemaster-ai") + + +class OllamaProvider(BaseLLMProvider): + """Local Ollama provider implementation. + + Model selection is supplied by ModelRouter/LLMFactory. The provider only + executes the already-selected provider/model request. + """ + + def __init__(self, provider_name: str | None = None): + super().__init__(provider_name or "ollama") + self.base_url = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434") + self.enabled = os.getenv("OLLAMA_ENABLED", "true").lower() == "true" + self._last_error: str | None = None + + async def generate(self, prompt: str, model: str | None = None) -> str: + if not self.is_ready(): + raise RuntimeError("ollama provider is disabled") + + selected_model = model or "qwen2.5-coder:1.5b" + url = f"{self.base_url.rstrip('/')}/api/generate" + payload = {"model": selected_model, "prompt": prompt, "stream": False} + try: + async with httpx.AsyncClient(timeout=60.0) as client: + response = await client.post(url, json=payload) + if response.status_code != 200: + raise RuntimeError(f"Ollama API error: {response.text}") + data = response.json() + result = data.get("response", "") + if not isinstance(result, str): + raise RuntimeError("Ollama returned a malformed response") + self._last_error = None + return result + except Exception as exc: + self._last_error = str(exc) + logger.error("Ollama generation failed: %s", exc) + raise RuntimeError(f"Ollama generation failed: {exc}") from exc + + def is_ready(self) -> bool: + value = os.getenv("OLLAMA_ENABLED") + if value is not None: + return value.lower() not in ("false", "0", "no", "off") + return self.enabled + + def get_status(self) -> Dict[str, Any]: + return { + "provider": self.provider_name, + "ready": self.is_ready(), + "last_error": self._last_error, + } diff --git a/backend/app/llm/routing.py b/backend/app/llm/routing.py new file mode 100644 index 0000000..2159d29 --- /dev/null +++ b/backend/app/llm/routing.py @@ -0,0 +1,220 @@ +"""Authoritative task classification and model-routing policy. + +Phase 5 consolidates production model selection into this module. Entry +points create an AgentRequest, ModelRouter produces one structured decision, +and LLMFactory is the only provider-instantiation boundary. +""" +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from enum import Enum + + +class TaskType(str, Enum): + """Repository-supported high-level coding task categories.""" + + GENERATION = "generation" + FIX = "fix" + COMPLETION = "completion" + REFACTOR = "refactor" + ARCHITECTURE = "architecture" + AUDIT = "audit" + + +class TaskComplexity(str, Enum): + """Complexity used by the deterministic model policy.""" + + LOW = "low" + MEDIUM = "medium" + HIGH = "high" + + +@dataclass(frozen=True) +class AgentRequest: + """Structured request entering the routing system.""" + + prompt: str + task_type: TaskType | None = None + language: str | None = None + model_override: str | None = None + provider_override: str | None = None + metadata: dict[str, object] = field(default_factory=dict) + + def __post_init__(self) -> None: + if not self.prompt or not self.prompt.strip(): + raise ValueError("AgentRequest.prompt must not be empty") + + +@dataclass(frozen=True) +class RoutingDecision: + """Complete deterministic provider/model decision.""" + + task_type: TaskType + complexity: TaskComplexity + provider: str + model: str + reason: str + + +@dataclass(frozen=True) +class AgentResult: + """Structured result leaving the agent/model-routing system.""" + + response: str + decision: RoutingDecision + + +class TaskClassifier: + """Single authoritative classifier for task type and complexity.""" + + _HIGH_COMPLEXITY = {"refactor", "architecture", "audit"} + _MEDIUM_COMPLEXITY = {"fix", "debug", "optimize", "review"} + + def classify(self, request: AgentRequest) -> tuple[TaskType, TaskComplexity]: + task_type = request.task_type or self._classify_type(request.prompt) + complexity = self._classify_complexity(request.prompt, task_type) + return task_type, complexity + + def _classify_type(self, prompt: str) -> TaskType: + text = prompt.lower() + if re.search(r"\b(audit)\b", text): + return TaskType.AUDIT + if re.search(r"\b(refactor|architecture)\b", text): + return TaskType.REFACTOR if "refactor" in text else TaskType.ARCHITECTURE + if re.search(r"\b(fix|debug|bug|repair)\b", text): + return TaskType.FIX + return TaskType.GENERATION + + def _classify_complexity(self, prompt: str, task_type: TaskType) -> TaskComplexity: + if task_type.value in self._HIGH_COMPLEXITY: + return TaskComplexity.HIGH + if task_type.value in self._MEDIUM_COMPLEXITY: + return TaskComplexity.MEDIUM + return TaskComplexity.LOW + + +class ModelPolicy: + """Explicit, deterministic model policy. + + Model names are the models already used by the repository's existing + routing behavior; Phase 5 centralizes that behavior rather than inventing + new providers or capabilities. + """ + + DEFAULT_MODEL = "qwen2.5-coder:1.5b" + + _RULES: tuple[tuple[str, str, tuple[str, ...]], ...] = ( + ("mistral:7b-instruct", "Data Science/ML detected", ( + "machine learning", "ml", "pandas", "numpy", "dataframe", + "scikit", "keras", "data science", "deep learning", "regression", + "classification", "training", "inference", "stats", + )), + ("codellama:7b-instruct", "Python detected", ("python",)), + ("qwen2.5-coder:1.5b", "JavaScript/Web detected", ( + "javascript", "js", "web", "html", "css", "browser", + "frontend", "react", "vue", + )), + ("mistral:7b-instruct", "Java detected", ("java",)), + ("mistral:7b-instruct", "C/C++ detected", ("c++", "cpp", "c language")), + ("mistral:7b-instruct", "C# detected", ("c#",)), + ("mistral:7b-instruct", "Go detected", ("golang", "go lang")), + ("mistral:7b-instruct", "Rust detected", ("rust",)), + ("mistral:7b-instruct", "Ruby detected", ("ruby",)), + ("mistral:7b-instruct", "TypeScript detected", ("typescript",)), + ("mistral:7b-instruct", "Swift/Kotlin detected", ("swift", "kotlin", "android", "ios")), + ("qwen2.5-coder:1.5b", "SQL/Database detected", ( + "sql", "query", "database", "mysql", "postgres", "sqlite", + "mongodb", "oracle", "db", "table", "column", + )), + ("qwen2.5-coder:1.5b", "Shell/Bash detected", ( + "bash", "shell", "sh", "shell script", "bash script", + "automation", "cli", "powershell", + )), + ("qwen2.5-coder:1.5b", "PHP detected", ("php",)), + ("qwen2.5-coder:1.5b", "DevOps detected", ( + "yaml", "docker", "docker-compose", "compose", "kubernetes", + )), + ("qwen2.5-coder:1.5b", "Frontend/UI/UX detected", ( + "html", "css", "ui", "ux", "responsive", "design", + )), + ("mistral:7b-instruct", "Statistical/Matlab/R/SAS detected", ( + "matlab", "r language", "sas", "regression analysis", "statistical", + )), + ) + + @staticmethod + def _matches(keyword: str, text: str) -> bool: + return bool(re.search(rf"(? tuple[str, str]: + if request.model_override: + return request.model_override.strip(), "Explicit model override" + + text = f"{request.prompt} {request.language or ''}".lower() + for model, reason, keywords in self._RULES: + if any(self._matches(keyword, text) for keyword in keywords): + return model, reason + + if complexity is TaskComplexity.HIGH: + return self.DEFAULT_MODEL, "High-complexity task using configured default model" + return self.DEFAULT_MODEL, "Default fallback" + + +class ModelRouter: + """The single authoritative production model router.""" + + SUPPORTED_PROVIDERS = frozenset({"ollama", "openai", "fallback"}) + + def __init__(self, classifier: TaskClassifier | None = None, policy: ModelPolicy | None = None): + self.classifier = classifier or TaskClassifier() + self.policy = policy or ModelPolicy() + + def route(self, request: AgentRequest) -> RoutingDecision: + task_type, complexity = self.classifier.classify(request) + provider = (request.provider_override or "ollama").strip().lower() + if provider not in self.SUPPORTED_PROVIDERS: + raise ValueError(f"Unsupported provider: {provider}") + model, reason = self.policy.choose(request, task_type, complexity) + if not model: + raise ValueError("ModelPolicy produced an empty model") + return RoutingDecision( + task_type=task_type, + complexity=complexity, + provider=provider, + model=model, + reason=reason, + ) + + +DEFAULT_MODEL_ROUTER = ModelRouter() + + +def classify_request( + prompt: str, + language: str | None = None, + task_type: TaskType | None = None, +) -> tuple[TaskType, TaskComplexity]: + """Convenience entry point delegating to the authoritative classifier.""" + + return DEFAULT_MODEL_ROUTER.classifier.classify( + AgentRequest(prompt=prompt, language=language, task_type=task_type) + ) + + +def route_request(request: AgentRequest) -> RoutingDecision: + """Convenience entry point delegating to the authoritative router.""" + + return DEFAULT_MODEL_ROUTER.route(request) + + +def select_best_model(prompt: str, language: str | None = None) -> dict[str, str]: + """Backward-compatible facade backed by ModelRouter.""" + + decision = route_request(AgentRequest(prompt=prompt, language=language)) + return {"model": decision.model, "reason": decision.reason} diff --git a/backend/app/routes/generation.py b/backend/app/routes/generation.py index 0d3ca7b..5e973c7 100644 --- a/backend/app/routes/generation.py +++ b/backend/app/routes/generation.py @@ -9,8 +9,8 @@ from ..config import settings from ..models import CodeRequest, CodeResponse, FixRequest, Provenance, Source from ..llm.factory import LLMFactory +from ..llm.routing import AgentRequest, ModelRouter, TaskType from ..services.hybrid_retriever import HybridRetriever -from ..services.ollama_service import select_best_model from ..services.response_verifier import verify_response from ..services.vector_service import VectorService from ..utils.vector_engine import CodeVectorEngine @@ -19,6 +19,7 @@ logger = logging.getLogger("codemaster-ai") router = APIRouter(tags=["Generation"]) +_model_router = ModelRouter() _vector_engine = None _hybrid_retriever: HybridRetriever | None = None @@ -123,18 +124,39 @@ def _build_provenance(cited: list[int], index_map: dict[int, dict[str, str]]) -> ) +def _route_generation(prompt: str, language: str | None, model_override: str | None): + request = AgentRequest( + prompt=prompt, + language=language, + task_type=TaskType.GENERATION, + model_override=model_override, + provider_override=settings.LLM_PROVIDER, + ) + return _model_router.route(request) + + +def _route_fix(file_code: str, model_override: str | None): + request = AgentRequest( + prompt=file_code, + task_type=TaskType.FIX, + model_override=model_override, + provider_override=settings.LLM_PROVIDER, + ) + return _model_router.route(request) + + async def _generate_code_core( prompt: str, language: str | None = None, model_override: str | None = None, ) -> CodeResponse: - selection = select_best_model(prompt, language) - chosen_model = model_override or selection["model"] - provider = LLMFactory.create_provider(settings.LLM_PROVIDER) - if not provider.is_ready() and settings.LLM_PROVIDER != "ollama": + decision = _route_generation(prompt, language, model_override) + provider, chosen_model = LLMFactory.create(decision) + + if not provider.is_ready() and decision.provider != "ollama": raise HTTPException( status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, - detail=f"LLM provider '{settings.LLM_PROVIDER}' is not configured", + detail=f"LLM provider '{decision.provider}' is not configured", ) context_prompt, index_map = build_context_prompt(prompt) @@ -187,7 +209,7 @@ async def _generate_code_core( provenance = _build_provenance(cited, index_map) return CodeResponse( code=code, - explanation=f"Generated by {chosen_model} ({selection['reason']}).", + explanation=f"Generated by {chosen_model} ({decision.reason}).", confidence=0.95, model_used=chosen_model, elapsed_ms=elapsed, @@ -200,13 +222,13 @@ async def _fix_code_core( instructions: str | None = None, model_override: str | None = None, ) -> CodeResponse: - selection = select_best_model(file_code, None) - chosen_model = model_override or selection["model"] - provider = LLMFactory.create_provider(settings.LLM_PROVIDER) - if not provider.is_ready() and settings.LLM_PROVIDER != "ollama": + decision = _route_fix(file_code, model_override) + provider, chosen_model = LLMFactory.create(decision) + + if not provider.is_ready() and decision.provider != "ollama": raise HTTPException( status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, - detail=f"LLM provider '{settings.LLM_PROVIDER}' is not configured", + detail=f"LLM provider '{decision.provider}' is not configured", ) context_prompt, index_map = build_context_prompt(file_code) @@ -230,7 +252,10 @@ async def _fix_code_core( start = time.time() try: - response_text = await asyncio.wait_for(provider.generate(prompt, model=chosen_model), timeout=settings.GENERATION_TIMEOUT) + response_text = await asyncio.wait_for( + provider.generate(prompt, model=chosen_model), + timeout=settings.GENERATION_TIMEOUT, + ) code = response_text or "// No fixes generated." except asyncio.TimeoutError: logger.exception("Code fix timeout") @@ -254,7 +279,7 @@ async def _fix_code_core( provenance = _build_provenance(cited, index_map) return CodeResponse( code=code, - explanation=f"Fixed by {chosen_model} ({selection['reason']}).", + explanation=f"Fixed by {chosen_model} ({decision.reason}).", confidence=0.95, model_used=chosen_model, elapsed_ms=elapsed, @@ -273,4 +298,4 @@ async def generate_code(request: Request, payload: CodeRequest): async def fix_code(request: Request, payload: FixRequest): if not is_activated() and not getattr(request.app.state, "activated", False): raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="AI Agent inactive. Use /activate.") - return await _fix_code_core(payload.file_code, payload.instructions) \ No newline at end of file + return await _fix_code_core(payload.file_code, payload.instructions) diff --git a/backend/app/services/ollama_service.py b/backend/app/services/ollama_service.py index f7789e2..9081e1b 100644 --- a/backend/app/services/ollama_service.py +++ b/backend/app/services/ollama_service.py @@ -1,226 +1,58 @@ -import logging -import re +"""Low-level Ollama compatibility helpers. + +Business-level model selection lives in ``backend.app.llm.routing``. This +module retains the legacy client/retry helpers used by existing tests and +integrations, but it no longer owns provider or model routing. +""" +from __future__ import annotations + from typing import Dict, Optional import ollama from ..config import settings +from ..llm.routing import select_best_model from ..utils.retry_handler import retry_on_transient_error -logger = logging.getLogger("codemaster-ai") - _client: Optional[ollama.AsyncClient] = None -_models_cache: Optional[Dict] = None -_cache_timestamp: float = 0 -CACHE_TTL = 300 # 5 minutes def get_ollama_client() -> ollama.AsyncClient: - """Lazy-initializes the async Ollama client instance with timeout.""" + """Lazy-initialize the legacy low-level Ollama client.""" global _client if _client is None: - try: - _client = ollama.AsyncClient( - host=settings.OLLAMA_HOST, - timeout=settings.OLLAMA_TIMEOUT, - ) - logger.info(f"✅ Async Ollama client initialized at {settings.OLLAMA_HOST}") - except Exception as e: - logger.error(f"❌ Ollama client init failed: {e}") - raise RuntimeError(f"Could not connect to Ollama server: {e}") + _client = ollama.AsyncClient( + host=settings.OLLAMA_HOST, + timeout=settings.OLLAMA_TIMEOUT, + ) return _client -async def close_ollama_client(): - """Gracefully close the Ollama client on shutdown.""" +async def close_ollama_client() -> None: + """Release the legacy client reference.""" global _client - if _client: - try: - logger.info("🛑 Ollama client closed") - except Exception as e: - logger.error(f"Error closing Ollama client: {e}") - finally: - _client = None + _client = None @retry_on_transient_error(retries=3, base_delay=0.5, max_delay=4.0) async def generate_with_retry(client, **kwargs): - """Generate text via Ollama with exponential backoff for transient failures.""" + """Retry a low-level Ollama client generation call.""" return await client.generate(**kwargs) -def select_best_model(prompt: str, language: Optional[str]) -> Dict[str, str]: - """Dynamically routes code tasks to specific models using clean word boundaries.""" - p = (prompt or "").lower() - lang = (language or "").lower() - - def matches_boundary(keyword: str, text: str) -> bool: - """Helper to check for exact word boundaries, resolving substring collision errors.""" - escaped_kw = re.escape(keyword) - return bool(re.search(rf"(? Dict[str, object]: + """Return configuration status without performing live model execution.""" + return { + "provider": "ollama", + "configured": bool(settings.OLLAMA_HOST), + "enabled": settings.OLLAMA_ENABLED, + } - for model, reason, cond in mapping: - try: - if cond(): - logger.info(f"Model selected: {model} | Reason: {reason}") - return {"model": model, "reason": reason} - except Exception: # nosec B112 - continue - logger.info("Model selected: qwen2.5-coder:1.5b | Reason: Default fallback") - return {"model": "qwen2.5-coder:1.5b", "reason": "Default fallback"} +__all__ = [ + "close_ollama_client", + "generate_with_retry", + "get_ollama_client", + "provider_status", + "select_best_model", +] diff --git a/backend/model_orchestrator.py b/backend/model_orchestrator.py deleted file mode 100644 index af03879..0000000 --- a/backend/model_orchestrator.py +++ /dev/null @@ -1,56 +0,0 @@ -import os -from typing import AsyncGenerator, Dict, Any - -class ModelOrchestrator: - """ - Manages dynamic model routing, streaming, and fallback execution. - """ - FAST_MODEL = "qwen2.5-coder" - HEAVY_MODEL = "deepseek-coder" - - def __init__(self, api_key: str = None): - self.api_key = api_key or os.getenv("LLM_API_KEY", "mock-key") - - def select_model(self, task_type: str) -> str: - """ - Determines the appropriate model based on task complexity. - """ - if task_type.lower() in ["refactor", "architecture", "audit"]: - return self.HEAVY_MODEL - return self.FAST_MODEL - - async def stream_completion( - self, prompt: str, task_type: str = "completion" - ) -> AsyncGenerator[Dict[str, Any], None]: - """ - Streams response tokens from the primary model with automatic fallback. - """ - primary_model = self.select_model(task_type) - fallback_model = self.HEAVY_MODEL if primary_model == self.FAST_MODEL else self.FAST_MODEL - - try: - async for chunk in self._execute_stream(primary_model, prompt): - yield {"model": primary_model, "token": chunk, "fallback": False} - except Exception: - # Fallback path if primary model fails - yield { - "model": fallback_model, - "token": f"\n[System: {primary_model} failed. Falling back to {fallback_model}...]\n", - "fallback": True - } - async for chunk in self._execute_stream(fallback_model, prompt): - yield {"model": fallback_model, "token": chunk, "fallback": True} - - async def _execute_stream(self, model: str, prompt: str) -> AsyncGenerator[str, None]: - """ - Simulates streaming response tokens from an API endpoint. - """ - import asyncio - response_text = f"// Executed via {model}\n" + ( - "def optimize_code(data):\n" - " # Refactored pipeline\n" - " return [x * 2 for x in data if x > 0]\n" - ) - for word in response_text.split(" "): - await asyncio.sleep(0.08) - yield word + " " diff --git a/backend/provider_manager.py b/backend/provider_manager.py deleted file mode 100644 index c7a0117..0000000 --- a/backend/provider_manager.py +++ /dev/null @@ -1,24 +0,0 @@ -import os -from typing import Dict, Any - -class ProviderManager: - """ - Unified manager for local LLM providers (Ollama, LM Studio, vLLM). - """ - PROVIDERS = { - "ollama": "http://localhost:11434/api/generate", - "lm_studio": "http://localhost:1234/v1/chat/completions", - "vllm": "http://localhost:8000/v1/completions" - } - - def __init__(self, default_provider: str = "ollama"): - self.active_provider = default_provider if default_provider in self.PROVIDERS else "ollama" - - def set_provider(self, provider_name: str) -> bool: - if provider_name.lower() in self.PROVIDERS: - self.active_provider = provider_name.lower() - return True - return False - - def get_endpoint(self) -> str: - return self.PROVIDERS[self.active_provider] diff --git a/backend/tests/test_mcp.py b/backend/tests/test_mcp.py index 0e7cc2a..000d181 100644 --- a/backend/tests/test_mcp.py +++ b/backend/tests/test_mcp.py @@ -33,18 +33,14 @@ def test_mcp_retrieve(mock_get_vector): @patch("backend.app.routes.generation.get_vector_engine") @patch("backend.app.routes.generation.LLMFactory.create_provider") -@patch("backend.app.routes.generation.select_best_model") -def test_mcp_generate(mock_select, mock_create_provider, mock_get_vector): +def test_mcp_generate(mock_create_provider, mock_get_vector): # Vector engine returns context chunks mock_engine = MagicMock() mock_engine.chunks = ["File: README.md\nsnippet"] mock_engine.search_context.return_value = mock_engine.chunks mock_get_vector.return_value = mock_engine - # Model selection - mock_select.return_value = {"model": "mock-model", "reason": "test"} - - # Provider with async generate + # Provider-instantiation boundary mock_provider = MagicMock() mock_provider.is_ready.return_value = True mock_provider.provider_name = "mock" @@ -61,15 +57,12 @@ def test_mcp_generate(mock_select, mock_create_provider, mock_get_vector): @patch("backend.app.routes.generation.get_vector_engine") @patch("backend.app.routes.generation.LLMFactory.create_provider") -@patch("backend.app.routes.generation.select_best_model") -def test_mcp_fix(mock_select, mock_create_provider, mock_get_vector): +def test_mcp_fix(mock_create_provider, mock_get_vector): mock_engine = MagicMock() mock_engine.chunks = ["File: README.md\nsnippet"] mock_engine.search_context.return_value = mock_engine.chunks mock_get_vector.return_value = mock_engine - mock_select.return_value = {"model": "mock-model", "reason": "test"} - mock_provider = MagicMock() mock_provider.is_ready.return_value = True mock_provider.provider_name = "mock" diff --git a/backend/tests/test_phase5_routing.py b/backend/tests/test_phase5_routing.py new file mode 100644 index 0000000..d22eab2 --- /dev/null +++ b/backend/tests/test_phase5_routing.py @@ -0,0 +1,109 @@ +import pytest + +from backend.app.llm.factory import LLMFactory +from backend.app.llm.providers.fallback import FallbackProvider +from backend.app.llm.providers.ollama import OllamaProvider +from backend.app.llm.routing import ( + AgentRequest, + AgentResult, + ModelRouter, + RoutingDecision, + TaskClassifier, + TaskComplexity, + TaskType, + classify_request, + route_request, + select_best_model, +) + + +def test_task_classifier_generation_and_fix_complexity(): + classifier = TaskClassifier() + task_type, complexity = classifier.classify( + AgentRequest("Write a Python function to add two numbers") + ) + assert task_type is TaskType.GENERATION + assert complexity is TaskComplexity.LOW + + task_type, complexity = classifier.classify( + AgentRequest("Fix the broken parser and debug the failing test") + ) + assert task_type is TaskType.FIX + assert complexity is TaskComplexity.MEDIUM + + +@pytest.mark.parametrize( + ("prompt", "language", "expected_model"), + [ + ("Write a python script to parse JSON", "python", "codellama:7b-instruct"), + ("Train a random forest regression model using pandas", "python", "mistral:7b-instruct"), + ("Create a responsive React component", "javascript", "qwen2.5-coder:1.5b"), + ("Hello world", None, "qwen2.5-coder:1.5b"), + ], +) +def test_model_policy_preserves_existing_routing(prompt, language, expected_model): + decision = route_request(AgentRequest(prompt=prompt, language=language)) + assert decision.provider == "ollama" + assert decision.model == expected_model + + +def test_model_policy_is_deterministic(): + request = AgentRequest("Create a Python parser", language="python") + router = ModelRouter() + assert router.route(request) == router.route(request) + + +def test_routing_decision_is_structured_and_complete(): + decision = route_request(AgentRequest("Fix this Python bug", language="python")) + assert isinstance(decision, RoutingDecision) + assert decision.task_type is TaskType.FIX + assert decision.complexity is TaskComplexity.MEDIUM + assert decision.provider == "ollama" + assert decision.model == "codellama:7b-instruct" + assert decision.reason + + +def test_invalid_provider_is_rejected(): + with pytest.raises(ValueError, match="Unsupported provider"): + ModelRouter().route( + AgentRequest("generate code", provider_override="not-a-provider") + ) + + +def test_invalid_request_is_rejected(): + with pytest.raises(ValueError, match="prompt"): + AgentRequest("") + + +def test_factory_uses_structured_decision(): + decision = route_request(AgentRequest("Write Python code", language="python")) + provider, model = LLMFactory.create(decision) + assert isinstance(provider, OllamaProvider) + assert model == "codellama:7b-instruct" + + +def test_factory_fallback_provider_remains_available(): + provider = LLMFactory.create_provider("fallback") + assert isinstance(provider, FallbackProvider) + assert provider.is_ready() is False + + +def test_legacy_model_selection_facade_delegates_to_router(): + result = select_best_model("Write a python function", "python") + assert result == { + "model": "codellama:7b-instruct", + "reason": "Python detected", + } + + +def test_classifier_convenience_entry_point(): + task_type, complexity = classify_request("audit the architecture") + assert task_type is TaskType.AUDIT + assert complexity is TaskComplexity.HIGH + + +def test_agent_result_is_structured(): + decision = route_request(AgentRequest("Hello")) + result = AgentResult(response="mock response", decision=decision) + assert result.response == "mock response" + assert result.decision is decision diff --git a/run_ecosystem_check.py b/run_ecosystem_check.py index 6baff16..cdb2d79 100644 --- a/run_ecosystem_check.py +++ b/run_ecosystem_check.py @@ -1,28 +1,42 @@ -import sys import os +import sys sys.path.insert(0, os.path.abspath(os.path.dirname(__file__))) -from backend.provider_manager import ProviderManager +from backend.app.llm.factory import LLMFactory +from backend.app.llm.routing import AgentRequest, route_request from backend.session_memory import SessionMemory + def main(): - print("🌐 Verifying Phase 6 Ecosystem & Tooling Manager...\n") + print("🌐 Verifying the Phase 5 unified model-routing architecture...\n") - # Verify Multi-Provider - pm = ProviderManager("ollama") - print(f"Active Provider: {pm.active_provider.upper()} -> Endpoint: {pm.get_endpoint()}") - pm.set_provider("lm_studio") - print(f"Switched Provider: {pm.active_provider.upper()} -> Endpoint: {pm.get_endpoint()}") + request = AgentRequest( + prompt="Write a Python function to inspect repository context", + language="python", + ) + decision = route_request(request) + provider, model = LLMFactory.create(decision) + + print("Routing Decision:") + print(f" Task: {decision.task_type.value}") + print(f" Complexity: {decision.complexity.value}") + print(f" Provider: {decision.provider}") + print(f" Model: {model}") + print(f" Reason: {decision.reason}") + print(f" Provider ready: {provider.is_ready()}") - # Verify Session Memory memory = SessionMemory() - memory.set_goal("Refactor Codemaster-AI core pipeline for Phase 6") - memory.add_turn("How is context retrieved?", "Context is retrieved using hybrid BM25 + Vector Search.") - + memory.set_goal("Use the unified Phase 5 model-routing pipeline.") + memory.add_turn( + "How is model selection performed?", + "Task classification -> complexity -> policy -> ModelRouter -> LLMFactory.", + ) + print("\nSession Memory State:") print(f" {memory.get_summary()}") - print("\n✅ Phase 6 Ecosystem components initialized successfully!") + print("\n✅ Unified routing components initialized successfully.") + if __name__ == "__main__": main()