🧠 Implement expandable reasoning system with enhanced UI and agent capabilities

README.md

@@ -42,9 +42,9 @@ uvx --from codetide codetide-cli --help
 ```
 ## AgentTide
 
-AgentTide consists of a demo, showing how CodeTide can integrate with LLMs and augment code generation and condebase related workflows. If you ask Tide to describe himself, he will say something like this: I'm the next-generation, precision-driven software engineering agent built on top of CodeTide. You can use it via the command-line interface (CLI) or a beautiful interactive UI.
+AgentTide is a next-generation, precision-driven software engineering agent built on top of CodeTide. It is ready to help you dig deep into your codebase, automate code changes, and provide intelligent, context-aware assistance. You can use it via the command-line interface (CLI) or a beautiful interactive UI.
 
-> **Demo available:** Try AgentTide live on Hugging Face Spaces: [https://mclovinittt-agenttidedemo.hf.space/](https://mclovinittt-agenttidedemo.hf.space/)
+> **Try AgentTide live:** [https://mclovinittt-agenttidedemo.hf.space/](https://mclovinittt-agenttidedemo.hf.space/)
 
 ---
 
@@ -57,46 +57,44 @@ AgentTide consists of a demo, showing how CodeTide can integrate with LLMs and a
 
 **AgentTide CLI**
 
-To use the AgentTide conversational CLI, you must install the `[agents]` extra and launch via:
+To use the AgentTide conversational CLI, install the `[agents]` extra and launch via:
 
 ```sh
 uvx --from codetide[agents] agent-tide
 ```
 
-This will start an interactive terminal session with AgentTide.
-
-You can also pass the `--project_path` argument to start AgentTide on a specific path:
+This starts an interactive terminal session with AgentTide. You can specify a project path:
 
 ```sh
 uvx --from codetide[agents] agent-tide --project_path /path/to/your/project
 ```
 
-If you do not provide the `--project_path` argument, AgentTide will start in the current directory by default.
+If `--project_path` is not provided, AgentTide starts in the current directory.
 
 **AgentTide UI**
 
-To use the AgentTide web UI, you must install the `[agents-ui]` extra and launch via:
+To use the AgentTide web UI, install the `[agents-ui]` extra and launch:
 
 ```sh
 uvx --from codetide[agents-ui] agent-tide-ui
 ```
 
-This will start a web server for the AgentTide UI. Follow the on-screen instructions to interact with the agent in your browser at [http://localhost:9753](http://localhost:9753) (or the port you specified)
+This starts a web server for the AgentTide UI. Interact with the agent in your browser at [http://localhost:9753](http://localhost:9753) (or your specified port).
 
-### Why Try AgentTide? ([Full Guide & Tips Here](codetide/agents/tide/ui/chainlit.md))
+### Why Use AgentTide? ([Full Guide & Tips Here](codetide/agents/tide/ui/chainlit.md))
 
-**Local-First & Private:** All code analysis and patching is performed locally. Your code never leaves your machine.
+- **Local-First & Private:** All code analysis and patching is performed locally. Your code never leaves your machine.
 - **Transparent & Stepwise:** See every plan and patch before it's applied. Edit, reorder, or approve steps—you're always in control.
 - **Context-Aware:** AgentTide loads only the relevant code identifiers and dependencies for your request, making it fast and precise.
 - **Human-in-the-Loop:** After each step, review the patch, provide feedback, or continue—no black-box agent behavior.
 - **Patch-Based Editing:** All changes are atomic diffs, not full file rewrites, for maximum clarity and efficiency.
 
 **Usage Tips:**  
 - If you know the exact code context, specify identifiers directly in your request (e.g., `module.submodule.file_withoutextension.object`).  
-- You can use the `plan` command to generate a step-by-step implementation plan for your request, review and edit the plan, and then proceed step-by-step.
-- The `commit` command allows you to review and finalize changes before they are applied.
+- Use the `plan` command to generate a step-by-step implementation plan for your request, review and edit the plan, and then proceed step-by-step.
+- Use the `commit` command to review and finalize changes before they are applied.
 
- See the [chainlit.md](codetide/agents/tide/ui/chainlit.md) for full details and advanced workflows, including the latest specifications for these commands!
+See [chainlit.md](codetide/agents/tide/ui/chainlit.md) for full details and advanced workflows, including the latest specifications for these commands!
 
 ---
 
@@ -156,7 +154,7 @@ CodeTide provides the following tools for agents:
 2. **`getRepoTree`**: Explore the repository structure.
 
 #### Example: Initializing an LLM with CodeTide
-Here’s a snippet from `agent_tide.py` demonstrating how to initialize an LLM with CodeTide as an MCP server:
+Here's a snippet from `agent_tide.py` demonstrating how to initialize an LLM with CodeTide as an MCP server:
 
 ```python
 from aicore.llm import Llm, LlmConfig
@@ -176,7 +174,7 @@ def init_llm() -> Llm:
     return llm
 ```
 
-This setup allows the LLM to leverage CodeTide’s tools for codebase interactions.
+This setup allows the LLM to leverage CodeTide's tools for codebase interactions.
 
 CodeTide can now be used as an MCP Server! This allows seamless integration with AI tools and workflows. Below are the tools available:
 The available tools are:
@@ -517,7 +515,7 @@ if __name__ == "__main__":
 
 ## 🧠 Philosophy
 
-CodeTide is about giving developers structure-aware tools that are **fast, predictable, and private**. Your code is parsed, navigated, and queried as a symbolic graph - not treated as a black box of tokens. Whether you’re building, refactoring, or feeding context into an LLM - **you stay in control**.
+CodeTide is about giving developers structure-aware tools that are **fast, predictable, and private**. Your code is parsed, navigated, and queried as a symbolic graph - not treated as a black box of tokens. Whether you're building, refactoring, or feeding context into an LLM - **you stay in control**.
 
 > Like a tide, your codebase evolves - and CodeTide helps you move with it, intelligently.
 
@@ -539,7 +537,7 @@ Instead, it uses:
 
 ## 🗺️ Roadmap
 
-Here’s what’s next for CodeTide:
+Here's what's next for CodeTide:
 
 - 🧩 **Support more languages** already integrated with [Tree-sitter](https://tree-sitter.github.io/tree-sitter/)  
   → **TypeScript** is the top priority. **Now available in Beta**
@@ -554,11 +552,11 @@ Here’s what’s next for CodeTide:
 
 ## 🤖 Agents Module: AgentTide
 
-> **Demo available:** Try AgentTide live on Hugging Face Spaces: [https://mclovinittt-agenttidedemo.hf.space/](https://mclovinittt-agenttidedemo.hf.space/)
+> **Try AgentTide live:** [https://mclovinittt-agenttidedemo.hf.space/](https://mclovinittt-agenttidedemo.hf.space/)
 
-CodeTide now includes an `agents` module, featuring **AgentTide**—a precision-driven software engineering agent that connects directly to your codebase and executes your requests with full code context.
+CodeTide now includes an `agents` module, featuring **AgentTide**—a production-ready, precision-driven software engineering agent that connects directly to your codebase and executes your requests with full code context.
 
-**AgentTide** leverages CodeTide’s symbolic code understanding to:
+**AgentTide** leverages CodeTide's symbolic code understanding to:
 - Retrieve and reason about relevant code context for any request
 - Generate atomic, high-precision patches using strict protocols
 - Apply changes directly to your codebase, with robust validation
@@ -567,14 +565,15 @@ CodeTide now includes an `agents` module, featuring **AgentTide**—a precision-
 - Source: [`codetide/agents/tide/agent.py`](codetide/agents/tide/agent.py)
 
 ### What It Does
-AgentTide acts as an autonomous agent that:
-- Connects to your codebase using CodeTide’s parsing and context tools
-- Interacts with users via a conversational interface
-- Identifies relevant files, classes, and functions for any request
-- Generates and applies diff-style patches, ensuring code quality and requirements fidelity
+AgentTide is an autonomous, precision-driven software engineering agent that:
+- Connects to your codebase using CodeTide's parsing and context tools
+- Interacts with users via a conversational interface (CLI or UI)
+- Identifies relevant files, classes, and functions for any request using advanced identifier resolution and code search
+- Generates and applies atomic, diff-style patches using a strict protocol, ensuring code quality and requirements fidelity
+- Supports stepwise planning, patch review, and human-in-the-loop approval for every change
 
 ### Example Usage
-To use AgentTide, ensure you have the `aicore` package installed (`pip install codetide[agents]`), then instantiate and run the agent:
+To use AgentTide programmatically, ensure you have the `aicore` package installed (`pip install codetide[agents]`), then instantiate and run the agent:
 
 ```python
 from codetide import CodeTide
@@ -599,10 +598,10 @@ if __name__ == "__main__":
     asyncio.run(main())
 ```
 
-AgentTide will prompt you for requests, retrieve the relevant code context, and generate precise patches to fulfill your requirements.
+AgentTide will prompt you for requests, retrieve the relevant code context, and generate precise, atomic patches to fulfill your requirements. All changes are patch-based and require explicit approval before being applied.
 
-**Disclaimer:**
-AgentTide is designed for focused, context-aware code editing, not for generating entire applications from vague ideas. While CodeTide as a platform can support larger workflows, the current version of AgentTide is optimized for making precise, well-scoped changes. For best results, provide one clear request at a time. AgentTide does not yet have access to your terminal or the ability to execute commands, but support for test-based validation is planned in future updates.
+**Note:**  
+AgentTide is designed for focused, context-aware code editing, not for generating entire applications from vague ideas. For best results, provide one clear request at a time. AgentTide does not execute code or shell commands, but support for test-based validation is planned in future updates.
 
 For more details, see the [agents module source code](codetide/agents/tide/agent.py).
 

codetide/__init__.py

@@ -14,6 +14,7 @@
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 from typing import Optional, List, Tuple, Union, Dict
 from datetime import datetime, timezone
+from collections import defaultdict
 from pathlib import Path
 import traceback
 import asyncio
@@ -97,10 +98,27 @@ def relative_filepaths(self)->List[str]:
         return [
             str(filepath.relative_to(self.rootpath)).replace("\\", "/") for filepath in self.files
         ]
+
+    @property
+    def relative_directories(self) -> List[str]:
+        dirs = set()
+        for filepath in self.files:
+            p = filepath.resolve().parent
+            while p != self.rootpath:
+                dirs.add(p.relative_to(self.rootpath).as_posix())
+                p = p.parent
+        return sorted(dirs)
+
+    @property
+    def filenames_mapped(self)->Dict[str, str]:
+        return {
+            filepath.name: str(filepath.relative_to(self.rootpath)).replace("\\", "/")
+            for filepath in self.files
+        }
 
     @property
     def cached_ids(self)->List[str]:
-        return self.codebase.non_import_unique_ids+self.relative_filepaths
+        return self.codebase.non_import_unique_ids + self.relative_filepaths + self.relative_directories
 
     @property
     def repo(self)->Optional[pygit2.Repository]:
@@ -412,6 +430,7 @@ def _get_changed_files(self) -> Tuple[List[Path], bool]:
         """
         file_deletion_detected = False
         files = self._find_code_files()  # Dict[Path, datetime]
+        print("found code files")
 
         changed_files = []
 
@@ -528,6 +547,97 @@ def _is_file_content_valid(filepath :Path)->bool:
 
         return True
 
+    @staticmethod
+    def _is_subdirectory(identifier: str) -> bool:
+        """
+        Check if an identifier represents a module/subdirectory.
+
+        Args:
+            identifier: A string or Path object to check
+
+        Returns:
+            True if the identifier ends with '/' (indicating a module), False otherwise
+        """
+        if isinstance(identifier, Path):
+            return False
+        elif identifier.endswith("/"):
+            return True
+        else:
+            return False
+
+    def get_module_identifiers(self, module_ids: List[str]) -> Dict[str, List[str]]:
+        """
+        Get all file identifiers that belong to specified modules.
+
+        Args:
+            module_ids: List of module identifier strings (directories)
+
+        Returns:
+            Dictionary mapping module names to lists of relative file paths within each module
+        """
+        module_paths = {
+            self.rootpath / module_id
+            for module_id in module_ids
+        }
+        modules_identifiers = defaultdict(list)
+        for filepath in self.files:
+            for module_path in module_paths:
+                if filepath.is_relative_to(module_path):
+                    modules_identifiers[module_path.name].append(
+                        str(filepath.relative_to(self.rootpath))
+                    )
+                    break
+
+        # Log the results
+        logger.info(f"Found {len(modules_identifiers)} modules")
+        for module_name, identifiers in modules_identifiers.items():
+            logger.info(f"Module '{module_name}' contains {len(identifiers)} identifiers")
+
+        return modules_identifiers
+
+    def inject_identifiers_from_modules(self, unique_ids: List[str]) -> List[str]:
+        """
+        Expand module identifiers into their constituent file identifiers.
+
+        Takes a list of identifiers that may include module directories, finds all files
+        within those modules, and replaces the module identifiers with individual file paths.
+
+        Args:
+            unique_ids: List of identifiers, may include both files and modules (ending with '/')
+
+        Returns:
+            Expanded list with module identifiers replaced by their constituent file identifiers
+        """
+        modules_identifiers = [
+            unique_id for unique_id in unique_ids if self._is_subdirectory(unique_id)
+        ]
+        identifiers_per_module = self.get_module_identifiers(module_ids=modules_identifiers)
+
+        unique_ids = [
+            unique_id for unique_id in unique_ids
+            if unique_id not in modules_identifiers
+        ]
+        for identifiers in identifiers_per_module.values():
+            unique_ids.extend(identifiers)
+
+        return unique_ids
+
+    def precheck(self, unique_ids: List[str]) -> Dict[Path, str]:
+        """
+        Preprocess and validate identifiers before further operations.
+
+        Expands any module identifiers into their constituent files and validates
+        that all identifiers correspond to actual files.
+
+        Args:
+            unique_ids: List of file or module identifiers to precheck
+
+        Returns:
+            Dictionary mapping validated file paths to their identifier strings
+        """
+        unique_ids = self.inject_identifiers_from_modules(unique_ids)
+        return self._precheck_id_is_file(unique_ids)
+
     def _precheck_id_is_file(self, unique_ids : List[str])->Dict[Path, str]:
         """
         Preload file contents for the given IDs if they correspond to known files.
@@ -580,7 +690,7 @@ def get(
             f"Formats: string={as_string}, list={as_string_list}"
         )
 
-        requested_files = self._precheck_id_is_file(code_identifiers)
+        requested_files = self.precheck(code_identifiers)
         return self.codebase.get(
             unique_id=code_identifiers,
             degree=context_depth,
@@ -600,8 +710,6 @@ def _as_file_paths(self, code_identifiers: Union[str, List[str]])->List[str]:
                 as_file_paths.append(code_identifier)
             elif element := self.codebase.cached_elements.get(code_identifier):
                 as_file_paths.append(element.file_path)
-            else: ### covers new files
-                as_file_paths.append(element)
 
         return as_file_paths
 
@@ -615,8 +723,11 @@ def get_unique_paths(path_list):
         unique_paths = []
 
         for path in path_list:
-            # Normalize the path to use OS-appropriate separators
-            normalized = os.path.normpath(path)
+            if isinstance(path, str) and path.endswith("/"):
+                normalized = path
+            else:
+                # Normalize the path to use OS-appropriate separators
+                normalized = os.path.normpath(path)
 
             # Only add if we haven't seen this normalized path before
             if normalized not in seen: