Merge remote-tracking branch 'origin/main' into kvbm-big-offload-pr

Author: da-x

.coderabbit.yaml

@@ -19,6 +19,9 @@ reviews:
     auto_incremental_review: false
   suggested_labels: false
   suggested_reviewers: false
+  sequence_diagrams: false
+  related_issues: false
+  related_prs: false
   finishing_touches:
     docstrings:
       enabled: false

.devcontainer/README.md

@@ -211,7 +211,7 @@ This setup allows you to use Git commands normally within the container without
 ### Step 5: Wait for Initialization
 
 The container will automatically:
-- Mount your local code to `/home/ubuntu/dynamo`
+- Mount your local code to `/workspace`
 - Run `post-create.sh` to build the project and configure the environment
 
 If `post-create.sh` fails, you can try to debug or [submit](https://github.com/ai-dynamo/dynamo/issues) an issue on GitHub.
@@ -223,13 +223,13 @@ If `post-create.sh` fails, you can try to debug or [submit](https://github.com/a
 If you make changes to Rust code and want to compile, use [cargo build](https://doc.rust-lang.org/cargo/commands/cargo-build.html). This will update Rust binaries such as dynamo-run.
 
 ```bash
-cd /home/ubuntu/dynamo && cargo build --locked --profile dev
+cd /workspace && cargo build --locked --profile dev
 ```
 
-Verify that builds are in the pre-defined `dynamo/.build/target` and not `dynamo/workspace`:
+Verify that builds are in the pre-defined `target` directory:
 ```bash
 $ cargo metadata --format-version=1 | jq -r '.target_directory'
-/home/ubuntu/dynamo/.build/target  <-- this is the target path
+/workspace/target  <-- this is the target path
 ```
 
 If cargo is not installed and configured properly, you will see one or more errors, such as the following:
@@ -244,22 +244,22 @@ Lastly, before pushing code to GitHub, remember to run `cargo fmt` and `cargo cl
 If you make changes to Rust code and want to propagate to Python bindings then can use [maturin](https://www.maturin.rs/#usage) (pre-installed). This will update the Python bindings with your new Rust changes.
 
 ```bash
-cd /home/ubuntu/dynamo/lib/bindings/python && maturin develop
+cd /workspace/lib/bindings/python && maturin develop
 ```
 
 ## What's Inside
 Development Environment:
 - Rust and Python toolchains
 - GPU acceleration
 - VS Code or Cursor extensions for Rust and Python
-- Persistent build cache in `.build/` directory enables fast incremental builds (only changed files are recompiled) via `cargo build --locked --profile dev`
+- Persistent build cache in `target/` directory enables fast incremental builds (only changed files are recompiled) via `cargo build --locked --profile dev`
 - Edits to files are propagated to local repo due to the volume mount
 - SSH and GPG agent passthrough orchestrated by devcontainer
 
 File Structure:
-- Local dynamo repo mounts to `/home/ubuntu/dynamo`
+- Local dynamo repo mounts to `/workspace`
 - Python venv in `/opt/dynamo/venv`
-- Build artifacts in `dynamo/.build/target`
+- Build artifacts in `/workspace/target`
 - Hugging Face cache preserved between sessions (either mounting your host .cache to the container, or your `HF_HOME` to `/home/ubuntu/.cache/huggingface`)
 - Bash memory preserved between sessions at `/home/ubuntu/.commandhistory` using docker volume `dynamo-bashhistory`
 - Precommit preserved between sessions at `/home/ubuntu/.cache/precommit` using docker volume `dynamo-precommit-cache`
@@ -268,7 +268,7 @@ File Structure:
 
 To look at the docs run:
 ```bash
-cd ~/dynamo/.build/target/doc && python3 -m http.server 8000
+cd /workspace/target/doc && python3 -m http.server 8000
 ```
 
 VSCode will automatically port-forward and you can check them out in your browser.
@@ -376,7 +376,7 @@ If you encounter build errors or strange compilation issues, try running `cargo
 If `cargo clean` doesn't resolve the issue, it is possible that some of the files were created by root (using the `run.sh` script). You can manually remove the build target by going to your host (outside the container), and remove the target:
 
 ```bash
-sudo rm -rf <your dynamo path on the host machine>/.build/target
+sudo rm -rf <your dynamo path on the host machine>/target
 ```
 
 ### Volume Corruption Issues
@@ -426,7 +426,8 @@ If you see errors like "container is not running" or "An error occurred setting
    # If missing, build the dev image first, then build local-dev
    export FRAMEWORK=VLLM  # Replace with VLLM, SGLANG, or TRTLLM
    ./container/build.sh --framework $FRAMEWORK
-   ./container/build.sh --dev-image dynamo:latest-${FRAMEWORK,,} --framework $FRAMEWORK
+   # change to lower case portable way across shells
+   ./container/build.sh --dev-image dynamo:latest-$(echo "$FRAMEWORK" | tr '[:upper:]' '[:lower:]') --framework "$FRAMEWORK"
    # Now you have dynamo:latest-vllm-local-dev
    ```
 

.devcontainer/devcontainer.json.j2

@@ -29,7 +29,7 @@
                 // Disable automatic copying of .gitconfig to avoid errors
                 "dev.containers.copyGitConfig": false,
                 "terminal.integrated.defaultProfile.linux": "bash",
-                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+                "terminal.integrated.cwd": "/workspace",
 
                 "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
                 "python.linting.enabled": true,
@@ -38,7 +38,7 @@
                 "rust-analyzer.checkOnSave.command": "clippy",
                 "rust-analyzer.checkOnSave.enable": true,
                 "rust-analyzer.cargo.buildScripts.enable": true,
-                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.cargo.targetDir": "/workspace/target",
                 "rust-analyzer.procMacro.enable": true,
                 "rust-analyzer.completion.autoimport.enable": true,
 
@@ -56,16 +56,10 @@
             }
         }
     },
-    "workspaceFolder": "/home/ubuntu/dynamo",
-    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "workspaceFolder": "/workspace",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=cached",
     "userEnvProbe": "interactiveShell",
-    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
-    "containerEnv": {
-        "DYNAMO_HOME": "/home/ubuntu/dynamo",
-        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
-        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
-        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
-    },
+    "postCreateCommand": "/bin/bash /workspace/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
     "remoteEnv": {
         // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
         "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",

.devcontainer/post-create.sh

@@ -4,8 +4,9 @@
 
 set -eu
 
-# Set DYNAMO_HOME if not already set
-export DYNAMO_HOME=${DYNAMO_HOME:-/home/ubuntu/dynamo}
+# Use WORKSPACE_DIR if set (should be set by Dockerfile.local_dev)
+# Otherwise, use /workspace as fallback
+WORKSPACE_DIR="${WORKSPACE_DIR:-/workspace}"
 
 # Ensure we're not running as root
 if [ "$(id -u)" -eq 0 ]; then
@@ -53,11 +54,12 @@ show_and_run() {
 set -x
 
 # Pre-commit hooks
-cd $DYNAMO_HOME && pre-commit install && retry pre-commit install-hooks
+cd $WORKSPACE_DIR && pre-commit install && retry pre-commit install-hooks
 pre-commit run --all-files || true # don't fail the build if pre-commit hooks fail
 
-# Set build directory
-export CARGO_TARGET_DIR=${CARGO_TARGET_DIR:-$DYNAMO_HOME/.build/target}
+# Use CARGO_TARGET_DIR from environment (should be set by Dockerfile.local_dev)
+# Fallback to $WORKSPACE_DIR/target if not set
+export CARGO_TARGET_DIR=${CARGO_TARGET_DIR:-$WORKSPACE_DIR/target}
 mkdir -p $CARGO_TARGET_DIR
 
 uv pip uninstall --yes ai-dynamo ai-dynamo-runtime 2>/dev/null || true
@@ -78,27 +80,16 @@ cargo build --locked --profile dev --features dynamo-llm/block-manager
 # fi
 
 # install ai-dynamo-runtime
-(cd $DYNAMO_HOME/lib/bindings/python && retry maturin develop)
+(cd $WORKSPACE_DIR/lib/bindings/python && retry maturin develop)
 
 # install ai-dynamo
-cd $DYNAMO_HOME && retry env DYNAMO_BIN_PATH=$CARGO_TARGET_DIR/debug uv pip install -e .
+cd $WORKSPACE_DIR && retry env DYNAMO_BIN_PATH=$CARGO_TARGET_DIR/debug uv pip install -e .
 
 { set +x; } 2>/dev/null
 
 echo -e "\n" >> ~/.bashrc
 echo "# === This section is generated by the post-create.sh script ===" >> ~/.bashrc
 
-# Check if PYTHONPATH is already set in environment
-if [ -n "${PYTHONPATH:-}" ]; then
-    # PYTHONPATH exists, replace /workspace with $DYNAMO_HOME
-    export PYTHONPATH=$(echo "$PYTHONPATH" | sed "s|/workspace|$DYNAMO_HOME|g")
-    # Also add to .bashrc for persistence
-    if ! grep -q "export PYTHONPATH=" ~/.bashrc; then
-        echo "# PYTHONPATH modified from /workspace to use DYNAMO_HOME" >> ~/.bashrc
-        show_and_run echo "export PYTHONPATH=\"$PYTHONPATH\"" >> ~/.bashrc
-    fi
-fi
-
 if ! grep -q "export GPG_TTY=" ~/.bashrc; then
     show_and_run echo 'export GPG_TTY=$(tty)' >> ~/.bashrc
 fi
@@ -123,7 +114,7 @@ else
     echo "⚠️ SSH agent forwarding not configured - SSH_AUTH_SOCK is not set"
 fi
 
-show_and_run $DYNAMO_HOME/deploy/sanity_check.py
+show_and_run $WORKSPACE_DIR/deploy/sanity_check.py
 
 cat <<EOF
 

.devcontainer/sglang/devcontainer.json

@@ -32,7 +32,7 @@
                 // Disable automatic copying of .gitconfig to avoid errors
                 "dev.containers.copyGitConfig": false,
                 "terminal.integrated.defaultProfile.linux": "bash",
-                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+                "terminal.integrated.cwd": "/workspace",
 
                 "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
                 "python.linting.enabled": true,
@@ -41,7 +41,7 @@
                 "rust-analyzer.checkOnSave.command": "clippy",
                 "rust-analyzer.checkOnSave.enable": true,
                 "rust-analyzer.cargo.buildScripts.enable": true,
-                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.cargo.targetDir": "/workspace/target",
                 "rust-analyzer.procMacro.enable": true,
                 "rust-analyzer.completion.autoimport.enable": true,
 
@@ -59,16 +59,10 @@
             }
         }
     },
-    "workspaceFolder": "/home/ubuntu/dynamo",
-    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "workspaceFolder": "/workspace",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=cached",
     "userEnvProbe": "interactiveShell",
-    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
-    "containerEnv": {
-        "DYNAMO_HOME": "/home/ubuntu/dynamo",
-        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
-        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
-        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
-    },
+    "postCreateCommand": "/bin/bash /workspace/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
     "remoteEnv": {
         // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
         "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",

.devcontainer/trtllm/devcontainer.json

@@ -32,7 +32,7 @@
                 // Disable automatic copying of .gitconfig to avoid errors
                 "dev.containers.copyGitConfig": false,
                 "terminal.integrated.defaultProfile.linux": "bash",
-                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+                "terminal.integrated.cwd": "/workspace",
 
                 "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
                 "python.linting.enabled": true,
@@ -41,7 +41,7 @@
                 "rust-analyzer.checkOnSave.command": "clippy",
                 "rust-analyzer.checkOnSave.enable": true,
                 "rust-analyzer.cargo.buildScripts.enable": true,
-                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.cargo.targetDir": "/workspace/target",
                 "rust-analyzer.procMacro.enable": true,
                 "rust-analyzer.completion.autoimport.enable": true,
 
@@ -59,16 +59,10 @@
             }
         }
     },
-    "workspaceFolder": "/home/ubuntu/dynamo",
-    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "workspaceFolder": "/workspace",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=cached",
     "userEnvProbe": "interactiveShell",
-    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
-    "containerEnv": {
-        "DYNAMO_HOME": "/home/ubuntu/dynamo",
-        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
-        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
-        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
-    },
+    "postCreateCommand": "/bin/bash /workspace/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
     "remoteEnv": {
         // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
         "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",

.devcontainer/vllm/devcontainer.json

@@ -32,7 +32,7 @@
                 // Disable automatic copying of .gitconfig to avoid errors
                 "dev.containers.copyGitConfig": false,
                 "terminal.integrated.defaultProfile.linux": "bash",
-                "terminal.integrated.cwd": "/home/ubuntu/dynamo",
+                "terminal.integrated.cwd": "/workspace",
 
                 "python.defaultInterpreterPath": "/opt/dynamo/venv/bin/python",
                 "python.linting.enabled": true,
@@ -41,7 +41,7 @@
                 "rust-analyzer.checkOnSave.command": "clippy",
                 "rust-analyzer.checkOnSave.enable": true,
                 "rust-analyzer.cargo.buildScripts.enable": true,
-                "rust-analyzer.cargo.targetDir": "/home/ubuntu/dynamo/.build/target",
+                "rust-analyzer.cargo.targetDir": "/workspace/target",
                 "rust-analyzer.procMacro.enable": true,
                 "rust-analyzer.completion.autoimport.enable": true,
 
@@ -59,16 +59,10 @@
             }
         }
     },
-    "workspaceFolder": "/home/ubuntu/dynamo",
-    "workspaceMount": "source=${localWorkspaceFolder},target=/home/ubuntu/dynamo,type=bind,consistency=cached",
+    "workspaceFolder": "/workspace",
+    "workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=cached",
     "userEnvProbe": "interactiveShell",
-    "postCreateCommand": "/bin/bash /home/ubuntu/dynamo/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
-    "containerEnv": {
-        "DYNAMO_HOME": "/home/ubuntu/dynamo",
-        "CARGO_HOME": "/home/ubuntu/dynamo/.build/.cargo",
-        "RUSTUP_HOME": "/home/ubuntu/dynamo/.build/.rustup",
-        "CARGO_TARGET_DIR": "/home/ubuntu/dynamo/.build/target"
-    },
+    "postCreateCommand": "/bin/bash /workspace/.devcontainer/post-create.sh", // Runs cargo build and pip installs packages
     "remoteEnv": {
         // Optional convenience tokens passed from host. SSH agent is forwarded via your IDE setting, not here by default.
         "GITHUB_TOKEN": "${localEnv:GITHUB_TOKEN}",