Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
2c2de7f
Add Pi bridge integration
mmprotest Jun 28, 2026
803fefa
Fix Pi integration scaffold
mmprotest Jun 28, 2026
f40c655
Merge pull request #393 from mmprotest/mmprotest/fix-pi-integration-s…
mmprotest Jun 28, 2026
f01f842
Harden Pi bridge and runtime integration
mmprotest Jun 28, 2026
598571f
Merge pull request #394 from mmprotest/mmprotest/harden-pi-integratio…
mmprotest Jun 28, 2026
e734d66
Fix Pi integration release blockers
mmprotest Jun 28, 2026
a7df7f8
Merge pull request #395 from mmprotest/mmprotest/fix-pi-integration-r…
mmprotest Jun 28, 2026
603d6c9
Minor changes
Jun 28, 2026
add9d9c
Changes
Jun 28, 2026
9f8db58
More changes
Jun 28, 2026
9f159d2
Changes
Jun 28, 2026
2be6098
Changes
Jun 28, 2026
56c68dc
Prepare Pi Villani release v0.1.3
Jun 28, 2026
b86ac29
Rebuild Pi Villani extension adapter
mmprotest Jun 28, 2026
13b3ce7
Merge pull request #396 from mmprotest/mmprotest/rebuild-node-pi-exte…
mmprotest Jun 28, 2026
f73e46a
Fix Pi model proxy completion path
mmprotest Jun 28, 2026
f1f2580
Merge pull request #397 from mmprotest/mmprotest/fix-pi-model-proxy-r…
mmprotest Jun 28, 2026
e3b29c1
Fix Pi Villani proxy and bridge waits
mmprotest Jun 28, 2026
0d723cc
Merge pull request #398 from mmprotest/mmprotest/fix-pi-integration-t…
mmprotest Jun 28, 2026
07e031c
Restore Pi proxy context conversion
mmprotest Jun 28, 2026
c84f1be
Merge pull request #399 from mmprotest/mmprotest/replace-modelproxy-m…
mmprotest Jun 28, 2026
090d19b
Fix Pi lifecycle timeout cleanup
mmprotest Jun 28, 2026
4e31849
Merge pull request #400 from mmprotest/mmprotest/fix-pi-extension-lif…
mmprotest Jun 28, 2026
861623a
Fix Pi Villani proxy and bridge diagnostics
mmprotest Jun 28, 2026
d530082
Merge pull request #401 from mmprotest/mmprotest/fix-pi-villani-integ…
mmprotest Jun 28, 2026
afc58ff
Fix Pi bridge stdio run acknowledgement
mmprotest Jun 28, 2026
86ace74
Merge pull request #402 from mmprotest/mmprotest/fix-python-bridge-ru…
mmprotest Jun 28, 2026
4848e1a
Fix Pi bridge subprocess stdin isolation
mmprotest Jun 28, 2026
bc6cc36
Merge pull request #403 from mmprotest/mmprotest/fix-pi-bridge-run-ha…
mmprotest Jun 28, 2026
9e63fa6
Fix Pi final message shape
mmprotest Jun 28, 2026
5614673
Merge pull request #404 from mmprotest/mmprotest/fix-pi-crash-after-m…
mmprotest Jun 28, 2026
0e5f757
Restore Pi model proxy response conversion
mmprotest Jun 28, 2026
defa154
Merge pull request #405 from mmprotest/mmprotest/fix-pi-response-conv…
mmprotest Jun 28, 2026
e56406d
Harden Pi tool call arguments
mmprotest Jun 28, 2026
6c3ac10
Merge pull request #406 from mmprotest/mmprotest/fix-pi-villani-tool-…
mmprotest Jun 28, 2026
127f9b5
Fix Pi approval and tool progress events
mmprotest Jun 28, 2026
6ea1a29
Merge pull request #407 from mmprotest/mmprotest/fix-pi-approval-and-…
mmprotest Jun 28, 2026
d84629e
Fix Pi bridge tool result lifecycle
mmprotest Jun 28, 2026
827f1d8
Merge pull request #408 from mmprotest/mmprotest/fix-pi-integration-u…
mmprotest Jun 28, 2026
14810b2
Fix Pi bridge lifecycle event handling
mmprotest Jun 28, 2026
e52c2fb
Merge pull request #409 from mmprotest/mmprotest/fix-pi-villani-bridg…
mmprotest Jun 28, 2026
e2e97d9
Fix Pi Villani UI final output
mmprotest Jun 28, 2026
73065b6
Merge pull request #410 from mmprotest/mmprotest/fix-messy-ui-output-…
mmprotest Jun 28, 2026
837d6dd
Clean Pi Villani progress rendering
mmprotest Jun 28, 2026
96bf959
Merge pull request #411 from mmprotest/mmprotest/fix-pi-villani-ui-to…
mmprotest Jun 28, 2026
a82da16
Fix Pi Villani result renderer
mmprotest Jun 28, 2026
ba816a9
Merge pull request #412 from mmprotest/mmprotest/fix-pi-villani-final…
mmprotest Jun 28, 2026
035db8e
Remove Pi Villani custom renderer
mmprotest Jun 28, 2026
ceb56a6
Merge pull request #413 from mmprotest/mmprotest/fix-pi-villani-build…
mmprotest Jun 28, 2026
37995b3
Fix Pi Villani UI status copy
mmprotest Jun 29, 2026
c526533
Merge pull request #414 from mmprotest/mmprotest/fix-pi-villani-ui-co…
mmprotest Jun 29, 2026
630423a
Fix Pi Villani UI status and approval copy
mmprotest Jun 29, 2026
304a100
Merge pull request #415 from mmprotest/mmprotest/fix-pi-villani-ui-co…
mmprotest Jun 29, 2026
f968fe6
Updated Villani-isms
Jun 29, 2026
b6a39de
Fix Pi approval UI duplication
mmprotest Jun 29, 2026
3efef13
Merge pull request #416 from mmprotest/mmprotest/fix-approval-duplica…
mmprotest Jun 29, 2026
8144b96
Changes
Jun 29, 2026
e5abf25
Fix Pi release test bridge launch
mmprotest Jun 29, 2026
d76029f
Merge pull request #417 from mmprotest/mmprotest/fix-npm-test-failure…
mmprotest Jun 29, 2026
fcdf90c
Changes
Jun 29, 2026
103ca33
Release Pi Villani v0.1.5
Jun 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions .github/workflows/build-pi-villani-runtime-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Build Pi Villani Runtime Release
permissions:
contents: write
on:
workflow_dispatch:
push:
tags: ["pi-villani-runtime-v*"]
jobs:
build:
strategy:
matrix:
include:
- key: win32-x64
os: windows-latest
ext: zip
- key: darwin-arm64
os: macos-14
ext: tar.gz
- key: darwin-x64
os: macos-26
ext: tar.gz
- key: linux-x64
os: ubuntu-latest
ext: tar.gz
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: {python-version: '3.11'}
- run: python -m pip install -e . pyinstaller
- run: pyinstaller packaging/villani_runtime.spec --distpath dist-runtime/villani-code
- run: python packaging/smoke_test_runtime.py dist-runtime/villani-code/villani-code${{ runner.os == 'Windows' && '.exe' || '' }}
- shell: bash
run: |
name="villani-runtime-v0.1.5-${{ matrix.key }}.${{ matrix.ext }}"
cd dist-runtime
if [[ "${{ matrix.ext }}" == "zip" ]]; then 7z a ../$name villani-code; else tar -czf ../$name villani-code; fi
cd ..
python - "$name" > "${name}.sha256" <<'PY'
import hashlib
import pathlib
import sys

path = pathlib.Path(sys.argv[1])
digest = hashlib.sha256(path.read_bytes()).hexdigest()
print(f"{digest} {path.name}")
PY
mkdir -p smoke-asset
python - "$name" smoke-asset <<'PY'
import sys, tarfile, zipfile
from pathlib import Path
archive, dest = Path(sys.argv[1]), Path(sys.argv[2])
if archive.suffix == '.zip':
with zipfile.ZipFile(archive) as zf:
zf.extractall(dest)
else:
with tarfile.open(archive, 'r:gz') as tf:
tf.extractall(dest)
PY
python packaging/smoke_test_runtime.py smoke-asset/villani-code/villani-code${{ runner.os == 'Windows' && '.exe' || '' }}
- uses: actions/upload-artifact@v4
with:
name: runtime-${{ matrix.key }}
path: "villani-runtime-v0.1.5-${{ matrix.key }}.${{ matrix.ext }}*"
release:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v4
with:
path: artifacts
- run: find artifacts -name '*.sha256' -exec cat {} \; > checksums.txt
- uses: softprops/action-gh-release@v2
with:
files: |
artifacts/**/villani-runtime-v0.1.5-*
checksums.txt
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
# Keep the Pi runtime PyInstaller spec tracked
!packaging/villani_runtime.spec

# Node
node_modules/
npm-debug.log*
*.tgz

__pycache__/
*.pyc
.venv/
Expand Down
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,21 @@ Autonomous pass:
villani-code --villani-mode --base-url http://127.0.0.1:1234 --model your-model --repo /path/to/repo
```

## Use Villani Code in Pi

# @mmprotest/pi-villani
Install with:
```bash
pi install npm:@mmprotest/pi-villani
```
Provides `/villani <task>`.

Runtime version: `v0.1.5`.

## Pi Usage

Run Villani with `/villani <task>`. Active runs are still cleaned up automatically on process/session cancellation or error cleanup.

## Reports

- [Villani Code Terminal-Bench 2.0 Qwen3.6 27B Report](docs/Villani_Code_Terminal_Bench_2_Qwen27B_Report.pdf)
Expand Down
86 changes: 86 additions & 0 deletions integrations/pi-villani/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# @mmprotest/pi-villani
Install with:
```bash
pi install npm:@mmprotest/pi-villani
```
Provides `/villani <task>`.

Runtime version: `v0.1.5`.

## Usage

Run Villani with `/villani <task>`. Active runs are still cleaned up automatically on process/session cancellation or error cleanup.

# Villani Code

**Flagship coding-agent performance from small local models.**

Villani Code is a local-first coding-agent runtime designed to make smaller open models do real repository work: navigate files, run commands, make patches, survive verification, and keep working through messy terminal environments.

The thesis is simple: small models do not just need better weights. They need a better runtime.

## Terminal-Bench 2.0: Qwen3.6 27B full-suite run

Villani Code achieved a **196/445 lower-bound score** on the full Terminal-Bench 2.0 suite using **Qwen3.6 27B**.

That is **44.0%** across **89 tasks** with **5 attempts per task**.

### Headline result

| System | Model | Terminal-Bench 2.0 accuracy |
|---|---|---:|
| Codex CLI | GPT-5-Codex | 44.3% |
| **Villani Code** | **Qwen3.6 27B** | **44.0%** |
| Mini-SWE-Agent | GPT-5-Codex | 41.3% |
| Claude Code | Claude Sonnet 4.5 | 40.1% |
| Dakou Agent | Qwen 3 Coder 480B | 27.2% |
| little-coder | Qwen3.6-35B-A3B | 24.6% |
| Bash Agent | TermiGen-32B | 19.3% |
| little-coder | Qwen3.5-9B | 9.2% |

## Qwen3.5 9B same-model runtime comparison

Villani Code was also tested against Claude Code using the same model: **Qwen3.5 9B**.

Same model. Same tasks. Different agent runtime.

Villani Code won.


| Runner | Score | Success rate |
|---|---:|---:|
| **Villani Code + Qwen3.5 9B** | **38/60** | **63.3%** |
| Claude Code + Qwen3.5 9B | 26/60 | 43.3% |

**Villani Code delivered a 46% relative performance improvement over Claude Code.**

This comparison covers **12 overlapping Terminal-Bench tasks**, with **5 runs per task**, for **60 runs per agent**.

Villani Code won **6 tasks**, tied **6 tasks**, and lost **0**.

## What Villani Code is

Villani Code is a terminal-first coding agent for:

- bounded bug fixes
- repo navigation and localization
- command-driven debugging
- test-guided patching
- local inference setups
- privacy-sensitive codebases
- smaller open model backends

It is built for the environment where most coding agents start to fall apart: smaller models, hard verification, constrained context, terminal noise, failed commands, and real repositories.

## What changed in the upgraded runtime

The latest Villani Code upgrade includes:

- new execution loop
- better local model integration
- cleaner tool handling
- improved failure recovery
- task-scoped memory system
- better state tracking across long-running coding tasks

The benchmark comparison evaluates the upgraded runtime as a whole.
5 changes: 5 additions & 0 deletions integrations/pi-villani/docs/release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Release checklist
1. Publish matching GitHub runtime release `pi-villani-runtime-v0.1.5`.
2. Verify assets and checksums.
3. Publish `@mmprotest/pi-villani`.
Install: `pi install npm:@mmprotest/pi-villani`.
Loading
Loading