Modal Vibe: A scalable AI coding platform

The Modal Vibe repo demonstrates how you can build a scalable AI coding platform on Modal.

Users of the application can prompt an LLM to create sandboxed applications that service React through a UI.

Each application lives on a Modal Sandbox and contains a webserver accessible through Modal Tunnels.

For a high-level overview of Modal Vibe, including performance numbers and why they matter, see the accompanying blog post. For details on the implementation, read on.

How it's structured

• main.py is the entrypoint that runs the FastAPI controller that serves the web app and manages the sandbox apps.
• core contains the logic for SandboxApp model and LLM logic.
• sandbox contains a small HTTP server that gets put inside every Sandbox that's created, as well as some sandbox lifecycle management code.
• web contains the Modal Vibe website that users see and interact with, as well as the api server that manages Sandboxes.

How to run

First, set up the local environment:

python3 -m venv venv && source venv/bin/activate && pip install -r requirements.dev.txt

Deploy

To deploy to Modal, copy .env.example to a file called .env and add your ANTHROPIC_API_KEY. Also, create a Modal Secret called anthropic-secret so our applications can access it.

Then, deploy the application with Modal:

modal deploy -m main

Local Development

Run a load test:

modal run main.py::create_app_loadtest_function --num-apps 10

Delete a sandbox:

modal run main.py::delete_sandbox_admin_function --app-id <APP_ID>

Run an example sandbox HTTP server:

python -m sandbox.server