Skip to content

feat(server): support multi ingress gateway mode #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Pangjiping wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat(server): support multi ingress gateway mode #161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ddaf379
feat(server): support multi ingress gateway mode
Pangjiping Feb 3, 2026
b032495
feat(server): rename `tunnel` mode to `direct` mode
Pangjiping Feb 3, 2026
ff15bda
feat(server): ingress mode for kubernetes service
Pangjiping Feb 3, 2026
69d369d
chore(server): raise exception when header mode configured
Pangjiping Feb 4, 2026
d8baa48
Merge remote-tracking branch 'origin/main' into feat/server/ingress-g…
Pangjiping Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ The pluggable architecture allows implementing custom runtimes by:

**Features:**
- Dynamic endpoint generation based on sandbox ID and port
- Supports both domain-based and wildcard-domain routing
- Supports both domain-based and wildcard routing
- Reverse proxy to sandbox container ports
- Automatic cleanup when sandbox terminates

Expand Down
15 changes: 15 additions & 0 deletions server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,21 @@ cp example.batchsandbox-template.yaml ~/batchsandbox-template.yaml
```
Further reading on Docker container security: https://docs.docker.com/engine/security/

**Ingress exposure (direct | gateway)**
```toml
[ingress]
mode = "direct" # docker runtime only supports direct
# gateway.address = "*.example.com" # host only (domain or IP[:port]); scheme is not allowed
# gateway.route.mode = "wildcard" # wildcard | uri (header not yet supported)
```
- `mode=direct`: default; required when `runtime.type=docker` (client ↔ sandbox direct reachability, no L7 gateway).
- `mode=gateway`: configure external ingress.
- `gateway.address`: wildcard domain required when `gateway.route.mode=wildcard`; otherwise must be domain, IP, or IP:port. Do not include scheme; clients decide http/https.
- `gateway.route.mode`: `wildcard` (host-based wildcard), `uri` (path-prefix). `header` is not yet supported.
- Response format examples:
- `wildcard`: `<sandbox-id>-<port>.example.com/path/to/request`
- `uri`: `10.0.0.1:8000/<sandbox-id>/<port>/path/to/request`

### (Optional) Egress sidecar for `networkPolicy`

- Configure the sidecar image (used only when requests include `networkPolicy`):
Expand Down
16 changes: 16 additions & 0 deletions server/README_zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,22 @@ seccomp_profile = "" # 配置文件路径或名称；为空使用 Docker
```
更多 Docker 安全参考：https://docs.docker.com/engine/security/

**Ingress 暴露（direct | gateway）**
```toml
[ingress]
mode = "direct" # Docker 运行时仅支持 direct（直连，无 L7 网关）
# gateway.address = "*.example.com" # 仅主机（域名/IP 或 IP:port），不允许带 scheme
# gateway.route.mode = "wildcard" # wildcard | uri（header 暂未支持）
```
- `mode=direct`：默认；当 `runtime.type=docker` 时必须使用（客户端与 sandbox 直连，不经过网关）。
- `mode=gateway`：配置外部入口。
- `gateway.address`：当 `gateway.route.mode=wildcard` 时必须是泛域名；其他模式需为域名/IP 或 IP:port。不允许携带 scheme，客户端自行选择 http/https。
- `gateway.route.mode`：`wildcard`（域名泛匹配）、`uri`（基于路径前缀）；`header` 模式暂未支持。
- 返回示例：
- `wildcard`：`<sandbox-id>-<port>.example.com/path/to/request`
- `header`：`10.0.0.1:8000/path/to/request`，请求头 `OPEN-SANDBOX-INGRESS: <sandbox-id>-<port>`
- `uri`：`10.0.0.1:8000/<sandbox-id>/<port>/path/to/request`

### （可选）Egress sidecar 配置与使用

- 配置镜像（仅在请求携带 `networkPolicy` 时注入）：
Expand Down
4 changes: 4 additions & 0 deletions server/example.config.k8s.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,7 @@ workload_provider = "batchsandbox"
# Path to the BatchSandbox template file
# Replace with your path
batchsandbox_template_file = "~/batchsandbox-template.yaml"

[ingress]
# Ingress exposure mode: direct (default) or gateway
mode = "direct"
4 changes: 4 additions & 0 deletions server/example.config.k8s.zh.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,7 @@ workload_provider = "batchsandbox"
# Path to the BatchSandbox template file
# Replace with your path
batchsandbox_template_file = "~/batchsandbox-template.yaml"

[ingress]
# Ingress exposure mode: direct (default) or gateway
mode = "direct"
10 changes: 5 additions & 5 deletions server/example.config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,8 @@
# -----------------------------------------------------------------
host = "127.0.0.1"
port = 8080

log_level = "INFO"

# Shared API key for the OPEN-SANDBOX-API-KEY header (leave empty only for dev)
api_key = ""

# api_key = "your-secret-api-key" # Optional: Uncomment to enable API key authentication

[runtime]
# Runtime selection (docker | kubernetes)
Expand All @@ -49,3 +45,7 @@ apparmor_profile = ""
pids_limit = 512
# Seccomp profile: empty string uses Docker default; set to an absolute path for a custom profile
seccomp_profile = ""

[ingress]
# Ingress exposure mode: direct (default) or gateway
mode = "direct"
10 changes: 5 additions & 5 deletions server/example.config.zh.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,8 @@
# -----------------------------------------------------------------
host = "127.0.0.1"
port = 8080

log_level = "INFO"

# Shared API key for the OPEN-SANDBOX-API-KEY header (leave empty only for dev)
api_key = ""

# api_key = "your-secret-api-key" # Optional: Uncomment to enable API key authentication

[runtime]
# Runtime selection (docker | kubernetes)
Expand All @@ -49,3 +45,7 @@ apparmor_profile = ""
pids_limit = 512
# Seccomp profile: empty string uses Docker default; set to an absolute path for a custom profile
seccomp_profile = ""

[ingress]
# Ingress exposure mode: direct (default) or gateway
mode = "direct"
134 changes: 117 additions & 17 deletions server/src/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@

from __future__ import annotations

import ipaddress
import logging
import os
import re
from pathlib import Path
from typing import Any, Literal, Optional

Expand All @@ -38,31 +40,123 @@
CONFIG_ENV_VAR = "SANDBOX_CONFIG_PATH"
DEFAULT_CONFIG_PATH = Path.home() / ".sandbox.toml"

_DOMAIN_RE = re.compile(r"^(?=.{1,253}$)(?!-)[A-Za-z0-9-]{1,63}(?:\.[A-Za-z0-9-]{1,63})+$")
_WILDCARD_DOMAIN_RE = re.compile(r"^\*\.(?!-)[A-Za-z0-9-]{1,63}(?:\.[A-Za-z0-9-]{1,63})+$")
_IPV4_WITH_PORT_RE = re.compile(r"^(?P<ip>(?:\d{1,3}\.){3}\d{1,3})(?::(?P<port>\d{1,5}))?$")

class RouterConfig(BaseModel):
"""Configuration for external sandbox router endpoints."""
INGRESS_MODE_DIRECT = "direct"
INGRESS_MODE_GATEWAY = "gateway"
GATEWAY_ROUTE_MODE_WILDCARD = "wildcard"
GATEWAY_ROUTE_MODE_HEADER = "header"
GATEWAY_ROUTE_MODE_URI = "uri"

domain: Optional[str] = Field(
default=None,
description="Base domain used to expose sandbox endpoints (e.g., 'opensandbox.io').",

def _is_valid_ip(host: str) -> bool:
try:
ipaddress.ip_address(host)
return True
except ValueError:
return False


def _is_valid_ip_or_ip_port(address: str) -> bool:
match = _IPV4_WITH_PORT_RE.match(address)
if not match:
return False
ip_str = match.group("ip")
if not _is_valid_ip(ip_str):
return False
port_str = match.group("port")
if port_str is None:
return True
try:
port = int(port_str)
except ValueError:
return False
return 1 <= port <= 65535


def _is_valid_domain(host: str) -> bool:
return bool(_DOMAIN_RE.match(host))


def _is_wildcard_domain(host: str) -> bool:
return bool(_WILDCARD_DOMAIN_RE.match(host))


class GatewayRouteModeConfig(BaseModel):
"""Routing strategy for gateway ingress exposure."""

mode: Literal[
GATEWAY_ROUTE_MODE_WILDCARD,
GATEWAY_ROUTE_MODE_HEADER,
GATEWAY_ROUTE_MODE_URI,
] = Field(
...,
description="Routing mode used by the gateway (wildcard, header, uri).",
)

class Config:
populate_by_name = True


class GatewayConfig(BaseModel):
"""Gateway mode configuration for ingress exposure."""

address: str = Field(
...,
description="Gateway host used to expose sandboxes (domain or IP, may include :port; scheme is not allowed).",
min_length=1,
)
wildcard_domain: Optional[str] = Field(
route: GatewayRouteModeConfig = Field(
...,
description="Routing mode configuration used by the gateway.",
)


class IngressConfig(BaseModel):
"""Configuration for exposing sandbox ingress."""

mode: Literal[INGRESS_MODE_DIRECT, INGRESS_MODE_GATEWAY] = Field(
default=INGRESS_MODE_DIRECT,
description="Ingress exposure mode (direct or gateway).",
)
gateway: Optional[GatewayConfig] = Field(
default=None,
alias="wildcard-domain",
description="Wildcard domain pattern (e.g., '*.opensandbox.io') used for sandbox endpoints.",
min_length=1,
description="Gateway configuration required when mode = 'gateway'.",
)

@model_validator(mode="after")
def validate_domain_choice(self) -> "RouterConfig":
if bool(self.domain) == bool(self.wildcard_domain):
raise ValueError("Exactly one of domain or wildcard-domain must be specified in [router].")
def validate_ingress_mode(self) -> "IngressConfig":
if self.mode == INGRESS_MODE_GATEWAY and self.gateway is None:
raise ValueError("gateway block must be provided when ingress.mode = 'gateway'.")
if self.mode == INGRESS_MODE_DIRECT and self.gateway is not None:
raise ValueError("gateway block must be omitted unless ingress.mode = 'gateway'.")

if self.mode == INGRESS_MODE_GATEWAY and self.gateway:
route_mode = self.gateway.route.mode
address_raw = self.gateway.address
hostport = address_raw
if "://" in address_raw:
raise ValueError("ingress.gateway.address must not include a scheme; clients choose http/https.")

if route_mode == GATEWAY_ROUTE_MODE_WILDCARD:
if not _is_wildcard_domain(hostport):
raise ValueError(
"ingress.gateway.address must be a wildcard domain (e.g., *.example.com) "
"when gateway.route.mode is wildcard."
)
else:
if "*" in hostport:
raise ValueError(
"ingress.gateway.address must not contain wildcard when gateway.route.mode is not wildcard."
)
if not (_is_valid_domain(hostport) or _is_valid_ip_or_ip_port(hostport)):
raise ValueError(
"ingress.gateway.address must be a valid domain, IP, or IP:port when gateway.route.mode is not wildcard."
)
return self

class Config:
populate_by_name = True


class ServerConfig(BaseModel):
"""FastAPI server configuration."""
Expand Down Expand Up @@ -203,7 +297,7 @@ class AppConfig(BaseModel):
runtime: RuntimeConfig = Field(..., description="Sandbox runtime configuration.")
kubernetes: Optional[KubernetesRuntimeConfig] = None
agent_sandbox: Optional["AgentSandboxRuntimeConfig"] = None
router: Optional[RouterConfig] = None
ingress: Optional[IngressConfig] = None
docker: DockerConfig = Field(default_factory=DockerConfig)

@model_validator(mode="after")
Expand All @@ -213,6 +307,8 @@ def validate_runtime_blocks(self) -> "AppConfig":
raise ValueError("Kubernetes block must be omitted when runtime.type = 'docker'.")
if self.agent_sandbox is not None:
raise ValueError("agent_sandbox block must be omitted when runtime.type = 'docker'.")
if self.ingress is not None and self.ingress.mode != INGRESS_MODE_DIRECT:
raise ValueError("ingress.mode must be 'direct' when runtime.type = 'docker'.")
elif self.runtime.type == "kubernetes":
if self.kubernetes is None:
self.kubernetes = KubernetesRuntimeConfig()
Expand Down Expand Up @@ -314,7 +410,11 @@ def get_config_path() -> Path:
"AppConfig",
"ServerConfig",
"RuntimeConfig",
"RouterConfig",
"IngressConfig",
"GatewayConfig",
"GatewayRouteModeConfig",
"INGRESS_MODE_DIRECT",
"INGRESS_MODE_GATEWAY",
"DockerConfig",
"KubernetesRuntimeConfig",
"DEFAULT_CONFIG_PATH",
Expand Down
46 changes: 46 additions & 0 deletions server/src/services/helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,13 @@
from typing import Dict, Optional

from src.api.schema import Sandbox, SandboxFilter
from src.config import (
GATEWAY_ROUTE_MODE_HEADER,
GATEWAY_ROUTE_MODE_URI,
GATEWAY_ROUTE_MODE_WILDCARD,
INGRESS_MODE_GATEWAY,
IngressConfig,
)

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -141,9 +148,48 @@ def matches_filter(sandbox: Sandbox, filter_: SandboxFilter) -> bool:
return True


# ============================================================================
# Ingress helpers
# ============================================================================
def format_ingress_endpoint(
ingress_config: Optional[IngressConfig],
sandbox_id: str,
port: int,
) -> Optional[str]:
"""
Build an ingress-based endpoint string for a sandbox.

Returns None when ingress is not in gateway mode or when the route mode is
not supported (e.g., header mode is intentionally skipped until Endpoint
schema can carry headers).
"""
if not ingress_config or ingress_config.mode != INGRESS_MODE_GATEWAY:
return None
gateway_cfg = ingress_config.gateway
if gateway_cfg is None:
return None

address = gateway_cfg.address
route_mode = gateway_cfg.route.mode

if route_mode == GATEWAY_ROUTE_MODE_WILDCARD:
base = address[2:] if address.startswith("*.") else address
return f"{sandbox_id}-{port}.{base}"

if route_mode == GATEWAY_ROUTE_MODE_URI:
return f"{address}/{sandbox_id}/{port}"

if route_mode == GATEWAY_ROUTE_MODE_HEADER:
# TODO(Pangjiping): Header mode intentionally not emitted until Endpoint schema supports headers.
raise RuntimeError(f"Unsupported route mode: {route_mode}")

return None


__all__ = [
"parse_memory_limit",
"parse_nano_cpus",
"parse_timestamp",
"format_ingress_endpoint",
"matches_filter",
]
Loading
Loading