Skip to content

Latest commit

 

History

History
442 lines (345 loc) · 18.3 KB

runtime-modes.md

File metadata and controls

442 lines (345 loc) · 18.3 KB

Runtime Modes Matrix

This document is the single source of truth for Xelma Backend's runtime mode flags — the environment variables that change how endpoints behave. Refer to it when setting up a local environment, debugging unexpected endpoint behavior, or choosing the right flags for a deployment profile.

Not to be confused with app feature flags. Which routes exist on each entrypoint is a separate concern, owned by AppFeatures in src/app-factory.ts and documented in the "Feature flags" section of CONTRIBUTING.md. The env vars below change behaviour; the feature flags change surface area.

Startup tip: The server logs the active mode flags at boot. Look for Active DATA_MODE=..., Bet mode: ..., ROUNDS_MOCK_MODE=..., and Soroban money-path policy: ... in the console output. Each log line references this document: Runtime modes documented at docs/runtime-modes.md.


Quick-reference table

Flag Config key / env var Values Default Where parsed
DATA_MODE config.app.dataMode live, mock live src/config/index.ts
DATA_STORE config.app.dataStore postgres, memory auto (see below) src/config/index.ts
BET_STUB_MODE process.env.BET_STUB_MODE true, false true src/services/bet.service.ts
ROUNDS_MOCK_MODE config.app.roundsMockMode true, false false src/config/index.ts
API_ONLY process.env.API_ONLY true, false false src/index.ts
SOROBAN_FAIL_CLOSED config.soroban.failClosed true, false false src/config/index.ts

DATA_STORE auto-derivation

When DATA_MODE=mock, the config defaults DATA_STORE to memory. You can override it explicitly: DATA_MODE=mock DATA_STORE=postgres is valid. When DATA_MODE=live, DATA_STORE defaults to postgres.


Flag-by-flag behavior matrix

DATA_MODE

Controls whether price and stats data come from live external APIs or from in-memory mock data. This is the highest-level mode switch.

Endpoint DATA_MODE=live (default) DATA_MODE=mock
GET /api/prices CoinGecko (30 s cache), falls back to stale cache, then static defaults Static in-memory array (mockData.prices)
GET /api/rounds Prisma / Postgres (hackathon_rounds table) Same — Prisma is always used for rounds
GET /api/leaderboard Prisma / Postgres leaderboard table In-memory seed (mockLeaderboard) when DATA_STORE=memory
GET /api/stats Prisma / Postgres aggregation MOCK_PLATFORM_STATS constants (zero-value defaults)
GET /api/health Live Soroban RPC readiness check Soroban isReady() flag only (no RPC call)

Implementation: src/services/priceService.ts reads config.app.dataMode; src/services/stats.service.ts falls back to MOCK_PLATFORM_STATS when the DB is empty or unreachable.

BET_STUB_MODE

Controls whether /api/bets endpoints submit transactions on-chain via Soroban or just record the intent locally.

BET_STUB_MODE Behavior Use case
true (default) Bets recorded locally; no on-chain calls. Returns { state: "stub" }. Local dev, demos, hackathon — no Soroban keypairs needed
false Bets submitted to Soroban smart contract via sorobanService.placeBet / placePrecisionBet. Production or Stellar testnet with deployed contract

Affected endpoints: POST /api/bets/up-down, POST /api/bets/precision

Implementation: src/services/bet.service.ts (recordUpDownBet, recordPrecisionBet)

The active mode is logged at startup: Bet mode: STUB (no on-chain calls) or Bet mode: ON-CHAIN (Soroban).

MAX_STAKE

Circuit breaker on bet / prediction size, enforced in the Zod schemas (src/utils/max-stake.util.ts) so it applies identically to stub and on-chain paths.

Setting Value
Env var MAX_STAKE (alias: MAX_PREDICTION_AMOUNT)
Unit XLM (same unit as every amount field; never stroops)
Default 1000000 (also used if the value is unset or not a positive number)
Over-max result 400 with a field error on amount

Affected endpoints: POST /api/bets/*, POST /api/rounds/:id/bet*, POST /api/predictions/submit and batch, and legends predictions.

Data retention (expired challenges and idempotency keys)

SchedulerService runs retentionService.runAllPolicies() daily at 03:00 under the run-retention-policies distributed lock. In memory mode the MemoryHousekeepingService sweep runs the same auth-challenge and idempotency cleanups.

Data Expiry / TTL Deleted when
AuthChallenge expiresAt (challenge lifetime) and RETENTION_AUTH_CHALLENGES_TTL_DAYS (default 7) expiresAt passed or older than the TTL
IdempotencyKey Per-key expiresAt: 10 min default for checkIdempotency/storeIdempotencyResult, 24 h default for locks expiresAt passed
Chat messages / audit logs RETENTION_CHAT_MESSAGES_TTL_DAYS / RETENTION_AUDIT_LOGS_TTL_DAYS (default 90) older than the TTL

Expired idempotency keys are also ignored on read, so a stale key cannot replay a response even before the job runs. Tests: src/tests/retention-expiry.spec.ts, src/tests/idempotency.spec.ts.

SOROBAN_FAIL_CLOSED

Controls whether money paths (bet placement and round resolve) abort when Soroban chain verification fails, or silently continue with database-only settlement.

SOROBAN_FAIL_CLOSED Behavior Use case
false (default) Fail-open: log a warning and proceed with DB-only on Soroban failure Local demos, hackathons without a live contract
true Fail-closed: abort bet/resolve when chain verification fails Production / real stakes — recommended

Affected paths: UP_DOWN round create, placeBet, and resolveRound call sites in round.service, round.routes, and resolution.service. Policy helper: sorobanService.applyMoneyPathFailure.

Production recommendation: set SOROBAN_FAIL_CLOSED=true so a broken or unavailable Soroban path cannot silently skip on-chain verification.

The active mode is logged at startup: Soroban money-path policy: FAIL-CLOSED ... or FAIL-OPEN ....

ROUNDS_MOCK_MODE

Controls whether the round listing endpoint skips Soroban and the database and returns mock data immediately.

ROUNDS_MOCK_MODE Behavior
false (default) Fallback chain: Soroban → Database → Mock. Tries on-chain first, falls back to DB, then mock data as last resort.
true Skips Soroban and database entirely. Returns mock rounds from getMockRounds() immediately.

Affected endpoints: GET /api/rounds/active (production), GET /api/rounds (hackathon)

Implementation: src/services/round.service.ts (getRoundsForApi); checked in both src/routes/rounds.routes.ts and src/routes/rounds.ts. Which of those two routers is mounted is decided by the app mode — see CONTRIBUTING.md.


Recommended combinations

Operator diagnostics

The full application exposes GET /api/admin/runtime-flags to administrators. It returns only a whitelist of non-secret mode and scheduler flags, including dataMode, dataStore, roundsMockMode, and the scheduler state. It never returns DATABASE_URL, JWT secrets, Soroban secrets, or other credentials. The endpoint is not mounted by the hackathon app.

Public price and stats responses use per-endpoint browser/CDN caches (Issue #871): the price feeds (GET /api/prices, GET /api/price) default to 30 seconds (PRICES_CACHE_TTL_SECONDS), while the landing-page counters (GET /api/stats) default to a short 5 seconds (STATS_CACHE_TTL_SECONDS) so a CDN never pins stale numbers. The stats TTL also caps the origin-side in-process stats cache, so the short window is real end to end (Issue #876). Health responses are always no-store; Redis and database caching are separate concerns from HTTP caching.

1. Full local development (no external deps)

DATA_MODE=mock
BET_STUB_MODE=true
ROUNDS_MOCK_MODE=true
  • No CoinGecko calls, no Soroban, no database required.
  • All endpoints return mock data.
  • Fastest setup for UI prototyping.

2. Database-backed local development (no blockchain)

DATA_MODE=live
BET_STUB_MODE=true
ROUNDS_MOCK_MODE=false
DATABASE_URL=postgresql://...
  • Real DB, real CoinGecko prices, stub bets.
  • Good for testing DB migrations and queries locally.

3. Full blockchain testnet

DATA_MODE=live
BET_STUB_MODE=false
ROUNDS_MOCK_MODE=false
SOROBAN_CONTRACT_ID=...
SOROBAN_ADMIN_SECRET=...
SOROBAN_ORACLE_SECRET=...
SOROBAN_FAIL_CLOSED=true
DATABASE_URL=postgresql://...
  • Live CoinGecko, on-chain bets, real DB.
  • Closest to production; money paths abort if chain verification fails.

4. Hackathon / demo

DATA_MODE=mock
BET_STUB_MODE=true
ROUNDS_MOCK_MODE=true
  • No infrastructure needed. Run npm run dev:hackathon.

5. API-only stateless node (split deployment)

API_ONLY=true
DATA_MODE=live
DATABASE_URL=postgresql://...
  • Skips oracle polling, schedulers, and price ticker.
  • Still serves HTTP and WebSocket transport.

How the flags interact

┌──────────────────────────────────────────────────────────────────┐
│                        DATA_MODE                                 │
│  ┌─────────────┐                     ┌──────────────────────┐    │
│  │    mock      │                     │        live          │    │
│  │             │                     │                      │    │
│  │ Prices:     │                     │ Prices: CoinGecko    │    │
│  │   mockData  │                     │ Stats:  Prisma/DB    │    │
│  │ Stats:      │                     │                      │    │
│  │   MOCK_     │                     │                      │    │
│  │   PLATFORM_ │                     │                      │    │
│  │   STATS     │                     │                      │    │
│  └──────┬──────┘                     └──────────┬───────────┘    │
│         │                                       │                │
│         └─────── DATA_STORE ────────────────────┘                │
│                  auto: memory              auto: postgres         │
│                  (can override)            (can override)         │
└──────────────────────────────────────────────────────────────────┘

BET_STUB_MODE (independent of DATA_MODE)
  true  → stub bets (no chain)
  false → on-chain bets via Soroban

ROUNDS_MOCK_MODE (independent of DATA_MODE)
  true  → skip soroban + db, return mock rounds
  false → soroban → db → mock fallback chain

The three flags are independent — you can mix and match them:

  • DATA_MODE=live + BET_STUB_MODE=true = real prices, stub bets
  • DATA_MODE=mock + BET_STUB_MODE=false = mock prices, on-chain bets
  • ROUNDS_MOCK_MODE=true + DATA_MODE=live = mock rounds, real prices/stats
  • etc.

This independence lets you isolate exactly which external services are needed for your current workflow.


Where to find the implementation

Flag Primary implementation file(s)
DATA_MODE src/config/index.ts, src/services/priceService.ts, src/services/stats.service.ts
DATA_STORE src/config/index.ts, src/repositories/
BET_STUB_MODE src/services/bet.service.ts
ROUNDS_MOCK_MODE src/config/index.ts, src/services/round.service.ts
SOROBAN_FAIL_CLOSED src/config/index.ts, src/services/soroban.service.ts
Mock data src/data/mockData.ts

Environment file templates

  • .env.example — Full production-ready template with all flags and documentation.
  • .env.hackathon.example — Minimal template for hackathon/demo mode (mock data, no DB).

Both files are in the repository root and include these flags with inline comments.


Docker Deployment Profiles & Soroban Bindings

The multi-stage Dockerfile packages both full production (with live Soroban contracts and database migrations) and lightweight hackathon/API-only deployment profiles.

Vendored Bindings & Dependency Resolution

  • The dependency @tevalabs/xelma-bindings is declared via "file:vendor/xelma-bindings".
  • The Dockerfile explicitly copies ./vendor in both deps and runner stages to guarantee offline/container build resolution.
  • docker/entrypoint.sh executes scripts/install-bindings.js --check when DATA_MODE=live or BET_STUB_MODE=false before booting the API server.

Container Profiles

Profile Environment Configuration Entrypoint Behavior
Full Production (Live) DATA_MODE=live, BET_STUB_MODE=false, API_MODE=full Verifies Soroban bindings, applies Prisma migrations, and boots full app dist/index.js.
Demo / Hackathon DATA_MODE=mock, API_MODE=hackathon, RUN_MIGRATIONS=false Boots lightweight mock demo server dist/server.js without requiring external database or Soroban keys.
API Only API_ONLY=true, BET_STUB_MODE=true Boots standard API server without running background schedulers or oracle loops.

Kubernetes Readiness vs Liveness Probes

Kubernetes relies on two distinct probe mechanisms to manage container lifecycles and traffic routing: Liveness Probes and Readiness Probes. Configuring these incorrectly can cause cascading failure loops in production.

Key Conceptual Differences

Probe Type Purpose Failure Consequence Recommended Endpoint
Liveness Probe Detects if the Node.js process is stuck, deadlocked, or unresponsive. Container restart (Kubelet kills the container and initiates restart backoff). GET / (Full mode) or lightweight HTTP ping
Readiness Probe Detects whether the backend container is ready to accept incoming user traffic. Traffic isolation (Pod removed from Service endpoints; no restarts). GET /health (Full mode) or GET /api/health (Hackathon mode)

Warning

Never probe downstream dependencies (PostgreSQL, Redis, Soroban RPC) in a Liveness Probe! If an external database or network connection has a transient 10-second outage, a deep liveness check fails and triggers pod restarts across all replicas simultaneously. This exacerbates cluster congestion ("restart storm"). External dependencies must only be checked in Readiness Probes.


Endpoint Semantics by Mode

1. Full Backend Mode (src/index.ts, API_MODE=full)

  • Liveness (GET /):
    • Returns 200 OK with { message: "Hello World! Xelma Backend is running", status: "OK" }.
    • Exercises Express routing and event loop without querying PostgreSQL or Soroban.
  • Readiness (GET /health or GET /metrics/readiness):
    • Evaluates Prisma PostgreSQL connection, Redis cache ping, Soroban RPC initialization, and Price Oracle staleness.
    • Status semantics:
      • healthy (200 OK): All services online and operational.
      • degraded (200 OK): Non-critical dependency (e.g. stale price oracle, uninitialized Soroban testnet) is impaired, but backend can still serve requests.
      • unhealthy (503 Service Unavailable): Critical dependency (PostgreSQL database) unreachable. Pod is immediately removed from load balancing until database connectivity recovers.

2. Hackathon / Mock Mode (src/server.ts, API_MODE=hackathon)

  • Liveness: GET /api or GET /api/health
  • Readiness (GET /api/health):
    • Returns lightweight in-memory and demo status. Database and Redis checks are only evaluated if explicitly configured (DATA_STORE=postgres, REDIS_URL).

Kubernetes Manifest Examples

Production Deployment (full mode)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: xelma-backend
  labels:
    app: xelma-backend
spec:
  replicas: 3
  selector:
    matchLabels:
      app: xelma-backend
  template:
    metadata:
      labels:
        app: xelma-backend
    spec:
      containers:
        - name: api
          image: ghcr.io/tevalabs/xelma-backend:latest
          ports:
            - containerPort: 3000
          env:
            - name: NODE_ENV
              value: "production"
            - name: DATA_MODE
              value: "live"
            - name: BET_STUB_MODE
              value: "false"
            - name: SOROBAN_FAIL_CLOSED
              value: "true"
          # Liveness: checks that the process is alive. Never check DB here!
          livenessProbe:
            httpGet:
              path: /
              port: 3000
            initialDelaySeconds: 15
            periodSeconds: 10
            timeoutSeconds: 3
            failureThreshold: 3
          # Readiness: checks database, redis, soroban, and price feeds
          readinessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 2

Demo / Hackathon Deployment (hackathon mode)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: xelma-hackathon
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: hackathon-api
          image: ghcr.io/tevalabs/xelma-backend:latest
          ports:
            - containerPort: 3001
          env:
            - name: API_MODE
              value: "hackathon"
            - name: DATA_MODE
              value: "mock"
          livenessProbe:
            httpGet:
              path: /api
              port: 3001
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 3
          readinessProbe:
            httpGet:
              path: /api/health
              port: 3001
            initialDelaySeconds: 5
            periodSeconds: 10
            timeoutSeconds: 3