Living document. GameDay (Event Mode) is being built PR by PR. This README tracks how to deploy and operate it as features land, and marks what works today vs. what's coming. The adoption-mode product is documented in
README.md; the deep design lives indocs/.
Databricks Quest runs in two modes from one codebase and one build:
- Adoption Mode (default) — the system-table-driven platform-adoption game.
Unchanged; see
README.md. - Event Mode (GameDay) — configurable, quest-pack-driven, team-based events with validation, scoring, leaderboards, host controls, and (optionally) many workspaces federating into one.
Nothing here breaks Adoption Mode. Event Mode is opt-in and off by default —
a deploy with no GameDay flags is the legacy adoption app, byte-for-byte: the
GameDay APIs return 404, the Event UI is hidden, and the GameDay schema
migrations are skipped. Enable GameDay explicitly with --event-mode (or set
QUEST_EVENT_MODE=on); the master/child roles imply it.
Live status lives in one place:
docs/STATUS.md — the authoritative per-PR tracker (what's
landed, what's deployable/testable, known gaps). In short: PR01–PR12 and the
federation plumbing (PR13–PR16) have landed — schema, quest packs, the
validation/scoring write path, event/team lifecycle, the player gameplay UI, the
host console, the live player leaderboard with hint-penalty scoring,
namespace-guarded team resource bootstrap/reset, two shipped sample quest packs,
security/observability hardening (request ids, structured logs, expanded
health, permission-model docs), the post-event report with JSON/CSV/Markdown
export for account follow-up, and release hardening (dual-mode README plus
troubleshooting, release-checklist, and manual-E2E/load-test guides). The GameDay
MVP is field-ready.
What this means for testing: the full loop is testable now — a host can create an event, import a pack, run the lifecycle, players join and submit attempts, validators score them, and the host monitors attempts, posts announcements, and adjusts scores from the console. See Testing.
Everything is selected by runtime parameters (env vars set from deploy.sh
flags). There are two orthogonal switches:
QUEST_EVENT_MODE(--event-mode) — the GameDay master switch, off by default. Off = legacy adoption app (GameDay APIs404, Event UI hidden, GameDay migrations skipped). On = GameDay surfaces activate. Implied by themaster/childroles.QUEST_ROLE— federation topology, only meaningful once Event Mode is on:
| Role | What it is | Lakebase | GameDay migrations | Scoring pipeline |
|---|---|---|---|---|
standalone (default) |
Single-workspace app | provisions its own local | only if --event-mode |
runs (adoption) |
master |
Owns the shared event DB + host console (implies Event Mode) | provisions its own | runs (incl. 002) |
runs (adoption) |
child |
One per attendee; writes to the master's DB (implies Event Mode) | points at master | skipped | skipped |
See adr/ADR_006_SHARED_LAKEBASE_MULTI_WORKSPACE_FEDERATION.md
for why children connect directly to the master's Lakebase over Postgres
(instead of HTTP between apps).
Event Mode requires the Lakebase backend. The optional SQL warehouse data backend (
--data-backend warehouse, toggled under Admin -> Data Backend) is read-only — it serves adoption reads directly from the scored Delta tables. GameDay writes (team bootstrap, attempts, live scoring) need Lakebase, so keep the active backend on Lakebase when running an event.
The Admin page (/api/admin/*) is gated by an allowlist, independent of
Event Mode and applies to every deploy, including the legacy adoption app.
Admins are stored in Lakebase (the quest_admins table), which is the
shared source of truth. In federation the master owns it and child apps read it
through the shared event-writer credential — so an admin is automatically an
admin across the standalone app, the master, and every child workspace. The
effective admin set is the union of that table and the deploy-time env
allowlist (the bootstrap/fallback).
Seed the initial admins with --admins:
./deploy.sh --admins "alice@corp.com,bob@corp.com"Behaviour:
--adminsprovided → those emails are seeded intoquest_admins; only admins see the Admin page (others get403and the nav item is hidden).--adminsomitted →deploy.shdefaults to the deploying user (so the page is never open to everyone) — except--role child, which seeds nothing and inherits the admin list from the master's shared DB.
Admins manage admins in-app. The Admin page has an Admin Access card to add/remove admins at runtime (or use the API directly):
# List admins
curl -s "$APP/api/admin/admins"
# Add an admin (any admin can grant admin; works from master or a child app)
curl -s -X POST "$APP/api/admin/admins" -H 'Content-Type: application/json' \
-d '{"email":"carol@corp.com"}'
# Remove an admin — from the MASTER/standalone app only (child role has no
# DELETE on the shared table; last-admin removal is refused to avoid lockout)
curl -s -X DELETE "$APP/api/admin/admins/carol@corp.com"Mechanics: --admins sets QUEST_ADMIN_ALLOWLIST in app.yaml (the seed +
fallback), and /api/profile returns is_admin so the frontend hides the nav.
The page is "open to all" only when nothing is configured anywhere — no env
allowlist and an empty/unreachable quest_admins (e.g. running locally
without deploy.sh).
The Host console (/api/host/*) is GameDay-only and gated separately from
the Admin page. A caller is a host when they are in QUEST_HOST_ALLOWLIST, are
an admin, or are listed in the event's event_hosts rows (managed in-app via
/api/host/events/{id}/hosts).
The gate is fail-closed. In Event Mode, if no host authority is configured
anywhere — empty allowlist, no admins, and no event_hosts for the event —
access is denied (not open). The only escape hatch is the dev-only
QUEST_HOST_OPEN=1, intended for local testing.
Seed the allowlist at deploy time:
./deploy.sh --event-mode --host-allowlist "host1@corp.com,host2@corp.com"Any --event-mode / --role master deploy requires either --host-allowlist
or --admins (defaulting to the deploying user, as before) and the deploy
script prints the effective host authority loudly so you can't ship an event
nobody can host. --host-allowlist writes QUEST_HOST_ALLOWLIST into
app.yaml. The frontend Host tab keys off the same rule (lobby.is_host), so
"see the tab → 403" and "allowlisted → no tab" can't happen.
Legacy adoption app (default). No GameDay surface at all:
./deploy.sh/api/health reports "event_mode": false; /api/events/*, /api/host/*, and
/api/federation/* return 404.
Single-workspace GameDay. For a small event run entirely in one workspace,
add --event-mode:
./deploy.sh --event-mode/api/health then reports "event_mode": true and migrations 001_gameday_core
002_federationapplied. You can then import a quest pack (below).
One master app on the admin workspace owns the shared Lakebase; each attendee workspace runs a child app that writes to it.
./deploy.sh --role master --event anz-gamedayThe master provisions its Lakebase, runs migrations, and creates the shared
INSERT-only event-writer role (quest_event_writer). It generates a random
secret for that role and prints a ready-to-copy child command in the success
banner — this is where the token comes from (see below).
You do not invent or fetch the token separately — the master deploy mints it
for you. deploy.sh --role master:
- generates a random secret (
python3 -c "import secrets; print(secrets.token_urlsafe(24))"), - creates/updates the
quest_event_writerPostgres role on the master Lakebase with that secret (INSERT on the fact tables + SELECT on the read views), and - prints it once in the success banner:
── Shared event-writer credential (give to child workspaces) ──
Children deploy with these flags (rotate per event):
./deploy.sh \
--role child \
--event anz-gameday \
--master-lakebase-host <master-host> \
--master-lakebase-user quest_event_writer \
--master-lakebase-token '<generated-secret>'
The <generated-secret> in that banner is the
--master-lakebase-token <event-writer-credential> you pass to children.
Important:
- Shown once, not persisted. The script does not store the secret. Copy it into your secret store immediately.
- Lost it? Re-run the master deploy. Provisioning does
ALTER ROLE ... PASSWORDon re-run, so it rotates the secret — then redistribute the new one. - It's a Postgres role password, not an OAuth token. Child workspace OAuth identities aren't accepted by the master's Lakebase, which is why a shared credential is used (see ADR_006). Treat it as event-scoped and rotate per event.
Prefer passing the token via an environment variable / secret rather than inline, so it doesn't land in shell history or process listings:
export QUEST_WRITER_TOKEN='<generated-secret>' # from the master banner / your secret store
./deploy.sh --role child \
--master-lakebase-host <master-lakebase-host> \
--master-lakebase-user quest_event_writer \
--master-lakebase-token "$QUEST_WRITER_TOKEN" \
--event anz-gameday \
--workspace-id ws-anzgt-01Notes:
-
Setting
--master-lakebase-hostalone defaults--roletochild. -
--master-lakebase-userdefaults toquest_event_writer; only pass it if you changed the role name on the master. -
The child skips local Lakebase provisioning, migrations, and the scoring pipeline — it only writes event facts to the shared DB.
-
--workspace-idis how federated writes are attributed; it defaults to the workspace host if omitted. Give each attendee workspace a unique value. -
The child checks itself into
event_workspaceson startup (a DB upsert, not an HTTP call), so it appears in the master's Workspaces health panel. -
Verify before the event — confirm reachability and that the credential is correctly INSERT-only:
PGPASSWORD="$QUEST_WRITER_TOKEN" python3 scripts/federation_spike.py \ --host <master-lakebase-host> --db quest_db --user quest_event_writer
./deploy.sh --helpQuests are configuration, not code. The single guided walkthrough is
docs/AUTHORING_QUEST_PACKS.md (scaffold →
author → lint → import → version-bump); the deep-dive references are
samples/QUEST_PACK_SCHEMA.md,
samples/SAMPLE_VALIDATOR_LIBRARY.md, and
samples/SAMPLE_QUEST_PACK_AI_BI.md. A
committed Cursor skill (.cursor/skills/quest-pack-author/) drives both the
create and update flows for agents.
Validator types. sql_assertion (read-only SELECT/WITH against the
team's bootstrapped namespace) and databricks_sdk/workspace_api (read-only
workspace lookups — dashboard_exists_for_team, table_exists,
genie_space_exists, job_exists_with_schedule, …) execute today;
manual routes to host review. The live, executable set is reported by
GET /api/health (validator_types + sdk_checks) so authors never claim a
check runs when it is actually host-reviewed. Always pair every databricks_sdk
task with a manual fallback so a pilot is never blocked when a check can't run.
Lint locally without a server:
python scripts/lint_quest_pack.py quest_packs/built_in/ai_bi_gameday.ymlLint / import / list against a running app (authenticated session or forwarded identity headers):
# Lint
curl -sX POST "$APP_URL/api/host/quest-packs/lint" \
-H 'Content-Type: application/json' \
--data "$(python3 -c 'import json; print(json.dumps({"manifest_yaml": open("quest_packs/built_in/ai_bi_gameday.yml").read()}))')"
# Import (immutable per (slug, version) — bump the version to change content)
curl -sX POST "$APP_URL/api/host/quest-packs/import" \
-H 'Content-Type: application/json' \
--data "$(python3 -c 'import json; print(json.dumps({"manifest_yaml": open("quest_packs/built_in/ai_bi_gameday.yml").read()}))')"
# List
curl -s "$APP_URL/api/host/quest-packs"Once a pack is imported, a host can run an event entirely over the API. Players
self-join; hosts can also place people on teams directly. Event status governs
play: only an active event accepts attempts (others return
409 EVENT_NOT_ACTIVE).
# 1. Create an event from an imported pack version (creator becomes owner-host)
curl -sX POST "$APP_URL/api/host/events" -H 'Content-Type: application/json' \
--data '{"title":"ANZ GameDay","pack_version_id":"<packver_id>","slug":"anz-gameday"}'
# 2. Create teams
curl -sX POST "$APP_URL/api/host/events/anz-gameday/teams" \
-H 'Content-Type: application/json' --data '{"name":"red","display_name":"Red Team"}'
# 3a. Bulk-import participants (teams created on demand, idempotent)
curl -sX POST "$APP_URL/api/host/events/anz-gameday/participants/import" \
-H 'Content-Type: application/json' \
--data '{"participants":[{"email":"ada@corp.com","team_name":"red"}]}'
# 3b. …or assign someone to a team directly (single team per event; reassigns)
curl -sX POST "$APP_URL/api/host/events/anz-gameday/teams/<team_id>/members" \
-H 'Content-Type: application/json' --data '{"user_id":"ada@corp.com"}'
# 4. Open play, then players join + submit
curl -sX POST "$APP_URL/api/host/events/anz-gameday/start"
curl -sX POST "$APP_URL/api/events/anz-gameday/join" \
-H 'Content-Type: application/json' --data '{"team_name":"red"}'
# 5. Lifecycle: pause | freeze | complete (and ready | archive)
curl -sX POST "$APP_URL/api/host/events/anz-gameday/freeze"Lifecycle state machine: draft → ready → active → paused/frozen → completed → archived. start stamps starts_at, complete stamps ends_at, freeze
stamps scoring_frozen_at. Players see joinable/active events at GET /api/events
and the lobby at GET /api/events/{id}. Every mutation writes an
event_audit_log row.
When Event Mode is on, the sidebar shows an Event tab (the player UI). It sits on the read endpoints below and the PR03 attempt endpoint — no API calls needed by hand:
- Lobby — pick an active event (
GET /api/events), see its teams and counts (GET /api/events/{id}), set a display name, and join a team (POST /api/events/{id}/join). - Quests — the event pack's quests with your team's progress bar
(
GET /api/events/{id}/quests). - Quest runner — open a quest (
GET /api/events/{id}/quests/{quest_id}) to read the narrative, work each task, reveal hints, and submit a JSONsubmission. The submit button callsPOST /api/events/{id}/tasks/{task_id}/attemptsand shows a live pass/fail/manual/error badge plus the player-safe per-validator messages and points awarded. Submissions are disabled unless the event isactiveand you have joined. - Team dashboard — score, rank, members, task progress, and recent scoring
(
GET /api/events/{id}/team).
Every event view now has a Standings tab (PR07, see below). Role behaviour:
master deployments show the Host Console instead of the player UI; child
deployments get the player UI and their Standings tab renders the event-wide
federated leaderboard (own-team rank highlighted). A child app is pinned to
its configured event; a standalone app lets the player choose from the event
list.
All players also see a host announcement banner at the top of the event
view (GET /api/events/{id}/announcements).
Facilitators run the event from a Host tab that appears inside the Event
view only for hosts (is_host on the lobby; backed by QUEST_HOST_ALLOWLIST).
Players never see it. It's gated by require_host, so a non-host gets a 403 on
the underlying endpoints even if they craft the URL.
What the host can do from the console (GET /api/host/events/{id} drives the
dashboard):
- Lifecycle controls — start / pause / freeze / complete / mark-ready /
archive. Only the legal next transitions render (
allowed_transitions), enforcing the PR04 state machine. Submissions are open only whileactive. - Teams & scores — ranked table of teams with points and member counts
(
GET /api/host/events/{id}/teams). - Validation attempts inspector — filter by
all/passed/failed/error/manual/ running(GET /api/host/events/{id}/attempts?status=) and expand any attempt to see per-validator results including the private diagnostics (GET /api/host/events/{id}/attempts/{attempt_id}) that players never see. - Announcement composer — post
info/warning/criticalbroadcasts (POST /api/host/events/{id}/announcements); players see them in the banner. - Manual score adjustment — add or subtract points for a team with a
required reason (
POST /api/host/events/{id}/adjustments). This writes amanual_adjustmentsaudit row and a matchingscoring_eventsledger row in one transaction, so the leaderboard updates immediately. Every action is written toevent_audit_log. - Quest pack import / lint — paste a manifest, lint it, then import a new
immutable version (
POST /api/host/quest-packs/lintand/import, PR02).
Note on the manual-adjustment ledger row: it carries a unique idempotency key (the adjustment id), so unlike base-points awards it is never deduplicated — a manual override is intentional and always lands.
Every event has a Standings tab (all roles). For standalone/master it calls
GET /api/events/{id}/leaderboard and renders:
- a podium (top 3) and a ranked table of all teams, with the caller's
own team highlighted (and surfaced with a
nullrank when on a team but not yet scored); - a recent activity feed — team + task names with signed point deltas, including task passes, hint penalties, and host manual adjustments;
- a freeze / final badge — when the event is
frozen/completed/archivedor hasscoring_frozen_atset, the board shows "Scoring frozen" / "Final results" and no new player scoring lands.
Ranking is deterministic. The event_leaderboard view ranks by
total_points DESC then last_scored_at ASC, so ties go to the team that
reached the total first — the order never flickers between refreshes.
The board is live. The Standings tab polls every ~10s on top of the manual
Refresh and pauses while the browser tab is hidden, so spectators and players
see scores move without reloading. Player attempt views poll the attempt record
after submit so queued/running validations resolve in-place.
Hint penalties now affect the score. Hint bodies are withheld until a player
reveals them. Revealing calls
POST /api/events/{id}/tasks/{task_id}/hints/{hint_id}/reveal, which returns the
body and charges the penalty once per team (idempotent — re-revealing is
free). The penalty is normalised to a non-positive delta, so an author can write
-10 (canonical) or 10 and it always subtracts. While the event is paused or
frozen the body is still shown but no penalty is charged (no new scoring when
play is closed).
What updates a team's score (all land in the scoring_events ledger):
| Source | source_type |
Sign |
|---|---|---|
| Passing a task's validators | validation |
+points (once per scope) |
| Revealing a hint | hint_penalty |
−penalty (once per team) |
| Host manual adjustment | manual_adjustment |
± (always lands) |
Two built-in, ready-to-run packs live in samples/packs/ and
prove the platform end-to-end:
- AI/BI GameDay (
ai_bi_gameday.yml) — govern a schema, build a trusted revenue model, ship an AI/BI dashboard + Genie space. 3 quests / 6 tasks. - Lakehouse Foundations (
lakehouse_foundations.yml) — the bronze→silver medallion loop with a provable quality gate. 3 quests / 6 tasks.
Each ships learning objectives, resources.seed_sql (per-team seed data that the
PR08 bootstrap runs), quest_completed unlock gating, sql_assertion +
databricks_sdk + manual validators, hints, and facilitator notes. Both lint
and import cleanly and are covered by tests/test_sample_packs.py.
Run flow and customization guide:
samples/packs/README.md. In short: import the pack →
create an event + teams → bootstrap resources → start → play. The first quest
opens with a warehouse-independent SELECT 1 check so teams can confirm their
warehouse binding before the timed quests.
When an event wraps, the host gets a Report panel in the Host console (and two API endpoints) that turn the event into an enablement/account leave-behind:
- Structured report —
GET /api/host/events/{event_id}/reportreturns event summary, leaderboard, a team×task completion matrix, validation failures, hint usage, blockers (hardest tasks), champions/high performers, and recommended follow-ups. The follow-ups are heuristics for the field motion: reinforce the tasks most teams got stuck on, review hint-heavy tasks for doc/product gaps, set up 1:1s for low-completion teams, and recognise top performers as potential champions/references. - Export —
GET /api/host/events/{event_id}/export?format=json|csv|markdowndownloads the report. CSV is team-centric (rank, points, completion %, hints used, plus a 0/1 column per task) and drops straight into a spreadsheet; cells are guarded against formula injection. Markdown is a readable recap. JSON is the full structured payload.
Every export is audited (report.export). The endpoints are host-only and
read-only, and degrade gracefully if Lakebase is only partially available.
The host can provision and tear down each team's Databricks resources from a Resources panel in the Host console — repeatably and safely.
- Per-team namespace — each team gets
catalog.schemacomputed byservices/namespace.py:catalog= the team'steam_catalog, or the event'sconfig_json.resource_namespace.catalog, or the defaultquest_<event-slug>;schema= the team'steam_schema, or<schema_prefix><team-name>. These are the same values the validators template (${team_catalog}/${team_schema}), so provisioned resources line up with what tasks check. - Dry-run first — "Dry-run plan" (
POST .../resources/plan) shows the exactCREATE CATALOG/SCHEMA(+ pack seed SQL) statements, flagging anything out-of-namespace, without running anything. No warehouse needed. - Bootstrap —
POST .../resources/bootstrapruns the plan on the configured SQL warehouse (QUEST_SQL_WAREHOUSE_ID), idempotently (IF NOT EXISTS), and seeds sample data from the pack's optionalresources.seed_sql. - Reset —
POST .../resources/resetdrops every team schema only within the event's namespace. It requiresconfirm: trueand refuses the whole plan if any target is outside the namespace (a reserved catalog likemain/system, a bare catalog, a wildcard, or another event's schema) — nothing is dropped. Every action (including a refusal) lands inevent_audit_log; resource health is tracked inevent_resources.
Safety: the namespace guard is pure and unit-tested, and is the sole authority on what is in-namespace — an executor bug cannot widen it. A reset can never touch a schema this event did not compute.
Add a pack seed section like this to auto-create warm-up data per team:
resources:
seed_sql:
- "CREATE TABLE IF NOT EXISTS ${team_catalog}.${team_schema}.warmup (id INT)"Once children are deployed and an event exists, the host works from the master:
-
Import roster — map generic lab users to real people/teams. CSV columns:
workspace_id(orworkspace_host),lab_user_email,team_name, optionaldisplay_name,real_email. Re-import is idempotent and re-attributes previously unmapped scores.curl -sX POST "$MASTER_APP_URL/api/host/events/<event_id>/roster/import" \ -H 'Content-Type: application/json' \ --data '{"csv":"workspace_id,lab_user_email,display_name,team_name\nws-anzgt-01,labuser+1@awsbricks.com,Ada Lovelace,Red Team"}'
-
Workspace health —
GET /api/host/events/<event_id>/workspaces -
Unmapped identities —
GET /api/host/events/<event_id>/identities/unmapped
The same panels are available in the master's Host Console UI; children see the event-wide leaderboard and their own team's rank in their Event tab.
Full operator runbook:
docs/10_EVENT_OPERATIONS_PLAYBOOK.md.
Tiered, because end-to-end gameplay depends on PR03 + PR04:
Tier 0 — local, no Databricks (works now):
cd frontend && npm install && npm run build
python -m compileall app notebooks
pytest tests/ # federation unit testsTier 1 — infra, on deploy (works now):
-
Standalone/master/child boot;
/api/healthshows migrations001+002. -
Migration 002 is idempotent (re-run safe).
-
Connectivity + credential scope — from a child, verify reachability, auth, and that the writer role is INSERT-only:
python scripts/federation_spike.py --host <master-host> --db quest_db --user quest_event_writer
-
Child startup check-in appears in the master Workspaces panel.
Tier 2 — federation reads (needs an events row):
- Roster import → teams/participants/identity map;
/api/federation/statusresolves a team;unmapped_identitiesandevent_leaderboardviews populate once ascoring_eventsrow exists. - Until PR04 adds event creation, seed an
eventsrow (and a samplescoring_eventsrow) directly to exercise these.
Tier 3 — full end-to-end (works now — PR03 + PR04):
- Host creates event + teams, players join, a player completes a quest → validated → scored → appears on the master leaderboard → child sees its rank.
- Standalone: same flow in one workspace (
--event-mode). Verified via the event-run API flow above; non-active statuses block attempts.
- Architecture:
docs/05_TARGET_ARCHITECTURE.md - Data model:
docs/07_DATA_MODEL.md - API contract:
docs/08_API_CONTRACT.md - Quest model + validation engine:
docs/06_QUEST_MODEL_AND_VALIDATION_ENGINE.md - PR sequence:
docs/13_PR_ALIGNED_SPRINT_PLAN.md - Federation decision:
adr/ADR_006_SHARED_LAKEBASE_MULTI_WORKSPACE_FEDERATION.md