Local Development: Change VS Code to use Goreman

[d2burkhalter]

What type of PR is this?

Enhancement for VS Code launch configurations that will align its debugger to use processes launched from Goreman with configs and docker compose in /_local to consolidate how we do local development. Also adds launch configurations for starting Armada with authorization and prometheus.

What this PR does / why we need it

• add launch config for running with Prometheus
• add launch config for running with Authorization
• add launch config for running with Prometheus and Authorization
• fix race condition of migration task running before Postgres is ready
• add task for full teardown of local development resources
• consolidate local development to use /_local

[greptile-apps]

Greptile Summary

This PR consolidates VS Code local development to use Goreman with dlv-dap, aligning it with the existing mage dev:up workflow and adding auth and Prometheus variants.

• VS Code configs rewritten: all launch configurations switch from direct mode: auto to mode: exec dlv-dap attach; six new compound configurations cover no-auth / auth / fake-executor × with/without Prometheus.
• New task orchestration: a dependsOrder: "sequence" pipeline runs prebuild → port-check → mage dev:up (goreman) → wait-for-dlv.sh before any debugger attaches, replacing the previous ad-hoc dep setup.
• mage dev:up extended: Dev.Up now accepts a comma-separated profiles string and an optional *bool DAP flag, routing auth/fake-executor tokens to the goreman procfile and all other tokens as docker-compose --profile flags — enabling the new Prometheus profile.

Confidence Score: 5/5

This is a developer-tooling-only change with no production code paths modified; merging carries no risk to running services.

All changes are confined to VS Code configs, shell scripts, procfiles, and the mage dev namespace. The core launch flow is well-structured with an explicit prebuild → goreman → dlv-readiness pipeline. The only notable quirk is that tasks.json passes "no-auth" as a profiles token when the doc comment recommends "", but this happens to work because no docker-compose profile with that name exists. No production code or migration files are touched.

.vscode/tasks.json — the "no-auth" token passed to mage dev:up works today but is semantically inconsistent with the updated Dev.Up implementation.

Important Files Changed

Filename	Overview
.vscode/launch.json	Replaced direct-launch mode with dlv-dap "exec" attach mode for all 13 services; added auth/prometheus compound configs; removed old env-var–based configs.
.vscode/tasks.json	Rewrote task definitions to orchestrate prebuild → port-check → goreman → dlv-ready sequence; "Start dependencies" calls mage dev:up no-auth -dap but "no-auth" is not a recognized token in the updated Dev.Up switch statement.
magefiles/dev.go	Dev.Up refactored from single profile string to comma-separated profiles+dap flag; properly handles auth/fake-executor goreman profiles, routes remaining tokens as docker-compose profiles, and invokes waitForKeycloak for auth mode.
_local/scripts/prebuild-services.sh	New script that warms the Go build cache by compiling all 9 services to /dev/null in parallel; includes stamp-based change detection and a heartbeat for long-running builds.
_local/scripts/wait-for-dlv.sh	New script that polls lsof for each dlv DAP port (2345–2353) serially with a 60s timeout per service; correctly varies the executor port based on standard vs fake-executor mode.
_local/compose/stack.yaml	Added prometheus service under the "prometheus" profile with host.docker.internal extra_host, health check via wget, and volume mount for prometheus.yml.
_local/prometheus.yml	New Prometheus scrape config targeting all 9 Armada services on host.docker.internal; port assignments match updated metricsPort values in service configs.
_local/scheduleringester/config.yaml	Fixed metrics config key from nested metrics.port to the correct top-level metricsPort: 9006, aligning with other service configs.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant VSCode as VS Code
    participant Tasks as tasks.json
    participant PB as prebuild-services.sh
    participant Mage as mage dev:up
    participant DC as Docker Compose
    participant Goreman as Goreman (Procfile)
    participant DLV as dlv dap (x8)
    participant WaitDLV as wait-for-dlv.sh

    VSCode->>Tasks: Launch compound (e.g. Armada no-auth)
    Tasks->>PB: Pre-build Go services
    PB->>PB: go build -o /dev/null (warm cache)
    PB-->>Tasks: Cache warm complete
    Tasks->>Tasks: Check for port conflicts
    Tasks->>Mage: "mage kind && mage dev:up no-auth -dap"
    Mage->>DC: docker compose up -d --wait (core services)
    DC-->>Mage: postgres / redis / pulsar healthy
    Mage->>Goreman: goreman -f no-auth-dap.Procfile start
    Goreman->>DLV: "go build ... -o dist/armada-* && dlv dap --listen=PORT"
    DLV-->>VSCode: DAP server listening at (first service)
    VSCode->>Tasks: endsPattern matched, proceed
    Tasks->>WaitDLV: wait-for-dlv.sh standard
    WaitDLV->>WaitDLV: poll lsof for ports 2345-2352
    WaitDLV-->>Tasks: All 8 dlv servers ready
    VSCode->>DLV: DAP launch request per service
    DLV->>DLV: "exec dist/armada-* --config _local/config.yaml"

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant VSCode as VS Code
    participant Tasks as tasks.json
    participant PB as prebuild-services.sh
    participant Mage as mage dev:up
    participant DC as Docker Compose
    participant Goreman as Goreman (Procfile)
    participant DLV as dlv dap (x8)
    participant WaitDLV as wait-for-dlv.sh

    VSCode->>Tasks: Launch compound (e.g. Armada no-auth)
    Tasks->>PB: Pre-build Go services
    PB->>PB: go build -o /dev/null (warm cache)
    PB-->>Tasks: Cache warm complete
    Tasks->>Tasks: Check for port conflicts
    Tasks->>Mage: "mage kind && mage dev:up no-auth -dap"
    Mage->>DC: docker compose up -d --wait (core services)
    DC-->>Mage: postgres / redis / pulsar healthy
    Mage->>Goreman: goreman -f no-auth-dap.Procfile start
    Goreman->>DLV: "go build ... -o dist/armada-* && dlv dap --listen=PORT"
    DLV-->>VSCode: DAP server listening at (first service)
    VSCode->>Tasks: endsPattern matched, proceed
    Tasks->>WaitDLV: wait-for-dlv.sh standard
    WaitDLV->>WaitDLV: poll lsof for ports 2345-2352
    WaitDLV-->>Tasks: All 8 dlv servers ready
    VSCode->>DLV: DAP launch request per service
    DLV->>DLV: "exec dist/armada-* --config _local/config.yaml"

Loading

_{Reviews (25): Last reviewed commit: "fix: cache prebuilds" | Re-trigger Greptile}

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[d2burkhalter]

@greptileai

[d2burkhalter]

@greptileai

[dejanzele]

host.docker.internal is a very OSX-specific feature, can we use something which would work on both Linux and OSX?

[d2burkhalter]

I think the extra_hosts property here should allow it to work the same on osx and linux

  prometheus:
    container_name: prometheus
    image: ${PROMETHEUS_IMAGE:-prom/prometheus:v3.11.3}
    extra_hosts:
      - "host.docker.internal:host-gateway"

[dejanzele]

We might have many sections in the README.md which could go in this docs/developer_guide.md, could you debloat the README also?

[d2burkhalter]

I think we should have large doc changes in a separate PR. Want to try and get a consensus for what we want to do with README.md, docs/ and website/ and go from there