Skip to content

chore: migrate diagnose & capture-screen to multi-gateway TARGETS_JSON #88

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Pigbibi merged 1 commit into from
Jul 7, 2026
Merged

chore: migrate diagnose & capture-screen to multi-gateway TARGETS_JSON #88

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 61 additions & 9 deletions .github/workflows/capture-screen.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ name: Capture Gateway Screen
on:
workflow_dispatch:
inputs:
target:
description: Gateway target (from IB_GATEWAY_TARGETS_JSON)
required: false
default: all
type: string
screen_action:
description: Optional interaction before capturing the screen
required: false
Expand All @@ -19,27 +24,74 @@ on:
default: false
type: boolean

env:
GCP_PROJECT_ID: interactivebrokersquant
GCP_WORKLOAD_IDENTITY_PROVIDER: projects/303168642265/locations/global/workloadIdentityPools/github-actions/providers/github-ibkr-gateway-main
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: ibkr-gateway-deploy@interactivebrokersquant.iam.gserviceaccount.com

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: false

jobs:
resolve:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.resolve.outputs.matrix }}
steps:
- name: Resolve gateway targets
id: resolve
env:
TARGETS_JSON: ${{ vars.IB_GATEWAY_TARGETS_JSON }}
SELECTED_TARGET: ${{ github.event.inputs.target || 'all' }}
run: |
import json, os, sys

raw = os.environ.get("TARGETS_JSON", "").strip()
if not raw:
raise SystemExit("IB_GATEWAY_TARGETS_JSON is required")

loaded = json.loads(raw)
targets = []
if isinstance(loaded, dict):
for name, config in loaded.items():
cfg = dict(config)
cfg["name"] = name

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve names embedded in object targets

The resolver now overwrites every object-form entry's name with the map key, while .github/workflows/main.yml uses target.setdefault("name", name) for the same IB_GATEWAY_TARGETS_JSON schema. If the repo variable already relies on an explicit name inside the target object (which main accepts), manual target selection by that gateway name fails here and the matrix runs under the wrong name; use setdefault and validate names consistently.

Useful? React with 👍 / 👎.

targets.append(cfg)
elif isinstance(loaded, list):
targets = loaded
else:
raise SystemExit("IB_GATEWAY_TARGETS_JSON must be an object or list")

selected_name = os.environ.get("SELECTED_TARGET", "all")
if selected_name == "all":
selected = targets
else:
selected = [t for t in targets if t["name"] == selected_name]
if not selected:
known = ", ".join(t["name"] for t in targets)
raise SystemExit(f"Unknown target {selected_name!r}. Known: {known}")

with open(os.environ["GITHUB_OUTPUT"], "a") as f:
f.write("matrix=" + json.dumps({"target": selected}, separators=(",", ":")) + "\n")
print("Targets: " + ", ".join(t["name"] for t in selected))
shell: python3

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use a valid Python shell for target resolution

Both migrated resolve jobs set shell: python3, but GitHub Actions only treats python as the built-in Python shell and custom shell strings must include {0} (per the workflow syntax docs). On hosted runners this is rejected as an invalid shell option before the Python code runs, so the workflow never produces needs.resolve.outputs.matrix; use shell: python or shell: python3 {0} here and in the matching diagnose.yml resolve step.

Useful? React with 👍 / 👎.


capture:
needs: resolve
if: needs.resolve.outputs.matrix != '{"target":[]}'
runs-on: ubuntu-latest
timeout-minutes: 8
strategy:
matrix: ${{ fromJson(needs.resolve.outputs.matrix) }}
Comment on lines +80 to +81

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Give each capture matrix artifact a unique name

When target is left as all and the matrix contains more than one gateway, each matrix job still reaches the upload step with the same artifact name ibkr-gateway-screen. The upload-artifact action rejects duplicate artifact names from matrix/multiple jobs, so only the first target can upload and the rest fail instead of producing screenshots; include matrix.target.name in the artifact name.

Useful? React with 👍 / 👎.

fail-fast: false
permissions:
contents: read
id-token: write
env:
GCE_USER: ${{ vars.IB_GATEWAY_GCE_USER }}
GCE_INSTANCE_NAME: ${{ vars.IB_GATEWAY_INSTANCE_NAME }}
GCE_ZONE: ${{ vars.IB_GATEWAY_ZONE }}
SSH_PRIVATE_KEY_SECRET_NAME: ${{ vars.IB_GATEWAY_SSH_PRIVATE_KEY_SECRET_NAME }}
TARGET_NAME: ${{ matrix.target.name }}
GCP_PROJECT_ID: ${{ matrix.target.gcp_project_id }}
GCP_WORKLOAD_IDENTITY_PROVIDER: ${{ matrix.target.gcp_workload_identity_provider }}
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: ${{ matrix.target.gcp_workload_identity_service_account }}
GCE_USER: ${{ matrix.target.gce_user }}
GCE_INSTANCE_NAME: ${{ matrix.target.gce_instance_name }}
GCE_ZONE: ${{ matrix.target.gce_zone }}
SSH_PRIVATE_KEY_SECRET_NAME: ${{ matrix.target.ssh_private_key_secret_name }}
Comment on lines +87 to +94

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass the selected container name to capture

This matrix env block only carries VM and SSH fields from TARGETS_JSON, while the capture commands later still run against the literal container ib-gateway. Targets deployed by main.yml can set container_name, so selecting any non-default gateway captures the wrong/default container or fails even though the VM target was resolved correctly; pass matrix.target.container_name through and default it to ib-gateway in the remote command.

Useful? React with 👍 / 👎.

SCREEN_ACTION: ${{ github.event.inputs.screen_action }}
WAIT_FOR_SECOND_FACTOR: ${{ github.event.inputs.wait_for_second_factor }}
steps:
Expand Down
75 changes: 64 additions & 11 deletions .github/workflows/diagnose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,83 @@ on:
branches: [ main ]
paths:
- '.github/workflows/diagnose.yml'
workflow_dispatch: {}

env:
GCP_PROJECT_ID: interactivebrokersquant
GCP_WORKLOAD_IDENTITY_PROVIDER: projects/303168642265/locations/global/workloadIdentityPools/github-actions/providers/github-ibkr-gateway-main
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: ibkr-gateway-deploy@interactivebrokersquant.iam.gserviceaccount.com
workflow_dispatch:
inputs:
target:
description: Gateway target (from IB_GATEWAY_TARGETS_JSON)
required: false
default: all
type: string

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: false

jobs:
resolve:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.resolve.outputs.matrix }}
steps:
- name: Resolve gateway targets
id: resolve
env:
TARGETS_JSON: ${{ vars.IB_GATEWAY_TARGETS_JSON }}
SELECTED_TARGET: ${{ github.event.inputs.target || 'all' }}
run: |
import json, os, sys

raw = os.environ.get("TARGETS_JSON", "").strip()
if not raw:
raise SystemExit("IB_GATEWAY_TARGETS_JSON is required")

loaded = json.loads(raw)
targets = []
if isinstance(loaded, dict):
for name, config in loaded.items():
cfg = dict(config)
cfg["name"] = name
targets.append(cfg)
elif isinstance(loaded, list):
targets = loaded
else:
raise SystemExit("IB_GATEWAY_TARGETS_JSON must be an object or list")

selected_name = os.environ.get("SELECTED_TARGET", "all")
if selected_name == "all":
selected = targets
else:
selected = [t for t in targets if t["name"] == selected_name]
if not selected:
known = ", ".join(t["name"] for t in targets)
raise SystemExit(f"Unknown target {selected_name!r}. Known: {known}")

with open(os.environ["GITHUB_OUTPUT"], "a") as f:
f.write("matrix=" + json.dumps({"target": selected}, separators=(",", ":")) + "\n")
print("Targets: " + ", ".join(t["name"] for t in selected))
shell: python3

diagnose:
needs: resolve
if: needs.resolve.outputs.matrix != '{"target":[]}'
runs-on: ubuntu-latest
timeout-minutes: 8
strategy:
matrix: ${{ fromJson(needs.resolve.outputs.matrix) }}
fail-fast: false
permissions:
contents: read
id-token: write
env:
GCE_USER: ${{ vars.IB_GATEWAY_GCE_USER }}
GCE_INSTANCE_NAME: ${{ vars.IB_GATEWAY_INSTANCE_NAME }}
GCE_ZONE: ${{ vars.IB_GATEWAY_ZONE }}
DEPLOY_PATH: ${{ vars.IB_GATEWAY_DEPLOY_PATH }}
SSH_PRIVATE_KEY_SECRET_NAME: ${{ vars.IB_GATEWAY_SSH_PRIVATE_KEY_SECRET_NAME }}
TARGET_NAME: ${{ matrix.target.name }}
GCP_PROJECT_ID: ${{ matrix.target.gcp_project_id }}
GCP_WORKLOAD_IDENTITY_PROVIDER: ${{ matrix.target.gcp_workload_identity_provider }}
GCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNT: ${{ matrix.target.gcp_workload_identity_service_account }}
GCE_USER: ${{ matrix.target.gce_user }}
GCE_INSTANCE_NAME: ${{ matrix.target.gce_instance_name }}
GCE_ZONE: ${{ matrix.target.gce_zone }}
DEPLOY_PATH: ${{ matrix.target.deploy_path }}
SSH_PRIVATE_KEY_SECRET_NAME: ${{ matrix.target.ssh_private_key_secret_name }}
Comment on lines +76 to +84

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass target container metadata to diagnostics

The diagnostic job now selects a target, but this env block drops the target's container_name and unit_suffix; the remote script below still checks hard-coded resources such as docker exec ib-gateway and ibkr-2fa-bot.*. For targets deployed with non-default container/unit names, diagnostics will inspect the default gateway or report missing services instead of the selected gateway, so carry these target fields through as main.yml does.

Useful? React with 👍 / 👎.

Comment on lines +83 to +84

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Carry target port settings into diagnostics

The migrated diagnostic env only passes deploy/SSH fields, but multi-gateway targets in main.yml can publish non-default live_host_port, paper_host_port, and vnc_host_port; the remote diagnostics below still greps only 4001|4002|4003|4004|5900. For any selected target using custom host ports, the report falsely shows the gateway ports as absent even though the gateway is listening on its configured ports, so propagate the port fields and build the port checks from them.

Useful? React with 👍 / 👎.

steps:
- name: Authenticate to Google Cloud
id: auth
Expand Down
Loading