Run CI on self-hosted runners#9540
Merged
Martin-Molinero merged 7 commits intoJun 17, 2026
Merged
Conversation
Switch all GitHub Actions workflows from the GitHub-hosted ubuntu-24.04 runners to self-hosted runners. The Benchmarks workflow targets a runner with the dedicated 'benchmark' label so benchmarking only runs on that machine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d-only Skip the 'Liberate disk space' step on self-hosted runners (only run it on github-hosted ones) and mount the benchmark /nas:/Data volume read-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Self-hosted runners are not allowed to run docker, so the docker run/exec (runInContainer) pattern fails. Run every workflow directly inside a job-level container: quantconnect/lean:foundation, matching the benchmarks workflow. Drops the Liberate disk space, docker helper and Start container steps; secrets are passed via the container env. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a concurrency group keyed to workflow + ref with cancel-in-progress so pushing a new commit to a branch cancels the still-running workflow for the previous commit. Applied to all CI/test workflows; rebase-org-branches is left untouched to avoid cancelling a rebase mid-push. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Inside a job-level container GitHub sets HOME=/github/home, so the global tool installs to $HOME/.dotnet/tools, not /root/.dotnet/tools. Use $HOME so dotnet-interactive is found and 'dotnet interactive jupyter install' works. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The container steps run under sh, where bash-only [[ ]] is unavailable. Rewrite the tag/stub-generation gate as a POSIX case + [ ] test so it behaves correctly on tag builds instead of silently being skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add docker options (--cpus 12 --memory 12g) to every workflow container so jobs running on high-core self-hosted hosts can't monopolize resources. Limits are per container, so concurrent jobs each get their own allotment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.QuiverQuant
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Alpaca
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Binance
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Bitfinex
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.ByBit
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Coinbase
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.dYdX
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Kraken
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.OANDA
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.RaiffeisenBankInternational
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.Brokerages.Tastytrade
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.QuiverQuantWallStreetBets
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.QuiverQuantWikipedia
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.Regalytics
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.SEC
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.SEC.FailsToDeliver
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.SmartInsiderIntentionsTransactions
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Martin-Molinero
added a commit
to QuantConnect/Lean.DataSource.Tiingo
that referenced
this pull request
Jun 17, 2026
Mirror QuantConnect/Lean#9540 (commit c57fd18b): run the build/test workflow on self-hosted runners directly inside a job-level quantconnect/lean:foundation container (--cpus 12 --memory 12g) instead of the GitHub-hosted ubuntu runner + addnab/docker-run wrapper. Drops the disk-cleanup step and the docker-run wrapper (build/test now run as a normal step inside the container; secrets via container env), and adds a concurrency group (cancel-in-progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Switches all GitHub Actions workflows from the GitHub-hosted
ubuntu-24.04runners to self-hosted runners.runs-on: self-hosted.benchmarklabel (runs-on: [self-hosted, benchmark]) so benchmarking only runs on that machine, keeping benchmark results consistent and isolated.Workflows updated:
gh-actions.yml(Build & Test Lean)api-tests.ymlregression-tests.ymlresearch-regression-tests.ymlreport-generator.ymlsyntax-tests.ymlvirtual-environments.ymlrebase-org-branches.ymlbenchmarks.yml→[self-hosted, benchmark]Related Issue
N/A
Motivation and Context
Move CI off GitHub-hosted runners onto self-hosted infrastructure, with benchmarks pinned to a dedicated labeled runner.
How Has This Been Tested?
Workflow YAML changes only; will be validated by the runners picking up the jobs.
Types of changes
🤖 Generated with Claude Code