Skip to content

Commit 51fe6d2

Browse files
htekdevCopilot
htekdev
and
Copilot
committed
feat: add 3 new error entries (concurrency-timing x1, yaml-syntax x1, known-unsolved x1)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent a206242 commit 51fe6d2

3 files changed

Lines changed: 521 additions & 0 deletions

File tree

‎errors/concurrency-timing/matrix-concurrent-git-push-non-fast-forward.yml‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
id: concurrency-timing-094
2+
title: 'Matrix Strategy Concurrent Jobs All Push to Same Branch — Non-Fast-Forward Rejection'
3+
category: concurrency-timing
4+
severity: error
5+
tags:
6+
- matrix
7+
- git-push
8+
- concurrent
9+
- non-fast-forward
10+
- race-condition
11+
- commit
12+
patterns:
13+
- regex: 'non-fast-forward'
14+
flags: 'i'
15+
- regex: 'Updates were rejected because the tip of your current branch is behind'
16+
flags: 'i'
17+
- regex: 'error: failed to push some refs.+hint: Updates were rejected'
18+
flags: 'is'
19+
error_messages:
20+
- "! [rejected] main -> main (non-fast-forward)"
21+
- "error: failed to push some refs to 'https://github.com/org/repo'"
22+
- "hint: Updates were rejected because the tip of your current branch is behind"
23+
- "hint: its remote counterpart. If you want to integrate the remote changes,"
24+
- "hint: use 'git pull' before pushing again."
25+
root_cause: |
26+
When a `strategy.matrix` job contains a `git commit && git push` step, ALL matrix
27+
instances run concurrently and each one independently checks out the repository at the
28+
SAME commit SHA at job startup.
29+
30+
If two or more instances try to push to the same branch:
31+
- Instance A checks out at commit X, makes changes, pushes → creates commit Y
32+
- Instance B checked out at commit X (before A pushed), makes its own changes, pushes
33+
→ rejected because the branch tip is now Y, not X. Instance B's push is
34+
behind the remote and non-fast-forward.
35+
36+
This fails for ANY workflow that uses a matrix to generate commits — for example:
37+
- Updating generated documentation files per each matrix target
38+
- Writing per-target outputs back to the repository
39+
- Version file updates or artifact manifests shared across matrix entries
40+
41+
The problem is structurally inherent to matrix parallelism: git push requires a
42+
linear history, but multiple concurrent runners checking out the same base commit
43+
and each pushing will create divergent histories. Only the first runner to push wins;
44+
all subsequent runners are rejected.
45+
46+
**Even with `concurrency:` at the workflow level**, this race condition occurs at the
47+
JOB level — concurrent matrix instances within the SAME workflow run share no
48+
synchronisation mechanism around git operations.
49+
50+
Source: Stack Overflow Q#78277579 (score 1, 520 views — Apr 2024) and
51+
Q#79047635 (score 3, 191 views — Oct 2024).
52+
fix: |
53+
**The canonical fix: separate the "matrix work" job from the "commit" job.**
54+
55+
Matrix jobs do their work and write outputs or upload artifacts. A separate downstream
56+
job (`needs: matrix-job`) runs ONCE after all matrix instances complete and performs
57+
a single `git commit && git push`.
58+
59+
This eliminates the race condition entirely — only one job ever touches the repository.
60+
61+
If matrix instances need to write files back, use `actions/upload-artifact` from each
62+
matrix instance, then download all artifacts in the commit job.
63+
64+
**Alternative: apply a `concurrency:` group at the job level** to serialise pushes
65+
(only one matrix instance pushes at a time). Each instance must also `git pull` before
66+
its push to incorporate the previous instance's commit.
67+
fix_code:
68+
- language: yaml
69+
label: 'Broken: matrix instances all push to main concurrently'
70+
code: |
71+
jobs:
72+
build:
73+
runs-on: ubuntu-latest
74+
strategy:
75+
matrix:
76+
target: [a, b, c]
77+
steps:
78+
- uses: actions/checkout@v4
79+
- name: Generate file for ${{ matrix.target }}
80+
run: echo "${{ matrix.target }}" > output/${{ matrix.target }}.txt
81+
- name: Commit and push
82+
run: |
83+
git config user.name github-actions
84+
git config user.email github-actions@github.com
85+
git add .
86+
git commit -m "Add ${{ matrix.target }}"
87+
git push # ❌ Instances B and C fail — A pushed first
88+
- language: yaml
89+
label: 'Fixed: matrix uploads artifacts, a single downstream job commits once'
90+
code: |
91+
jobs:
92+
build:
93+
runs-on: ubuntu-latest
94+
strategy:
95+
matrix:
96+
target: [a, b, c]
97+
steps:
98+
- uses: actions/checkout@v4
99+
- name: Generate file for ${{ matrix.target }}
100+
run: |
101+
mkdir -p output
102+
echo "${{ matrix.target }}" > output/${{ matrix.target }}.txt
103+
- name: Upload artifact
104+
uses: actions/upload-artifact@v4
105+
with:
106+
name: output-${{ matrix.target }}
107+
path: output/${{ matrix.target }}.txt
108+
109+
commit:
110+
runs-on: ubuntu-latest
111+
needs: build # runs AFTER all matrix instances complete
112+
steps:
113+
- uses: actions/checkout@v4
114+
- name: Download all artifacts
115+
uses: actions/download-artifact@v4
116+
with:
117+
path: output/
118+
merge-multiple: true
119+
- name: Commit and push once
120+
run: |
121+
git config user.name github-actions
122+
git config user.email github-actions@github.com
123+
git add output/
124+
git diff --staged --quiet || git commit -m "Add generated outputs"
125+
git push # ✅ Only one runner, no race condition
126+
- language: yaml
127+
label: 'Alternative: serialise pushes with concurrency + pull-before-push'
128+
code: |
129+
jobs:
130+
build:
131+
runs-on: ubuntu-latest
132+
concurrency:
133+
group: git-push-serialised
134+
cancel-in-progress: false # queue, don't cancel
135+
strategy:
136+
matrix:
137+
target: [a, b, c]
138+
steps:
139+
- uses: actions/checkout@v4
140+
- name: Generate file
141+
run: echo "${{ matrix.target }}" > output/${{ matrix.target }}.txt
142+
- name: Pull, commit, push serialised
143+
run: |
144+
git config user.name github-actions
145+
git config user.email github-actions@github.com
146+
git fetch origin main
147+
git reset --hard origin/main # sync with latest main
148+
git add output/
149+
git commit -m "Add ${{ matrix.target }}"
150+
git push # ✅ Serialised by concurrency group
151+
prevention:
152+
- 'Never include `git push` inside a matrix job — use a separate downstream job that runs once after all matrix instances complete.'
153+
- 'Use `actions/upload-artifact` / `actions/download-artifact` to collect matrix outputs; perform the single commit in a `needs:` downstream job.'
154+
- 'If matrix instances MUST commit independently, serialise them with a `concurrency:` group and always `git fetch` + `git reset --hard origin/main` immediately before each push.'
155+
- 'For simple file updates, consider passing the data via job outputs (`$GITHUB_OUTPUT`) to the downstream job instead of writing to the repository mid-matrix.'
156+
docs:
157+
- url: 'https://stackoverflow.com/questions/78277579/how-can-i-commit-and-push-changes-to-a-repository-in-concurrent-github-actions-workflow-runs'
158+
label: 'Stack Overflow — Commit/push in concurrent matrix runs: non-fast-forward rejection (Apr 2024)'
159+
- url: 'https://stackoverflow.com/questions/79047635/running-a-step-after-a-matrix-step-is-done-executing-in-github-actions'
160+
label: 'Stack Overflow — Running a step after matrix is done: commit race with git push (Oct 2024)'
161+
- url: 'https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/using-a-matrix-for-your-jobs'
162+
label: 'GitHub Docs — Using a matrix for your jobs'
163+
- url: 'https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow'
164+
label: 'GitHub Docs — Storing and sharing data from a workflow (artifacts)'

‎errors/known-unsolved/matrix-all-excluded-runs-one-empty-job.yml‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
id: known-unsolved-115
2+
title: 'Dynamic Matrix with All Entries Excluded Still Runs One Empty-Context Job Instance'
3+
category: known-unsolved
4+
severity: limitation
5+
tags:
6+
- matrix
7+
- exclude
8+
- dynamic-matrix
9+
- empty-context
10+
- job-skipping
11+
- known-limitation
12+
patterns:
13+
- regex: 'strategy.*matrix.*exclude.*\$\{\{|exclude.*needs\.\w+\.outputs'
14+
flags: 'i'
15+
- regex: 'matrix\.\w+.*undefined|matrix context.*empty'
16+
flags: 'i'
17+
error_messages:
18+
- "(no error — a single job instance runs with an empty matrix context when all combinations are excluded)"
19+
- "Error accessing ${{ matrix.image }} — matrix context is empty when all combinations were excluded"
20+
root_cause: |
21+
When a job's `strategy.matrix` uses dynamic `exclude:` conditions that evaluate at
22+
runtime and remove ALL generated combinations, GitHub Actions does NOT skip the job.
23+
Instead, it runs exactly ONE job instance with an empty `matrix` context object.
24+
25+
This is a documented behavioral design decision: GitHub Actions requires at least one
26+
completed job instance to fulfill a `needs:` dependency for downstream jobs. When all
27+
matrix entries are excluded, running one empty-context instance ensures downstream jobs
28+
do not get stuck in a "waiting for dependency" state.
29+
30+
**How it appears:**
31+
- The downstream job runs unexpectedly even though no work was intended
32+
- Steps that reference `${{ matrix.image }}` or `${{ matrix.target }}` see empty/undefined values
33+
- The job appears to succeed (exit 0) if no steps fail, but it did nothing useful
34+
- If a step fails when `matrix.image` is empty, the "unnecessary" job produces an error
35+
36+
**Common trigger:** Workflows that use outputs from a path-filter action to dynamically
37+
build `exclude:` lists. When no file paths changed, ALL entries are excluded, but one
38+
empty job still runs.
39+
40+
This is **different** from the `fromJSON([])` error (yaml-syntax-112): that error occurs
41+
when the matrix DIMENSION itself is an empty array. This limitation occurs when the
42+
matrix has valid dimensions but all COMBINATIONS are excluded at runtime.
43+
44+
Source: Stack Overflow Q#78821409 (score 1, 198 views — Aug 2024); GitHub Community
45+
discussions #17832 and #21725; GitHub Actions documented behaviour for matrix exclusion.
46+
fix: |
47+
There is no way to make GitHub Actions skip a matrix job entirely when all exclude
48+
conditions evaluate to true at runtime — the single empty-instance job always runs.
49+
50+
**Workaround 1 (recommended): Add an `if:` condition at the JOB level** that checks
51+
whether any real work needs to be done, based on the upstream job outputs:
52+
53+
```yaml
54+
jobs:
55+
docker:
56+
needs: changes
57+
if: |
58+
needs.changes.outputs.api == 'true' ||
59+
needs.changes.outputs.workers == 'true' ||
60+
needs.changes.outputs.library == 'true'
61+
strategy:
62+
matrix:
63+
image: [api, workers]
64+
```
65+
66+
This prevents the job from running at all when no changes are detected, without
67+
relying on `exclude:` to do the filtering.
68+
69+
**Workaround 2: Move filtering logic inside the job, not into `exclude:`.**
70+
Use a simple matrix (all possible values) and add `if:` conditions per-step or at
71+
the job level to skip steps for targets that should not run:
72+
73+
```yaml
74+
strategy:
75+
matrix:
76+
image: [api, workers, collector]
77+
steps:
78+
- if: |
79+
(matrix.image == 'api' && needs.changes.outputs.api == 'true') ||
80+
(matrix.image == 'workers' && needs.changes.outputs.workers == 'true')
81+
run: docker build -f ${{ matrix.image }}/Dockerfile .
82+
```
83+
84+
**Workaround 3: Use a `fromJSON` matrix that outputs the filtered list** (not
85+
a full matrix with exclusions). The upstream job outputs only the changed targets
86+
as a JSON array, so the matrix never has excluded combinations. If the array is
87+
empty, use a sentinel value (see yaml-syntax-112) to avoid the `fromJSON([])` error.
88+
fix_code:
89+
- language: yaml
90+
label: 'Broken: dynamic exclude runs one empty job when all entries are excluded'
91+
code: |
92+
jobs:
93+
changes:
94+
outputs:
95+
api: ${{ steps.filter.outputs.api }}
96+
workers: ${{ steps.filter.outputs.workers }}
97+
steps:
98+
- uses: dorny/paths-filter@v3
99+
id: filter
100+
with:
101+
filters: |
102+
api: ['api/**']
103+
workers: ['workers/**']
104+
105+
docker:
106+
needs: changes
107+
strategy:
108+
matrix:
109+
image: [api, workers]
110+
isApi:
111+
- ${{ needs.changes.outputs.api == 'true' }}
112+
isWorkers:
113+
- ${{ needs.changes.outputs.workers == 'true' }}
114+
exclude:
115+
- isApi: false
116+
image: api
117+
- isWorkers: false
118+
image: workers
119+
steps:
120+
# ❌ When no files changed: ALL entries excluded but ONE empty job still runs
121+
# matrix.image is undefined — steps may fail or silently produce nothing
122+
- run: docker build ${{ matrix.image }}
123+
- language: yaml
124+
label: 'Fixed: job-level if: prevents the job from running when no changes detected'
125+
code: |
126+
jobs:
127+
changes:
128+
outputs:
129+
api: ${{ steps.filter.outputs.api }}
130+
workers: ${{ steps.filter.outputs.workers }}
131+
steps:
132+
- uses: dorny/paths-filter@v3
133+
id: filter
134+
with:
135+
filters: |
136+
api: ['api/**']
137+
workers: ['workers/**']
138+
139+
docker:
140+
needs: changes
141+
# ✅ Job-level if: prevents execution when nothing changed
142+
if: |
143+
needs.changes.outputs.api == 'true' ||
144+
needs.changes.outputs.workers == 'true'
145+
strategy:
146+
matrix:
147+
image: [api, workers]
148+
steps:
149+
# Use per-step conditions for which images to actually build
150+
- if: |
151+
(matrix.image == 'api' && needs.changes.outputs.api == 'true') ||
152+
(matrix.image == 'workers' && needs.changes.outputs.workers == 'true')
153+
run: docker build ${{ matrix.image }}
154+
prevention:
155+
- 'Use a job-level `if:` condition to prevent a matrix job from running entirely when no work is needed — do not rely solely on `exclude:` to suppress execution.'
156+
- 'Prefer filtering the matrix list in an upstream job (output only the items that need processing) over using complex `exclude:` logic.'
157+
- 'Add a guard step at the start of matrix jobs that exits early with `exit 0` if the matrix context is empty or the specific image/target should not be processed.'
158+
- 'Understand that GitHub Actions will always run at least one job instance from a matrix to fulfil `needs:` dependencies — this is by design, not a bug.'
159+
- 'Test your workflow with no changes (empty path filter results) to verify that the matrix job behaves as expected rather than running with empty context.'
160+
docs:
161+
- url: 'https://stackoverflow.com/questions/78821409/github-actions-matrix-job-running-despite-false-conditions'
162+
label: 'Stack Overflow — Matrix job running despite false conditions when all excluded (Aug 2024, 198 views)'
163+
- url: 'https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/using-a-matrix-for-your-jobs#excluding-matrix-configurations'
164+
label: 'GitHub Docs — Excluding matrix configurations'
165+
- url: 'https://github.com/orgs/community/discussions/17832'
166+
label: 'GitHub Community — Empty matrix still runs one job (known limitation)'

0 commit comments

Comments
 (0)