Two jobs in .github/workflows/security-policy.yml discard their own findings:
audit_rust-audit ends its scan step with exit 0 # Don't fail yet, let the summary job decide
audit_rust-deny runs cargo deny check ... || true
The deferral in the first comment does not happen. audit-summary reads
job results (needs.<job>.result), not step outputs, so the
vulnerabilities_found output that audit_rust-audit computes is consulted by
nobody. The job is green whatever cargo audit found, and the summary sees
success.
|| true in the second is the same defect without the explanatory comment.
This is the #844 fault class inverted: #844 was a gate that could not report a
bad result because the rollup never exited non-zero; this is a gate that cannot
report a bad result because the evidence is thrown away before the rollup can
see it.
Acceptance criteria
- Either the job fails on its own findings, or its findings reach a consumer
that fails — and the chosen path is the one the comment describes.
- If outputs are the chosen mechanism,
audit-summary actually reads them,
proven by a mutant: a planted advisory/denied licence turns the summary red.
- Reverting the mutant turns it green (positive control).
- Each job prints a denominator — advisories found, licences checked — so a
zero-finding run is distinguishable from a run that did not look.
- No
|| true and no unconditional exit 0 remains in a scan step in this
workflow.
Found while fixing #844.
Two jobs in
.github/workflows/security-policy.ymldiscard their own findings:audit_rust-auditends its scan step withexit 0 # Don't fail yet, let the summary job decideaudit_rust-denyrunscargo deny check ... || trueThe deferral in the first comment does not happen.
audit-summaryreadsjob results (
needs.<job>.result), not step outputs, so thevulnerabilities_foundoutput thataudit_rust-auditcomputes is consulted bynobody. The job is green whatever
cargo auditfound, and the summary seessuccess.|| truein the second is the same defect without the explanatory comment.This is the #844 fault class inverted: #844 was a gate that could not report a
bad result because the rollup never exited non-zero; this is a gate that cannot
report a bad result because the evidence is thrown away before the rollup can
see it.
Acceptance criteria
that fails — and the chosen path is the one the comment describes.
audit-summaryactually reads them,proven by a mutant: a planted advisory/denied licence turns the summary red.
zero-finding run is distinguishable from a run that did not look.
|| trueand no unconditionalexit 0remains in a scan step in thisworkflow.
Found while fixing #844.