Skip to content

audit_rust-audit and audit_rust-deny discard their own findings (exit 0 / || true) #849

Description

@hyperpolymath

Two jobs in .github/workflows/security-policy.yml discard their own findings:

  • audit_rust-audit ends its scan step with exit 0 # Don't fail yet, let the summary job decide
  • audit_rust-deny runs cargo deny check ... || true

The deferral in the first comment does not happen. audit-summary reads
job results (needs.<job>.result), not step outputs, so the
vulnerabilities_found output that audit_rust-audit computes is consulted by
nobody. The job is green whatever cargo audit found, and the summary sees
success.

|| true in the second is the same defect without the explanatory comment.

This is the #844 fault class inverted: #844 was a gate that could not report a
bad result because the rollup never exited non-zero; this is a gate that cannot
report a bad result because the evidence is thrown away before the rollup can
see it.

Acceptance criteria

  1. Either the job fails on its own findings, or its findings reach a consumer
    that fails — and the chosen path is the one the comment describes.
  2. If outputs are the chosen mechanism, audit-summary actually reads them,
    proven by a mutant: a planted advisory/denied licence turns the summary red.
  3. Reverting the mutant turns it green (positive control).
  4. Each job prints a denominator — advisories found, licences checked — so a
    zero-finding run is distinguishable from a run that did not look.
  5. No || true and no unconditional exit 0 remains in a scan step in this
    workflow.

Found while fixing #844.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions