Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/scancode-triage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# SPDX-FileCopyrightText: Provenant contributors
# SPDX-License-Identifier: Apache-2.0

name: ScanCode weekly triage

# Weekly triage of upstream ScanCode Toolkit issues for relevance to Provenant.
# Uses GitHub Models (free tier) for the judgment and a freshly built binary for
# ground-truth reproduction. Each actionable finding becomes a deduplicated
# Provenant issue; the full run summary lives in this workflow's run output.
on:
schedule:
- cron: "0 8 * * 1" # Mondays 08:00 UTC
workflow_dispatch:
inputs:
days:
description: "Issue-creation window in days"
required: false
default: "8"

# GITHUB_TOKEN needs models:read to call GitHub Models inference and issues:write
# to open per-finding issues.
permissions:
contents: read
models: read
issues: write

concurrency:
group: scancode-triage
cancel-in-progress: false

jobs:
triage:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4

# Build from main rather than downloading a release: a published release
# can lag main, which would falsely flag already-fixed issues as live bugs.
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Build provenant and the triage tool
run: |
cargo build --release --bin provenant
cargo build --release -p provenant-xtask --bin scancode-triage

- name: Run triage
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Passed via env (never interpolated into the run script) so a dispatch
# input cannot break out of the shell command. The tool also validates
# it is a positive integer.
TRIAGE_DAYS: ${{ github.event.inputs.days || '8' }}
# Set the model in repo Settings -> Variables (SCANCODE_TRIAGE_MODEL).
# It is required — the tool errors out if it is unset.
SCANCODE_TRIAGE_MODEL: ${{ vars.SCANCODE_TRIAGE_MODEL }}
run: |
./target/release/scancode-triage \
--days "${TRIAGE_DAYS}" \
--binary target/release/provenant \
--repo-root . \
--post-to "${{ github.repository }}"
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 5 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ glob = "0.3.3"
postcard = { version = "1.1.3", default-features = false, features = ["alloc"] }
rayon = "1.12.0"
regex = "1.12.4"
reqwest = { version = "0.13.3", default-features = false, features = [
"blocking",
"rustls",
] }
rkyv = "0.8.16"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["preserve_order"] }
Expand Down Expand Up @@ -160,10 +164,7 @@ quick-xml = "0.41.0"
rancor = "0.1.1"
rayon = { workspace = true }
regex = { workspace = true }
reqwest = { version = "0.13.3", default-features = false, features = [
"blocking",
"rustls",
] }
reqwest = { workspace = true }
rkyv = { workspace = true, features = ["smallvec-1"] }
rpm = { version = "0.25.0", default-features = false, features = [
"payload",
Expand Down
6 changes: 6 additions & 0 deletions xtask/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ path = "src/bin/generate_legalese_artifact/main.rs"
name = "classify-rule-overmatch"
path = "src/bin/classify_rule_overmatch.rs"

[[bin]]
name = "scancode-triage"
path = "src/bin/scancode-triage.rs"

[features]
# Only the golden-fixture update tools need the provenant `golden-tests` API.
# Scoping it here (instead of enabling it on the dependency crate-wide) lets the
Expand All @@ -82,10 +86,12 @@ postcard = { workspace = true }
provenant = { path = "..", package = "provenant-cli" }
rayon = { workspace = true }
regex = { workspace = true }
reqwest = { workspace = true }
rkyv = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sha2 = { workspace = true }
tempfile = { workspace = true }
url = { workspace = true }
yaml_serde = { workspace = true }
zstd = { workspace = true }
Expand Down
66 changes: 66 additions & 0 deletions xtask/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ cargo run --manifest-path xtask/Cargo.toml --bin <command> -- ...
| `generate-benchmark-chart` | Regenerate the benchmark duration-vs-files SVG from timing rows in `docs/BENCHMARKS.md`. |
| `generate-index-artifact` | Regenerate the embedded license index artifact from ScanCode rules and licenses. |
| `classify-rule-overmatch` | Classify upstream license rules by overmatch-risk class and rank un-covered overlay candidates. |
| `scancode-triage` | Weekly triage of upstream ScanCode issues for Provenant relevance, via GitHub Models with real reproduction scans. |

## `benchmark-target`

Expand Down Expand Up @@ -582,3 +583,68 @@ cargo run --manifest-path xtask/Cargo.toml --bin classify-rule-overmatch
# Machine-readable output for further analysis.
cargo run --manifest-path xtask/Cargo.toml --bin classify-rule-overmatch -- --json
```

## `scancode-triage`

### Purpose

`scancode-triage` checks recent [ScanCode Toolkit](https://github.com/aboutcode-org/scancode-toolkit)
issues for ones that also affect Provenant and are worth fixing. The binary does
the plumbing; a [GitHub Models](https://docs.github.com/github-models) LLM does
the judgment. It is normally run on a weekly schedule by
[`.github/workflows/scancode-triage.yml`](../.github/workflows/scancode-triage.yml),
but is a plain xtask bin you can also run locally.

### Model and token

- Inference runs on GitHub Models via a GitHub token that carries `models:read`
(in CI, the built-in `GITHUB_TOKEN` with `permissions: models: read`; locally,
`gh auth token`).
- The model is **required and has no built-in default**: supply it via `--model`
or the `SCANCODE_TRIAGE_MODEL` env var. In CI it comes from the
`SCANCODE_TRIAGE_MODEL` repository variable (Settings → Variables); the tool
errors out if it is unset. Choose a model whose free-tier limits cover a weekly
run.

### Usage

```bash
cargo run --manifest-path xtask/Cargo.toml --bin scancode-triage -- --help

# Report to stdout over the last N days (requires target/release/provenant built):
SCANCODE_TRIAGE_MODEL=<model-id> cargo run --release --manifest-path xtask/Cargo.toml \
--bin scancode-triage -- --days 8 --binary target/release/provenant

# Cheap check over a few recent candidates:
SCANCODE_TRIAGE_MODEL=<model-id> cargo run --release --manifest-path xtask/Cargo.toml \
--bin scancode-triage -- --days 21 --max-issues 5 --binary target/release/provenant

# Open a deduplicated Provenant issue per actionable finding:
SCANCODE_TRIAGE_MODEL=<model-id> cargo run --release --manifest-path xtask/Cargo.toml \
--bin scancode-triage -- --days 8 --binary target/release/provenant --post-to <owner>/<repo>
```

CLI arguments:

- `--days N`: issue-creation window (must be `>= 1`); ignored when `--since` is given.
- `--since YYYY-MM-DD`: explicit lower bound on issue creation date.
- `--binary PATH`: the provenant binary used for reproduction scans (default `target/release/provenant`).
- `--repo-root PATH`: Provenant repo root, used by the `list_parsers` tool.
- `--model ID`: GitHub Models model id (env `SCANCODE_TRIAGE_MODEL`).
- `--max-issues N`: cap candidates triaged (`0` = no cap).
- `--out PATH`: also write the run summary to a file (it is always printed to stdout).
- `--post-to owner/name`: open a deduplicated issue per actionable finding in that repo.

### What It Does

1. Fetches ScanCode issues created in the window and drops the `New license request:` tickets (license-catalog data, not detection bugs); the count is reported.
2. For each remaining candidate, runs a bounded tool-loop where the model classifies the issue and, via a `run_provenant` tool, reproduces detection bugs with real scans and checks parser coverage via `list_parsers`.
3. Emits a verdict per issue (`reproduces - worth fixing`, `already fixed/better in PV`, `not relevant`, `needs manual check`). The full run summary is printed to stdout (and, under Actions, the job step summary) — not filed as an issue.
4. With `--post-to`, opens one Provenant issue per `reproduces - worth fixing` finding, labeled `scancode-triage` and titled `[ScanCode #N] …`. Findings are deduplicated by that `#N` prefix, so re-runs never open a duplicate for a finding that still reproduces. Stale findings are left untouched — the tool only creates, never edits or closes.

### Notes

- It builds/uses a binary from the current tree rather than a published release: a release can lag `main`, so a release-based run could report issues already fixed on `main` as live bugs.
- Every value the model passes to a tool is treated as untrusted (the model is steered by attacker-controllable issue text): fixture fetches are restricted to GitHub hosts with a size cap and no cross-host redirects, and only read-only detection flags are allowed on the scanner.
- To stay within model input-size and rate limits, issues are triaged one at a time and license-request noise is filtered before any model call.
- Verdicts are model-assisted; treat `needs manual check` as "reproduce by hand."
Loading
Loading