Skip to content

fix(pypi): encode extras names fully, and mark the remaining raw sites safe - #1356

Merged
mstykow merged 1 commit into
mainfrom
fix/encode-pypi-extras-purls
Aug 11, 2026
Merged

fix(pypi): encode extras names fully, and mark the remaining raw sites safe#1356
mstykow merged 1 commit into
mainfrom
fix/encode-pypi-extras-purls

Conversation

@mstykow

@mstykow mstykow commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • The extras path (requests[socks]) assembled its PURL by hand so the bracket escapes could be lowercase, matching what ScanCode emits. That left every other character raw, so a lockfile name carrying both a bracket and a reserved character kept the latter unencoded. It now encodes through the crate and lowercases only the two bracket escapes, so the emitted spelling is unchanged and the gap is closed.
  • Completes the sweep. Every format!("pkg:…") that remains is safe, and now says why — the reason this class of bug survived is that every such site looked alike under grep.

Scope and exclusions

The remaining sites, each now carrying its justification:

  • pypi builders restrict the name to the PEP 508 charset before formatting — a charset that needs no percent-encoding — and encode the version separately. Verified: 1.0+cu118 and 4.%2A both parse and decode to the same components either way.
  • CocoaPods make_base_purl/make_base_purl_from_parts are internal map keys, never emitted. Emitted PURLs come from create_cocoapods_purl, which encodes through the crate.
  • Go's namespace splice was already documented: it preserves path-part case, which normalize_purl then lowercases host-only per purl-spec#308.
  • One site in src/output/cyclonedx.rs is inside mod tests.

How to verify

rg 'format!\("pkg:' src/ --glob '!*test*'

Every hit should now be adjacent to a comment saying why it is not an emission site or why the components need no encoding. Behaviourally, a uv.lock/poetry.lock entry named foo bar[extra] previously emitted an unencoded space; it now encodes, while requests[socks] is byte-identical to before.

Intentional differences from Python

  • The lowercase bracket escapes (%5b/%5d rather than the crate's %5B/%5D) are kept deliberately for ScanCode parity. Both decode identically.

Expected-output fixture changes

  • None; all 324 parser goldens, 44 assembly goldens, the PURL validity guard and the full 5,676-test lib suite pass unchanged.

…s safe

The extras path (`requests[socks]`) assembled its PURL by hand so the bracket
escapes could be lowercase, matching what ScanCode emits. That left every *other*
character raw, so a lockfile name carrying both a bracket and a reserved
character kept the latter unencoded. Encode through the crate and lowercase only
the two bracket escapes, which keeps the emitted spelling identical while closing
the gap.

The `format!("pkg:` sites that remain are all safe, and now say why — the whole
reason this class of bug survived is that every such site looked alike under
grep:

- The pypi builders restrict the name to the PEP 508 charset before formatting,
  which needs no percent-encoding, and encode the version separately.
- The CocoaPods ones are internal map keys that are never emitted; the emitted
  PURL comes from `create_cocoapods_purl`, which encodes through the crate.
- Go's namespace splice is already documented: it preserves path-part case, which
  `normalize_purl` then lowercases host-only per purl-spec#308.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

This PR corrects PyPI PURL construction for package names containing extras while documenting why remaining manual PURL construction sites are safe.

  • Routes Poetry and uv extras-bearing names through the shared PURL encoder, retaining lowercase bracket escapes for ScanCode parity.
  • Documents the encoding assumptions around Python, requirements.txt, and CocoaPods PURL helpers.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness, security, or repository-rule violations identified.

The changed Poetry and uv paths now encode complete PURL components through the shared helper while deliberately preserving the established bracket-escape spelling, and the remaining changes only document existing safety invariants.

Important Files Changed

Filename Overview
src/parsers/poetry_lock.rs Replaces partial manual encoding of extras-bearing PyPI names with shared PURL construction while preserving lowercase bracket escapes.
src/parsers/uv_lock.rs Applies the same complete encoding path to uv lockfile dependency names containing extras.
src/parsers/podfile_lock.rs Documents that the unencoded CocoaPods base PURL string is an internal map key rather than emitted output.
src/parsers/python/utils.rs Documents the validation and version-encoding guarantees that make local PyPI PURL assembly safe.
src/parsers/requirements_txt.rs Documents why validated distribution names and separately encoded versions can be assembled safely.

Reviews (1): Last reviewed commit: "fix(pypi): encode extras names fully, an..." | Re-trigger Greptile

@mstykow
mstykow merged commit 96f452d into main Aug 11, 2026
13 checks passed
@mstykow
mstykow deleted the fix/encode-pypi-extras-purls branch August 11, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant