Skip to content

fix(python): validate dependency names and drop unreachable PURL fallbacks - #1346

Merged
mstykow merged 3 commits into
mainfrom
investigate/purl-unknowns
Aug 11, 2026
Merged

fix(python): validate dependency names and drop unreachable PURL fallbacks#1346
mstykow merged 3 commits into
mainfrom
investigate/purl-unknowns

Conversation

@mstykow

@mstykow mstykow commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Several parsers guarded PURL construction with PackageUrl::new(type, name).ok() and fell back to format! on failure. That call validates the type — always a compile-time constant at these sites — and never the name, so the guard could not fire and the fallback was unreachable.
  • The same no-op guard was the only name check in the Python parsers, and setup.cfg takes names from free-form text: install_requires = a/b==2.0 emitted pkg:pypi/a/b@2.0, which no PURL parser accepts because pypi prohibits a namespace. Replacing the dead gate with a real one is the point of this change; removing the dead fallbacks is the tidy-up that goes with it.

Scope and exclusions

  • Included: PEP 508 distribution-name validation in build_python_dependency_purl; removal of unreachable format! fallbacks in conan, uv_lock, poetry_lock, pylock_toml, python/utils; removal of the sibling dead if …with_version(v).is_err() { return None } branches.
  • Explicit exclusions: pnpm_lock's equivalent fallback is also unreachable but is left alone — removing it cleanly means also removing three dead .ok()? exits inside the shared npm_purl, which yarn_lock and npm_lock use too.
  • No .expect() was introduced. The pypi builders keep an Option return, so an impossible arm degrades to no PURL rather than an unencoded one, and no new panic path enters a scanner that must survive hostile input.

How to verify

printf '[metadata]\nname = demo\n[options]\ninstall_requires =\n    a/b==2.0\n    requests>=2.0\n' > /tmp/p/setup.cfg
provenant --package --json-pp - /tmp/p | jq '[.files[].package_data[].dependencies[] | {purl, extracted_requirement}]'

Before: pkg:pypi/a/b@2.0. After: purl: null with extracted_requirement: "a/b==2.0" intact, and requests unaffected.

For the removals, the useful check is that nothing changed: scanning a fixture set covering pyproject.toml, setup.cfg, poetry.lock, uv.lock, pylock.toml and conanfile.txt is byte-identical before and after, once UIDs are normalised.

Intentional differences from Python

  • None. ScanCode omits the purl when it cannot build one and keeps the raw text in extracted_requirement; this converges on that.

Follow-up work

  • Created or intentionally deferred, all verified with fixtures while auditing these paths and none fixed here:
    • opam emits pkg:opam/conf gmp and pkg:opam/ocaml/evil into dependencies[].purl — the latter silently reinterpreted as namespace + name.
    • conan's no-version branch emits pkg:conan/my pkg for a ranged reference.
    • swift show-dependencies emits unencoded names for spaces in package names.
    • nuget CPM is not an invalid-PURL site but has a functional bug: assembly builds pkg:nuget/My Package with format! while the csproj parser builds pkg:nuget/My%20Package through the crate, and the PURL is a join key — so a central PackageVersion silently fails to resolve and extracted_requirement comes out null. A control fixture with the space removed resolves correctly.
    • Note for that sweep: PackageUrl::new also mutates names (lowercases for pypi/npm/github/hex/deb/bitbucket, _- for pypi), so swapping format! for the crate at those sites is a behaviour change, not a pure encoding fix.

Expected-output fixture changes

  • None; no fixture exercised an unreachable branch or an invalid name.

@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

The PR validates normalized Python distribution names before constructing dependency PURLs while preserving invalid requirements as PURL-less dependency records. It also removes unreachable PURL fallbacks across Python, Poetry, uv, pylock, and Conan parsing paths.

  • Retains malformed free-form Python requirements through parser and assembly output without emitting invalid PURLs.
  • Adds unit and full scanner/assembly coverage for the PURL-less dependency contract.
  • Simplifies PURL construction where the previous fallback branches could not be reached.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/parsers/python/utils.rs Adds PEP 508 distribution-name validation and propagates optional PURLs without dropping the dependency record.
src/parsers/python/setup_cfg.rs Preserves setup.cfg requirements when PURL construction declines an invalid distribution name.
src/parsers/python/pyproject.rs Retains Poetry and PEP 621 dependencies when PURL generation returns None.
src/parsers/python/scan_test.rs Exercises the real scanner and assembly pipeline and verifies PURL-less dependency visibility and datafile attribution.
src/parsers/python/test.rs Verifies invalid free-form distribution names omit PURLs while valid sibling requirements remain unaffected.
src/parsers/conan.rs Removes an unreachable constructor fallback while preserving name-only PURLs for ranged and bare references.
src/parsers/poetry_lock.rs Simplifies PackageUrl construction and cleanly propagates impossible constructor or version failures.
src/parsers/pylock_toml.rs Removes unreachable manual PURL assembly in favor of the PackageUrl builder.
src/parsers/uv_lock.rs Removes unreachable manual PURL fallback while retaining existing long-name handling.

Reviews (3): Last reviewed commit: "test(python): cover the purl-less setup...." | Re-trigger Greptile

Comment thread src/parsers/python/test.rs
@mstykow
mstykow force-pushed the investigate/purl-unknowns branch from 1dc704c to 43ea9db Compare August 11, 2026 21:09
@mstykow
mstykow changed the base branch from main to fix/hoist-purl-less-dependencies August 11, 2026 21:09
Base automatically changed from fix/hoist-purl-less-dependencies to main August 11, 2026 21:36
mstykow and others added 3 commits August 11, 2026 23:36
`PackageUrl::new` validates only the type, and every one of these call
sites passes a compile-time-constant type (`"conan"`, `PackageType::Pypi`
via `PACKAGE_TYPE.as_str()`), so the `Err` arm cannot be taken.
`PackageUrl::with_version` and `with_namespace` never fail for these
types either -- `with_version` returns `Ok` unconditionally, and none of
the affected types are in the namespace-prohibited list.

The `format!`-based fallbacks behind those arms were therefore dead, and
they built PURLs without percent-encoding, so if they ever had fired they
would have emitted a non-round-tripping PURL.

`build_python_dependency_purl` constructed a `PackageUrl` purely as a
gate and discarded it; it is now infallible and returns `String`.

Verified with a before/after scan over pypi and conan fixtures: output is
byte-identical.

Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
`build_python_dependency_purl` guarded PURL construction with
`PackageUrl::new(...).ok()`, which validates the *type* — a constant here — and
never the name, then assembled the PURL with `format!` anyway. Names arriving
from `setup.cfg` come from free-form text, so `install_requires = a/b==2.0`
emitted `pkg:pypi/a/b@2.0`: a PURL no parser accepts, since pypi prohibits a
namespace.

Replace the gate with the PEP 508 distribution-name check. The dependency is
still reported and `extracted_requirement` still carries the raw text, so
declining the PURL loses nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Greptile review: the name-validation change was covered only at the parser level,
so nothing verified that a dependency emitted with no PURL survives scanner and
assembly processing — which is exactly where a purl-less entry is most likely to
be dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@mstykow
mstykow force-pushed the investigate/purl-unknowns branch from 43ea9db to 7d17798 Compare August 11, 2026 21:36
@mstykow
mstykow merged commit d64804b into main Aug 11, 2026
13 checks passed
@mstykow
mstykow deleted the investigate/purl-unknowns branch August 11, 2026 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant