Add support for package dependency relationships

[wagoodman]

What would you like to be added:
Support tracking the full dependency graph for packages in the form of relationships, for the ecosystems that support extracting this information.

Why is this needed:
An SBOM is useful for at least listing what makes up a software artifact. However, it is more useful to know how a dependency is related to the artifact (is it a direct dependency? or a transitive dependency? is this dependency used by several other packages, or just one?).

Below is a list of each ecosystem that we could implement this for (really it's a list of all of the parsers for all catalogers). It doesn't mean that we should implement this entire list, there are some ecosystems that just don't raise up enough information to make adding relationships useful. This will have to be taken on a case-by-case basis.

• Apk Add support for dependency relationships for alpine (apk) #1063
• Dpkg Add support for dpkg dependency relationships #2040
• ALPM Add relationships for ALPM packages (arch linux) #2851
• Conan (conan.lock) Finalize Conan v2 support #2587
• Conan (conaninfo.txt) feat: add conaninfo.txt parser to detect conan packages in docker images #2234
• Conan (conanfile.txt)
• Dart (pubspec.lock)
• .NET (deps.json) Parse donet dependency trees #2143
• .NET (from binary)
• Elixir (mix.lock)
• Erlang (rebar.lock)
• Github actions workflows (workflows using actions)
• Golang (go.mod) Surface Rust dependency relationships #2353
• Golang (binary) Add relationships for go binary packages #2912
• Haskell (stack.yaml)
• Haskell (stack.yaml.lock)
• Haskell (cabal.project.freeze)
• Java (nested jars)
• Emit relationships for Java dependencies #3189
• Java (gradle.lockfile)
• Javascript (package.json) Syft Extract dependencies from Package.json in JavaScript Package Cataloger #3108
• Javascript (package-lock.json) Syft outputs devDependencies for package-lock.json files #2348 Is generating cyclonedx dependencies supported with the javascript-lock cataloger? #2305 Missing dependency relationships between direct dependencies and transient dependencies in NPM packages #3109
• Javascript (yarn.lock) Is generating cyclonedx dependencies supported with the javascript-lock cataloger? #2305
• Javascript (pnpm-lock.yaml) Is generating cyclonedx dependencies supported with the javascript-lock cataloger? #2305
• Kernel modules Add Linux Kernel cataloger #1694 <-- this PR adds kernel-to-module relationships, but we don't denote dependencies of components within those modules. Is that's what's needed here?
• Lua
• Nix Detect nix dependencies #3814
• OCaml opam
• PHP (installed.json)
• PHP (composer.lock)
• Portage (contents file)
• Python (poetry.lock) Add relationships for python poetry packages #2906
• Python (egg/wheel metadata) Add python wheel egg relationships #2903
• R (description file)
• RPM (db) Add support for RPM DB package relationships #2872
• Ruby (gemfile.lock)
• Ruby (specifications gemspec)
• Rust (cargo.lock) Surface Rust dependency relationships #2353
• Rust (binary) Add relationships for rust audit binary packages #3500
• SBOM
• Swift (package.resolved)
• Swift (Podfile.lock)

These are catalogers that have been deemed not possible / practical to implement raise up relationships for at this time:

• Python (setup.py): no relationship information
• Python (requirements.txt): no relationship information
• Python (Pipfile.lock): no relationship information to the package this is being installed for
• RPM (rpm file): there is relationship data, but not with potentially other installed packages

Notes:
This assumes that #556 is implemented, allowing for package catalogers to return relationships as first class evidence.

[hectorj2f]

@wagoodman I started to play with the package dependencies for cyclonedx https://github.com/hectorj2f/syft/tree/hectorj2f/add_dependencies_to_cyclonedx. I am only generating the dependencies for the components as cyclonedx format recommends. Let me know if you prefer to open a PR for that.

[wagoodman]

Some detail here regarding which ecosystems this will be feasible for in a static-analysis sense (not reaching out to external data sources, such as maven central).

SPDX 2.2 relationships are used to describe what will be added to the artifact package in terms of new relationship types. The used relationships in the breakdown below are:

• RUNTIME_DEPENDENCY_OF
• DEV_DEPENDENCY_OF
• BUILD_DEPENDENCY_OF
• DEPENDENCY_OF

Question: we might not be able to accurately determine build-vs-runtime dependency depending on the lack of context (e.g.
python). Should we just use DEPENDENCY_OF instead in these cases? (final answer: yes)

apk

Summary: direct runtime dependencies

• The D: section lists pull dependencies, which is a space delimited list of package names and categorized dependencies. E.g. D:scanelf so:libc.musl-x86_64.so.1.
• One problem is being able to figure which of the dependencies are package names, and which are other requirements.
  Relationships:
• RUNTIME_DEPENDENCY_OF for any packages listed as pull dependencies
• question: should we make package-to-file relationships for so: dependencies that are found by the resolver?

dpkg

Summary: direct runtime dependencies

• The Depends and Pre-Depends sections hold information about dependencies: "Both depends and pre-depends mention the dependencies a package needs before installing but pre-depends forces the installation and configuration of the dependency packages before even starting with the package that needs the dependencies" source

Relationships:

• RUNTIME_DEPENDENCY_OF for any packages listed as dependencies

golang

go.mod

Summary: flat-subset of transitive build dependencies.

• go.mod contains a subset of transitive dependencies (but possibly more than direct dependencies). Cannot reason about dependency-to-dependency relationships

Relationships:

• BUILD_DEPENDENCY_OF for any package listed in the go.mod
• For go.mod, we currently cannot determine if a dependency is for testing or not.

go binary buildinfo section

Summary: flat-transitive build dependencies.

• go binary buildinfo section contains transitive dependencies. Cannot reason about dependency-to-dependency relationships

Relationships:

• BUILD_DEPENDENCY_OF for any package listed in the binary buildinfo section.

java

pom.xml

Summary: flat-direct build dependencies.

• There is a <dependencies> section which describes direct build dependencies

Relationships:

• BUILD_DEPENDENCY_OF for any package listed the dependency section.

manifest

Does not contain dependency information

javascript

yarn.lock

Summary: flat psuedo-transitive runtime dependency pins

• Each package has a "dependencies" section, which lists only direct dependencies. Dependencies of these dependencies is NOT tracked. Also the dependencies are not pinned versions

Relationships:

• RUNTIME_DEPENDENCY_OF for any packages listed in the yarn.lock
• cannot use the "dependencies" section within each package (since it is not a pin)

package.lock

Summary: transitive dependency pins with full dependency-to-dependency graph

• Under the "dependencies" each pinned version specifies a loose version requirements for any packages that the pinned dependency requires under the "requires" section.
• The requires section always has names that map back to the same name in the "dependencies" section.

Relationships:

• RUNTIME_DEPENDENCY_OF for any packages listed as dependencies
• full dependency graph possible (dependency-to-dependency relationships)

package.json

Summary: flat-direct runtime and dev dependency version ranges.

• The "dependencies" section has a map of name:version values for direct dependencies
• The "devDependencies" section has name:version values for direct dev dependencies
• Note: version values are not pins, but range specifiers

Relationships:

• RUNTIME_DEPENDENCY_OF for any packages listed as dependencies
• DEV_DEPENDENCY_OF for any packages listed in devDependencies

php

composer.lock

Summary: direct dependency version pins

• Direct dependencies only within the "packages" and "packages-dev" sections
• versions are pinned

Relationships:

• RUNTIME_DEPENDENCY_OF for any packages listed as dependencies
• DEV_DEPENDENCY_OF for any packages listed in devDependencies

installed.json

Summary: No relationships possible

• list of installed package versions and their required packages, required packages are only loose version specifiers

Relationships:

• it's not clear that any relationships can be supported.

python

poetry

Summary: flat-transitive dependency dev and runtime relationships

• Lists transitive dependencies in flat fashion (no dependency-to-dependency relationships)
• Each dependency can be categorized (main and dev)
  Relationships:
• Can only describe that the main package relates to packages described within the poetry.lock (not a full transitive dependency graph)
• DEV_DEPENDENCY_OF for packages with a "dev" category
• BUILD_DEPENDENCY_OF for packages with a "main" category (question: should this be RUNTIME_DEPENDENCY_OF since these dependencies are not required in the python compile step for generating pycs?) We cannot really distinguish these in all cases, so it is safer to use DEPENDENCY_OF

pipfile

Summary: flat-transitive dependency dev and runtime relationships

• Lists transitive dependencies in flat fashion (no dependency-to-dependency relationships)
• Each dependency goes underneath one of two json sections: default and develop
  Relationships:
• Can only describe that the default package relates to packages described within the lockfile (not a full transitive dependency graph)
• DEV_DEPENDENCY_OF for packages with a "develop" category
• BUILD_DEPENDENCY_OF for packages with a "default" category (question: should this be RUNTIME_DEPENDENCY_OF since these dependencies are not required in the python compile step for generating pycs?) We cannot really distinguish these in all cases, so it is safer to use DEPENDENCY_OF

egg / dist metadata

Does not describe any relationships

[bureado]

Can you elaborate more on the why for this feature? From my reading of it, what you are trying to do is to determine why a package foo of version bar made it into the thing you are scanning with syft.

If so, I fear dumping the dependency tree might not answer that question. Parsing the package manager operations log can approximate an answer, but the only deterministic way I’m aware of to do this is to perform actual process introspection during the image build to know exactly what ended up calling e.g. dpkg -i over a file on disk.

Conversely, with a purl that is differentiated enough you can augment the syft output with dependencies and much more metadata that is publicly known and available. If syft says nano version 1.2 is in this Ubuntu container of release foo, anyone can readily obtain the dependencies of that package from public sources.

Don’t get me wrong, I’m a fan of taking as much primary source data from the package manager in the scanned instance as possible. And I think flat SBOMs can be limited in many scenarios (log4j being just the latest widely covered scenario) But I think how the feature surfaces, what it tries to solve and how it changes the syft experience for people that are expecting a flat output might be worth additional consideration.

Thank you for working on syft and for helping syft users and the industry realize better outcomes through a thoughtful approach to the existing package manager metadata.

[bureado]

Adding one thing to my comment above. It’s possible that the “why” for this feature is not “what other binary depended on this binary/made this binary materialize” but more of a transitive “what other software was needed to make this binary that then went into my image” and that’s where Build-Depends and Built-Using (in the case of dpkg) would be more useful but the in-artifact package manager metadata might not contain that information. Ideally, that information would carry in each packages own SBOM but in practice the trend seems to be that metadata will live in publicly queriable services. Meaning that maybe this augmentation of syft output could be a post-analyze stage?