[validation] Invalid externalReferences URLs may be present in the SBOM

[marob]

With DependencyTrack 4.11 validating SBOM with schema validation at upload, I've discovered that some SBOM may have invalid externalReferences.
For example:

"externalReferences": [
  {
    "type": "vcs",
    "url": "git@gitlab.com:behat-chrome/chrome-mink-driver.git"
  }
],

coming from composer dmore/chrome-mink-driver package.

Indeed, an externalReference should be a iri-reference or a #/definitions/bomLink according to the JsonSchema.

The problem is that git@gitlab.com:behat-chrome/chrome-mink-driver.git is neither a #/definitions/bomLink nor an iri-reference (that should be of the form scheme://... according to the RFC).

If we can "convert" git@gitlab.com:behat-chrome/chrome-mink-driver.git to https://gitlab.com/behat-chrome/chrome-mink-driver.git it would be nice, but if not, we'd better drop the reference than write an invalid one.

[fabian-zeindl-oebb]

This is not the only issues.
I have seen BOMs that contain whitespace around the URL inside the reference, like

"externalReferences": [
  {
    "type": "vcs",
    "url": "     git@gitlab.com:behat-chrome/chrome-mink-driver.git      "
  }
]

Or failed replacements like

"externalReferences": [
  {
    "type": "vcs",
    "url": "${repository.url}"
  }
]

[prabhu]

@marob, I am in favor of dropping the invalid external references and trimming the value. My personal opinion is that CycloneDX spec should be unopinionated and should not have such validations for external references.

Pull requests are welcome. If not, we can collect all such tickets and raise funds to hire an engineer.

[prabhu]

Add some validations and trimming here, here, and here.

Enable strict validation here

[setchy]

I think we may need to trim the url, and then ideally use the same regex DTrack is using to validate [IRI RFC 3987](https://datatracker.ietf.org/doc/html/rfc3987

We've had instances like the following being rejected

• url: UNKNOWN
• url: git@github.com:follow-redirects/follow-redirects.git

{
            "publisher": "UNKNOWN",
            "group": "",
            "name": "yum-metadata-parser",
            "version": "1.1.4",
            "description": "A fast YUM meta-data parser",
            "purl": "pkg:pypi/yum-metadata-parser@1.1.4",
            "externalReferences": [
                {
                    "type": "website",
                    "url": "UNKNOWN"
                }
            ],
   }
   
   {
   "author": "Ruben Verborgh <ruben@verborgh.org> (https://ruben.verborgh.org/)",
            "group": "",
            "name": "follow-redirects",
            "version": "1.15.6",
            "description": "HTTP and HTTPS modules that follow redirects.",
            "licenses": [
                {
                    "license": {
                        "id": "MIT",
                        "url": "https://opensource.org/licenses/MIT"
                    }
                }
            ],
            "purl": "pkg:npm/follow-redirects@1.15.6",
            "externalReferences": [
                {
                    "type": "vcs",
                    "url": "https://github.com/follow-redirects/follow-redirects"
                },
                {
                    "type": "vcs",
                    "url": "git@github.com:follow-redirects/follow-redirects.git"
                }
            ],
}

[setchy]

Started a discussion DependencyTrack/dependency-track#3775 to confirm.

[prabhu]

@setchy and all, @timmyteo has kindly volunteered to look into this. Please work with him to get this tested and released.

[setchy]

DependencyTrack/dependency-track#3775 (reply in thread)

Use https://www.npmjs.com/package/hosted-git-info to get the git urls fixed.
This worked for other JavaScript-based CycloneDX generators.

[timmyteo]

@setchy In your comment here, you mention that you saw the following in a generated BOM from cdxgen? git@github.com:follow-redirects/follow-redirects.git

I ran cdxgen against the codebase for follow-redirects but did not see that value in the output BOM. Can you please let me know what you were running cdxgen against so I can use the same in testing?

[marob]

I've started to work on it by enabling strict JSON Schema validation on this PR: #1128