-
Notifications
You must be signed in to change notification settings - Fork 734
feat: 4184 gguf parser (ai artifact cataloger) part 1 #4279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+4,951
−5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wagoodman
reviewed
Oct 14, 2025
wagoodman
reviewed
Oct 14, 2025
wagoodman
reviewed
Oct 14, 2025
wagoodman
reviewed
Oct 14, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
wagoodman
reviewed
Oct 15, 2025
This comment has been minimized.
This comment has been minimized.
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
wagoodman
reviewed
Oct 22, 2025
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
9c5279c to
b18f7bb
Compare
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
spiffcs
commented
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
spiffcs
commented
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
wagoodman
reviewed
Nov 13, 2025
Comment on lines
30
to
52
| // Read and validate the GGUF file header using LimitedReader to prevent OOM | ||
| // We use LimitedReader to cap reads at maxHeaderSize (50MB) | ||
| limitedReader := &io.LimitedReader{R: reader, N: maxHeaderSize} | ||
| headerData, err := readHeader(limitedReader) | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("failed to read GGUF header: %w", err) | ||
| } | ||
|
|
||
| // Create a temporary file for the library to parse | ||
| // The library requires a file path, so we create a temp file | ||
| tempFile, err := os.CreateTemp("", "syft-gguf-*.gguf") | ||
| if err != nil { | ||
| return nil, nil, fmt.Errorf("failed to create temp file: %w", err) | ||
| } | ||
| tempPath := tempFile.Name() | ||
| defer os.Remove(tempPath) | ||
|
|
||
| // Write the validated header data to temp file | ||
| if _, err := tempFile.Write(headerData); err != nil { | ||
| tempFile.Close() | ||
| return nil, nil, fmt.Errorf("failed to write to temp file: %w", err) | ||
| } | ||
| tempFile.Close() |
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggested change
| // Read and validate the GGUF file header using LimitedReader to prevent OOM | |
| // We use LimitedReader to cap reads at maxHeaderSize (50MB) | |
| limitedReader := &io.LimitedReader{R: reader, N: maxHeaderSize} | |
| headerData, err := readHeader(limitedReader) | |
| if err != nil { | |
| return nil, nil, fmt.Errorf("failed to read GGUF header: %w", err) | |
| } | |
| // Create a temporary file for the library to parse | |
| // The library requires a file path, so we create a temp file | |
| tempFile, err := os.CreateTemp("", "syft-gguf-*.gguf") | |
| if err != nil { | |
| return nil, nil, fmt.Errorf("failed to create temp file: %w", err) | |
| } | |
| tempPath := tempFile.Name() | |
| defer os.Remove(tempPath) | |
| // Write the validated header data to temp file | |
| if _, err := tempFile.Write(headerData); err != nil { | |
| tempFile.Close() | |
| return nil, nil, fmt.Errorf("failed to write to temp file: %w", err) | |
| } | |
| tempFile.Close() | |
| // Create a temporary file for the library to parse | |
| // The library requires a file path, so we create a temp file | |
| tempFile, err := os.CreateTemp("", "syft-gguf-*.gguf") | |
| if err != nil { | |
| return nil, nil, fmt.Errorf("failed to create temp file: %w", err) | |
| } | |
| tempPath := tempFile.Name() | |
| defer os.Remove(tempPath) | |
| // Read and validate the GGUF file header using LimitedReader to prevent OOM | |
| // We use LimitedReader to cap reads at maxHeaderSize (50MB) | |
| limitedReader := &io.LimitedReader{R: reader, N: maxHeaderSize} | |
| err := copyHeader(tempFile, limitedReader) | |
| if err != nil { | |
| return nil, nil, fmt.Errorf("failed to read GGUF header: %w", err) | |
| } | |
| // Write the validated header data to temp file | |
| if _, err := tempFile.Write(headerData); err != nil { | |
| tempFile.Close() | |
| return nil, nil, fmt.Errorf("failed to write to temp file: %w", err) | |
| } | |
| tempFile.Close() |
Now copyHeader only does a quick magic check and a simple copy()
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
wagoodman
approved these changes
Nov 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add new AI artifact cataloger for Syft that detects and parses GGUF (GPT-Generated Unified Format) model files. This includes a proper package type, parser, and metadata structure so Syft can identify and describe .gguf models like any other package in the
artifactssection of the syft sbom.What’s New
ModelPkg Type = "model"Happy to renamespace the cataloger from
aiartifact-> anything that comes up in this reviewIntegration hooks added in package_tasks.go and packagemetadata/names.go. Cataloger will work across both directory and image scans.
Output Support
Syft JSON: includes full gguf-file-metadata with all parsed fields.
CycloneDX 1.6 ML-BOM: emits machine-learning-model components with GGUF metadata encoded as properties.
SPDX: Separate PR
Demo
TODO
Fast Follow
Type of change
Checklist: