[CI] issue: 4705805 Add secret scan step #1145

ntsemah · 2025-11-05T08:54:46Z

Description

What

Make Secrets Scanner run as a step in CI

Why ?

HPCINFRA-2780

How ?

It is optional but for complex PRs please provide information about the design,
architecture, approach, etc.

Change type

What kind of change does this PR introduce?

Check list

Code follows the style de facto guidelines of this project
Comments have been inserted in hard to understand places
Documentation has been updated (if necessary)
Test has been added (if possible)

greptile-apps

Greptile Overview

Greptile Summary

Adds secret scanning as a new CI pipeline step that runs by default on all pull requests.

Key Changes:

Introduced new secret-scan Docker container (harbor.mellanox.com/toolbox/secret_scan:0.0.23)
Added Secret Scan step in matrix_job.yaml that executes /opt/nvidia/secret_scan.py to scan workspace files
Created do_secretscan parameter in proj_jjb.yaml (enabled by default) to control execution
Secret scan generates secret_scan.html report that gets archived as a build artifact
Uses existing mellanox_github_credentials for authentication
Follows the same pattern as the existing Copyrights check step

Minor improvements:

Cleaned up trailing whitespace in several lines

Confidence Score: 5/5

This PR is safe to merge with minimal risk - it adds a non-blocking security scan step using standard CI patterns
The implementation follows existing patterns (similar to Copyrights step), uses proper credential management, is enabled by default but can be disabled via parameter, and only adds scanning without modifying core functionality
No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
.ci/matrix_job.yaml	5/5	Added secret-scan container and Secret Scan step with proper credentials. Includes minor whitespace cleanup.
.ci/proj_jjb.yaml	5/5	Added `do_secretscan` parameter (default true) to enable/disable secret scanning. Minor whitespace fix.

Sequence Diagram

sequenceDiagram
    participant PR as Pull Request
    participant Jenkins as Jenkins Pipeline
    participant JJB as Job Config
    participant Matrix as Matrix Job
    participant Harbor as Harbor Registry
    participant Scanner as Secret Scanner
    participant Workspace as Source Files
    
    PR->>Jenkins: Trigger CI Build
    Jenkins->>JJB: Load job parameters
    JJB->>Jenkins: Enable secret scan (default: true)
    Jenkins->>Matrix: Execute pipeline steps
    Matrix->>Harbor: Pull secret_scan:0.0.23 image
    Harbor->>Scanner: Container ready
    Matrix->>Scanner: Mount workspace with auth
    Scanner->>Workspace: Scan for secrets
    Workspace->>Scanner: Return file contents
    Scanner->>Scanner: Analyze and generate report
    Scanner->>Jenkins: Save secret_scan.html
    Jenkins->>PR: Report results

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

ntsemah · 2025-11-05T11:45:35Z

link to build:
https://nbuprod.blsm.nvidia.com/swx-media-master/job/devops/job/testing_zone/job/ntsemah/job/secret-LibVMA/1/

.ci/matrix_job.yaml

greptile-apps

Greptile Overview

Greptile Summary

Integrated Nvidia's secret scanning tool into the CI pipeline and migrated infrastructure to new Docker registry.

Added new "Secret Scan" step using nbu-harbor.gtm.nvidia.com/toolbox/secret_scan:0.0.23 container
Configured secret scanning to run by default with do_secretscan: true parameter
Updated all Docker registry URLs from harbor.mellanox.com to nbu-harbor.gtm.nvidia.com
Secret scan step runs early in pipeline (after copyrights check) with GitHub token authentication
Generates HTML report as artifact for review

Confidence Score: 5/5

This PR is safe to merge with minimal risk
The changes are purely CI/infrastructure configuration updates with no impact on application code. The secret scanning addition improves security posture, and the registry migration follows standard infrastructure updates. All changes follow existing patterns in the codebase (similar to the copyrights check step).
No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
.ci/matrix_job.yaml	5/5	Added secret scanning step and migrated all Docker registry URLs from harbor.mellanox.com to nbu-harbor.gtm.nvidia.com
.ci/proj_jjb.yaml	5/5	Added do_secretscan boolean parameter with default true to enable secret scanning in CI pipeline

Sequence Diagram

sequenceDiagram
    participant GH as GitHub PR
    participant CI as CI Pipeline
    participant Container as Secret Scan Container
    participant Registry as nbu-harbor.gtm.nvidia.com
    participant Workspace as Workspace
    
    GH->>CI: Trigger CI Pipeline
    CI->>CI: Check do_secretscan parameter
    alt do_secretscan == true
        CI->>Registry: Pull secret_scan:0.0.23 image
        Registry-->>CI: Return container image
        CI->>Container: Start secret scan container
        CI->>Container: Pass GITHUB_TOKEN credential
        Container->>Workspace: Scan files in $WORKSPACE
        Container->>Container: Run /opt/nvidia/secret_scan.py
        Container->>Workspace: Generate secret_scan.html report
        Container-->>CI: Exit with status code
        CI->>CI: Archive *.html artifacts
    end
    CI->>GH: Report pipeline status

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

ntsemah · 2025-11-09T15:54:10Z

link to build:
https://nbuprod.blsm.nvidia.com/swx-media-master/job/devops/job/testing_zone/job/ntsemah/job/secret-LibVMA/2/

NirWolfer · 2025-11-10T08:45:12Z

bot:retest

NirWolfer · 2025-11-10T15:12:55Z

bot:retest

ntsemah · 2025-11-12T09:02:07Z

bot:retest

ntsemah · 2025-11-12T13:11:39Z

bot:retest

ntsemah · 2025-11-13T07:23:25Z

bot:retest

ntsemah · 2025-11-16T15:59:39Z

bot:retest

greptile-apps · 2025-11-16T16:04:23Z

Greptile Summary

Adds secret scanning step to CI pipeline using toolbox/secret_scan:0.0.27 container, controlled by do_secretscan parameter (default: true)
Migrates Docker registry references from harbor.mellanox.com to nbu-harbor.gtm.nvidia.com using ${registry_host} variable for consistency

Confidence Score: 5/5

This PR is safe to merge with minimal risk
The changes are straightforward CI configuration updates that add a security scanning feature and standardize registry references. The secret scan step follows the existing pattern used by other steps (Copyrights), and the registry migration properly uses variable substitution for maintainability.
No files require special attention

Sequence Diagram

sequenceDiagram
    participant CI as "CI Pipeline"
    participant Container as "secret-scan Container"
    participant Script as "/opt/nvidia/secret_scan.py"
    participant Workspace as "Repository Code"
    
    CI->>CI: "Check do_secretscan parameter"
    CI->>Container: "Launch toolbox/secret_scan:0.0.27"
    Container->>Script: "Execute with GITHUB_TOKEN"
    Script->>Workspace: "Scan $WORKSPACE for secrets"
    Script->>Script: "Generate secret_scan.html report"
    Script->>CI: "Return scan results"
    CI->>CI: "Archive *.html artifacts"

greptile-apps

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

ntsemah · 2025-11-16T16:58:04Z

bot:retest

ntsemah · 2025-11-17T08:35:06Z

link to build:
https://nbuprod.blsm.nvidia.com/swx-media-master/job/devops/job/testing_zone/job/ntsemah/job/secret-LibVMA/6/

Make Secrets Scanner run as a step in CI Signed-off-by: Noam Tsemah <[email protected]>

greptile-apps

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}
_{React with 👍 or 👎 to share your feedback on this new summary format}

ntsemah · 2025-11-17T11:14:16Z

bot:retest

greptile-apps bot reviewed Nov 5, 2025

View reviewed changes

NirWolfer suggested changes Nov 6, 2025

View reviewed changes

.ci/matrix_job.yaml Outdated Show resolved Hide resolved

.ci/matrix_job.yaml Outdated Show resolved Hide resolved

ntsemah force-pushed the HPCINFRA-2780 branch from 648b0ed to f1867eb Compare November 9, 2025 14:22

greptile-apps bot reviewed Nov 9, 2025

View reviewed changes

ntsemah force-pushed the HPCINFRA-2780 branch from f1867eb to 2585928 Compare November 16, 2025 16:02

greptile-apps bot reviewed Nov 16, 2025

View reviewed changes

ntsemah force-pushed the HPCINFRA-2780 branch from 2585928 to 9b298db Compare November 16, 2025 16:07

greptile-apps bot reviewed Nov 16, 2025

View reviewed changes

[CI] issue: 4705805 Add secret scan step

ad97ef9

Make Secrets Scanner run as a step in CI Signed-off-by: Noam Tsemah <[email protected]>

ntsemah force-pushed the HPCINFRA-2780 branch from 9b298db to ad97ef9 Compare November 17, 2025 09:16

greptile-apps bot reviewed Nov 17, 2025

View reviewed changes

NirWolfer approved these changes Nov 17, 2025

View reviewed changes

dpressle approved these changes Nov 17, 2025

View reviewed changes

[CI] issue: 4705805 Add secret scan step #1145

Are you sure you want to change the base?

[CI] issue: 4705805 Add secret scan step #1145

Uh oh!

Conversation

ntsemah commented Nov 5, 2025

Description

What

Why ?

How ?

Change type

Check list

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

ntsemah commented Nov 5, 2025

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

ntsemah commented Nov 9, 2025

Uh oh!

NirWolfer commented Nov 10, 2025

Uh oh!

NirWolfer commented Nov 10, 2025

Uh oh!

ntsemah commented Nov 12, 2025

Uh oh!

ntsemah commented Nov 12, 2025

Uh oh!

ntsemah commented Nov 13, 2025

Uh oh!

ntsemah commented Nov 16, 2025

Uh oh!

greptile-apps bot commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

ntsemah commented Nov 16, 2025

Uh oh!

ntsemah commented Nov 17, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

ntsemah commented Nov 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

greptile-apps bot commented Nov 16, 2025 •

edited

Loading