image scan error: failed to copy the image: write /tmp/fanal-3917775186: no space left on device

[amarahiqbal]

Description

I get error when I scan GPU images in my pipeline.

FATAL image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to get uncompressed layer sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9: failed to get the layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to populate: unable to open: failed to copy the image: write /tmp/fanal-3917775186: no space left on device

I do not get error for the CPU images.

Desired Behavior

The image scan output.

Actual Behavior

FATAL image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to get uncompressed layer sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9: failed to get the layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to populate: unable to open: failed to copy the image: write /tmp/fanal-3917775186: no space left on device

Reproduction Steps

1.
2.
3.
...

Target

Container Image

Scanner

Vulnerability

Output Format

Table

Mode

Client/Server

Debug Output

FATAL	image scan error: scan error: scan failed: failed analysis: analyze error: pipeline error: failed to analyze layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to get uncompressed layer sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9: failed to get the layer (sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9): unable to populate: unable to open: failed to copy the image: write /tmp/fanal-3917775186: no space left on device

Operating System

Linux

Version

Setting up trivy (0.49.1)

Checklist

• Run trivy image --reset
• Read the troubleshooting

[DmitriyLewen]

Hello @amarahiqbal

Looks like your pipeline uses small size of tmp dir.
Try TMPDIR env - https://aquasecurity.github.io/trivy/v0.49/docs/references/troubleshooting/#running-out-of-space-during-image-scans

Regards, Dmitriy

[anemiroffsumer]

Were you able to fix this ?

[DmitriyLewen]

Hello @amarahiqbal
Sorry for long answer.

Do you have enough free space in pipeline image to decompress image (maybe there is not enough free space in the pipeline image or your image is very large in size)?

[DmitriyLewen]

@anemiroffsumer, can you also answer the question above?

[amarahiqbal]

No. I have not got the fix yet. The error still persists.

I have also added the code for cleaning up the temp files and docker prune after the Trivy scans but that error doesn't seems to go.

[DmitriyLewen]

@amarahiqbal can you check the free space before starting Trivy?
Can you also unzip the scanned image to your local computer and check its size?

[anemiroffsumer]

Having the problem as well. Do we have a fix ?

[SergiuCornea]

I'm also experiencing the issue, is there any fix available?

[knqyf263]

Please try https://aquasecurity.github.io/trivy/v0.53/docs/references/troubleshooting/#running-out-of-space-during-image-scans

[ssensini]

For those still struggling, here's an example that worked without using bigger runner.

...
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/[email protected]
        with:
          image-ref: 'my-app:latest'
          format: 'sarif'
          output: 'trivy-results.sarif'
          trivyignores: '.trivyignore'
          timeout: 60m
        env:
          TMPDIR: '/tmp'
...

The first step executes aquasecurity/[email protected] (the most recent, for now) to scan the my-app:latest image for vulnerabilities, generating a SARIF format report saved as trivy-results.sarif. This step is configured with a 60-minute timeout, since I've got to "context deadline exceeded" multiple times due to heavy load with GPUs, and it uses /tmp as the temporary directory to prevent disk space issues.

Also, it references a .trivyignore file to exclude specific vulnerabilities from the scan. The turning point, in my case, was to redirect the temporary directory somewhere else, and using a .trivyignore file to specify those files I won't need to scan.

Hope this helps!