unable to parse terraform plan file "Invalid expression; Expected the start of an expression, but found an invalid expression token."

[hloeffler]

Description

trivy config ./tfplan.json --debug
leads to error:

2024-11-28T14:42:26+01:00 ERROR [terraform parser] Error parsing file module="root" file_path="main.tf" err="main.tf:2214,2-3: Invalid expression; Expected the start of an expression, but found an invalid expression token."

Desired Behavior

its just works

Actual Behavior

trivy config ./tfplan.json --debug
leads to error:

2024-11-28T14:42:26+01:00 ERROR [terraform parser] Error parsing file module="root" file_path="main.tf" err="main.tf:2214,2-3: Invalid expression; Expected the start of an expression, but found an invalid expression token."

Reproduction Steps

terraform plan --out tfplan.binary
terraform show -json tfplan.binary > tfplan.json
trivy config ./tfplan.json

Target

None

Scanner

Misconfiguration

Output Format

None

Mode

Standalone

Debug Output

❯ trivy config ./tfplan.json --debug
2024-11-28T14:44:22+01:00	DEBUG	No plugins loaded
2024-11-28T14:44:22+01:00	DEBUG	Default config file "file_path=trivy.yaml" not found, using built in values
2024-11-28T14:44:22+01:00	DEBUG	Cache dir	dir="/home/nilloeffler/.cache/trivy"
2024-11-28T14:44:22+01:00	DEBUG	Cache dir	dir="/home/nilloeffler/.cache/trivy"
2024-11-28T14:44:22+01:00	DEBUG	Parsed severities	severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-11-28T14:44:22+01:00	INFO	[misconfig] Misconfiguration scanning is enabled
2024-11-28T14:44:22+01:00	DEBUG	[misconfig] Checks successfully loaded from disk
2024-11-28T14:44:22+01:00	DEBUG	Enabling misconfiguration scanners	scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-11-28T14:44:22+01:00	DEBUG	Initializing scan cache...	type="memory"
2024-11-28T14:44:23+01:00	DEBUG	[misconfig] Scanning files for misconfigurations...	scanner="Terraform Plan JSON"
2024-11-28T14:44:23+01:00	DEBUG	[tfjson scanner] Scanning file	file_path="tfplan.json"
2024-11-28T14:44:23+01:00	DEBUG	[terraform scanner] Scanning directory	file_path="."
2024-11-28T14:44:23+01:00	DEBUG	[rego] Overriding filesystem for checks
2024-11-28T14:44:23+01:00	DEBUG	[rego] Embedded libraries are loaded	count=15
2024-11-28T14:44:23+01:00	DEBUG	[rego] Embedded checks are loaded	count=509
2024-11-28T14:44:23+01:00	DEBUG	[rego] Checks from disk are loaded	count=524
2024-11-28T14:44:23+01:00	DEBUG	[rego] Overriding filesystem for data
2024-11-28T14:44:23+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-11-28T14:44:23+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-11-28T14:44:23+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-11-28T14:44:26+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:2214,2-3: Invalid expression; Expected the start of an expression, but found an invalid expression token."
2024-11-28T14:44:26+01:00	INFO	[terraform scanner] Scanning root module	file_path="."
2024-11-28T14:44:26+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-11-28T14:44:26+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-11-28T14:44:26+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-11-28T14:44:28+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:2214,2-3: Invalid expression; Expected the start of an expression, but found an invalid expression token."
2024-11-28T14:44:28+01:00	DEBUG	[terraform parser] Loading module	module="root" module="root"
2024-11-28T14:44:28+01:00	INFO	[terraform parser] No files found, nothing to do.	module="root"
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Adapting modules...
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Adapted module(s) into state data.	count=0
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Using max routines	count=11
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Initialized Go check(s).	count=482
2024-11-28T14:44:28+01:00	DEBUG	[rego] Scanning inputs	count=1
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Finished applying rules.
2024-11-28T14:44:28+01:00	DEBUG	[terraform executor] Applying ignores...
2024-11-28T14:44:28+01:00	DEBUG	OS is not detected.
2024-11-28T14:44:28+01:00	INFO	Detected config files	num=1
2024-11-28T14:44:28+01:00	DEBUG	Scanned config file	file_path=""
2024-11-28T14:44:28+01:00	DEBUG	[vex] VEX filtering is disabled

Operating System

Ubuntu 24.04.1 LTS

Version

Version: 0.57.1
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-11-28 06:14:19.821903851 +0000 UTC
  NextUpdate: 2024-11-29 06:14:19.82190345 +0000 UTC
  DownloadedAt: 2024-11-28 12:00:39.497231976 +0000 UTC
Check Bundle:
  Digest: sha256:34fe41b4f92a89202ffe7f94c158884fe633a45751706735ebadce7a96ec7dec
  DownloadedAt: 2024-11-28 11:51:59.029461888 +0000 UTC

and also
❯ docker run -it ghcr.io/aquasecurity/trivy:canary  --version         
Version: 0.57.0-SNAPSHOT-06279924c

Checklist

• Run trivy clean --all
• Read the troubleshooting

[hloeffler]

with the ghcr.io/aquasecurity/trivy:canary i get better debug output but also not helping:
2024-11-28T13:49:26Z ERROR [terraform parser] Error parsing file module="root" file_path="main.tf" cause="\t.dockerconfigjson = \"{\\\"auths\\\":{\\\"europe-west3-docker.pkg.dev/XXX\\\":{\\\"auth\\\":\\\"XXX==\\\"}}}\"" err="main.tf:2214,2-3: Invalid expression; Expected the start of an expression, but found an invalid expression token."

[hloeffler]

btw same error with just
trivy config .
in the same workspace
(other workspaces are working fine)

[nikpivkin]

Hi @hloeffler !

Based on the error, do you seem to be using templates? Can you share a plan or original hcl config to investigate the issue?

[hloeffler]

sadly no,
i try to get it reproduce without leaking data

is there a way to get more debug output than "-debug" ?

[nikpivkin]

No. Are you using kubernetes_secret? You can just replace username/password/email with dummy data and share only one resource.

[aziotix]

I've faced with a similar error when converting Terraform (Azure infra) output to JSON format:
ERROR [terraform parser] Error parsing file module="root" file_path="main.tf" cause="\thidden-link: /app-insights-resource-id =\"/subscriptions/..."" err="main.tf:404,15-16: Invalid expression; Expected the start of an expression, but found an invalid expression token.

In my case the file included the lines Trivy could not parse correctly:

                        "tags": {
                            "environment": "...",
                            "hidden-link: /app-insights-resource-id": "/subscriptions/...",
                            "managed_by": "...",
                            "owner": "...",
                            "project": "..."
                        }

I used this workaround to clean JSON file (just remove the line "hidden-link: /app-insights-resource-id": "/subscriptions/...",):

      jq 'walk(
        if type == "object" then
          with_entries(select(.key | test("[/:]") | not))
        else
          .
        end
      )' tfplan.json > tfplan-cleaned.json

Then I was able to run the command with no issues: trivy config tfplan-cleaned.json.

[simar7]

cc @nikpivkin could you take another look at this?

[nikpivkin]

In some cases, we parse JSON objects as blocks, so the key hidden-link: /app-insights-resource-id results in a syntactically invalid child block. I am exploring whether we can use the configuration section in the Terraform plan to determine the schema and correctly restore blocks and attributes.

[nikpivkin]

Track #9635