Skip to content

fix(secretstores.googlecloud): Handle public GCP service account keys correctly#18785

Open
crflanigan wants to merge 7 commits intoinfluxdata:masterfrom
crflanigan:serviceaccount
Open

fix(secretstores.googlecloud): Handle public GCP service account keys correctly#18785
crflanigan wants to merge 7 commits intoinfluxdata:masterfrom
crflanigan:serviceaccount

Conversation

@crflanigan
Copy link
Copy Markdown
Contributor

@crflanigan crflanigan commented Apr 23, 2026

Summary

When using a normal GCP service-account key (the JSON downloaded from the Cloud Console, type: "service_account") with the new secret store

  • token = "@{id:token}" pattern in outputs.stackdriver, the Google auth library previously received an unscoped token that was rejected by the Monitoring API.

This change adds a tiny guarded default in Init():

  • If credType == "service_account" and no scopes are explicitly set, automatically use ["https://www.googleapis.com/auth/monitoring"].

The existing GDCH/STS flow (via sts_audience) is completely untouched. The new credential_scopes config option is also available for users who need a different set (e.g. cloud-platform).

No changes to stackdriver.go or any other plugin — keeps the PR as small and low-risk as possible

Checklist

Related issues

resolves #16326

CFK5A0 added 2 commits April 18, 2026 12:50
…standard service-account JSON keys

When using a normal GCP service-account key (the JSON downloaded from
the Cloud Console, type: "service_account") with the new secret store
+ token = "@{id:token}" pattern in outputs.stackdriver, the Google auth
library previously received an unscoped token that was rejected by the
Monitoring API.

This change adds a tiny guarded default in Init():

- If credType == "service_account" and no scopes are explicitly set,
  automatically use ["https://www.googleapis.com/auth/monitoring"].

The existing GDCH/STS flow (via sts_audience) is completely untouched.
The new `scopes` config option is also available for users who need a
different set (e.g. cloud-platform).

No changes to stackdriver.go or any other plugin — keeps the PR as
small and low-risk as possible
@crflanigan crflanigan changed the title secretstores/googlecloud: default minimal Cloud Monitoring scope for standard service-account JSON keys fix(secretstores/googlecloud): default minimal Cloud Monitoring scope for standard service-account JSON keys Apr 23, 2026
@telegraf-tiger telegraf-tiger Bot added the fix pr to fix corresponding bug label Apr 23, 2026
@srebhan srebhan changed the title fix(secretstores/googlecloud): default minimal Cloud Monitoring scope for standard service-account JSON keys fix(secretstores.googlecloud): Handle public GCP service account keys correctly Apr 27, 2026
Copy link
Copy Markdown
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @crflanigan! Just a small comment... Please run make docs before pushing. :-)

Comment thread plugins/secretstores/googlecloud/sample.conf Outdated
@srebhan srebhan added area/gcp Google Cloud plugins including cloud_pubsub, cloud_pubsub_push, stackdriver plugin/secretstores labels Apr 27, 2026
@srebhan srebhan self-assigned this Apr 27, 2026
Co-authored-by: Sven Rebhan <36194019+srebhan@users.noreply.github.com>
@crflanigan
Copy link
Copy Markdown
Contributor Author

@srebhan, I committed your suggestion :)

As for the make docs, do I need to do that now, or with this committed suggestion (thanks!) are we good to go?

@srebhan
Copy link
Copy Markdown
Member

srebhan commented Apr 28, 2026

@crflanigan you need to do it now and push the changes as CI checks if the README "configuration" section matches the sample.conf... :-)

@crflanigan
Copy link
Copy Markdown
Contributor Author

Alright, all done @srebhan!

Copy link
Copy Markdown
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @crflanigan!

@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Apr 29, 2026
@srebhan srebhan assigned skartikey and unassigned srebhan Apr 29, 2026
Comment thread plugins/secretstores/googlecloud/googlecloud.go Outdated
Comment thread plugins/secretstores/googlecloud/googlecloud.go Outdated
@skartikey
Copy link
Copy Markdown
Contributor

The auto-default to https://www.googleapis.com/auth/monitoring for service_account keys couples this generic secret store to one specific consumer (Stackdriver). The same store can hand tokens to outputs.cloud_pubsub, BigQuery sinks, etc., and those users will hit the same 403 you're fixing here, just with a different surface.

Two cleaner options:

  1. Broaden the default to cloud-platform - https://www.googleapis.com/auth/cloud-platform covers virtually every GCP API and is the standard "works for any service" scope. The Stackdriver flow keeps working, and so do Pub/Sub, BigQuery, etc.
  2. Drop the auto-default entirely and return an error in Init() when credType requires scopes and credential_scopes is empty, with a message pointing the user at the option. This is more friction but is honest about what the library actually needs.

Whichever you pick, please update the sample.conf wording to match - right now the comment says credential_scopes is "Required for public-GCP service-accounts" but the code silently fills it in, so the docs and behavior disagree.

crflanigan and others added 2 commits April 29, 2026 10:49
Co-authored-by: skartikey <s.kartikey@gmail.com>
Co-authored-by: skartikey <s.kartikey@gmail.com>
Copy link
Copy Markdown
Contributor

@skartikey skartikey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crflanigan A couple of comments.
CI: gofmt failure on the new field line is what's failing lint-linux/macos/windows. make fmt will fix it.

Comment thread plugins/secretstores/googlecloud/googlecloud.go Outdated
@telegraf-tiger
Copy link
Copy Markdown
Contributor

telegraf-tiger Bot commented May 1, 2026

@crflanigan
Copy link
Copy Markdown
Contributor Author

All set @skartikey @srebhan

Thanks for the feedback!

@crflanigan crflanigan requested a review from skartikey May 1, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/gcp Google Cloud plugins including cloud_pubsub, cloud_pubsub_push, stackdriver fix pr to fix corresponding bug plugin/secretstores ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for credentials_file Parameter in outputs.stackdriver Plugin

3 participants