(feat): adds publish support for aws secrets manager and parameter store #1953
Conversation
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
…ive destination validation - Add comprehensive destination conflict validation tests for all 5 destinations (S3, GCS, Vault, AWS Secrets Manager, AWS Parameter Store) in config_test.go - Add keyvalue-secrets.yaml example showing optimal format for AWS console key/value editor - Add aws-secrets-manager-keyvalue-format.md documentation explaining JSON vs key/value formats - Add destination-test-coverage.md documenting complete test matrix (10 conflict scenarios) This ensures proper validation of destination conflicts and provides clear guidance for using AWS Secrets Manager key/value format to enable the AWS console editor. Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
…tinations - Add test to verify Upload method returns NotImplementedError - Replace Parameter Store Upload implementation with NotImplementedError - Ensure consistency with Vault and Secrets Manager destinations - The publish command uses UploadUnencrypted for all structured destinations This removes unreachable code and aligns with the current design where: - S3/GCS: Upload encrypted files (Upload method) - Vault/Secrets Manager/Parameter Store: Upload decrypted JSON (UploadUnencrypted method) Test-driven fix: wrote failing test, then implemented the fix. Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
…avior - Update TestAWSParameterStoreDestination_EncryptedFile_Integration to test NotImplementedError - Remove test of legacy Upload functionality that's not used by publish command - Verify Upload method now returns NotImplementedError consistently with other destinations - All integration tests pass with real AWS credentials The publish command uses UploadUnencrypted for Parameter Store, so Upload method should return NotImplementedError like Vault and Secrets Manager destinations. Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
bruce-szalwinski-he
force-pushed
the
feature/aws-publishing-support
branch
from
094d6c9 to
292c9b0
Compare
| @@ -0,0 +1,123 @@ | |||
| package config | |||
There was a problem hiding this comment.
config_test was getting a little long, so added aws config tests as separate file.
Signed-off-by: bruce-szalwinski-he <[email protected]>
| parameterType = "SecureString" | ||
| } | ||
|
|
||
| // Ensure parameter path starts with / |
There was a problem hiding this comment.
https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-hierarchies.html - parameter hierarchies require leading /. parameter hierarchies was introduced in 2017, https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-ec2-systems-manager-adds-hierarchy-tagging-and-notification-support-for-parameter-store/
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
bruce-szalwinski-he
force-pushed
the
feature/aws-publishing-support
branch
from
ca8cf9f to
9427fab
Compare
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
|
@felixfontein I see that you have been busy preparing the 3.11.0 release, so may not have had time to look at this one. Just checking in to see if there is anything else I need to provide for this PR. |
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
Signed-off-by: bruce-szalwinski-he <[email protected]>
sabre1041
left a comment
sabre1041
left a comment
There was a problem hiding this comment.
Confirmed the functionality of both the AWS Secrets Manager and AWS Parameter Store
A question that this integration has raised is how to handle the AWS region. The existing S3 integration does not include a dedicated property and instead relies on the default properties that the golang AWS library leverages.
Why not use an aws_region property to each of the AWS related publishers instead of specific parameters. This simplifies the end user experience, but also adds functionality for the S3 publisher. If it is not included, default to the logic of the AWS library
| There are a few settings for AWS Secrets Manager that you can place in your destination rules: | ||
| * ``aws_secrets_manager_secret_name`` - The name of the secret in AWS Secrets Manager. If not specified, the filename will be used as the secret name. | ||
| * ``aws_region`` - The AWS region where the secret should be stored. This is required. |
There was a problem hiding this comment.
This should align to aws_secrets_manager_region
| There are a few settings for AWS Parameter Store that you can place in your destination rules: | ||
| * ``aws_parameter_store_path`` - The parameter path in AWS Parameter Store. If it ends with ``/``, the filename will be appended. If not specified, the filename will be used as the parameter name with a leading ``/``. | ||
| * ``aws_region`` - The AWS region where the parameter should be stored. This is required. |
There was a problem hiding this comment.
This should align to aws_parameter_store_region
2 participants
Adds AWS Secrets Manager and Parameter Store
publishsupport. Works same asvaultin that it uploads unencrypted results as plain text JSON.fixes #1942, #1105