Add expiry validation for S3 gateway requests #9710
+371
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zubron
force-pushed
the
fix/presigned-urls-ignore-expiration-9599
branch
from
01442fe to
3915765
Compare
zubron
added
the
include-changelog
zubron
force-pushed
the
fix/presigned-urls-ignore-expiration-9599
branch
3 times, most recently
from
77a8138 to
f5fb7ba
Compare
This change adds validation for the `X-Amz-Expires` parameter in presigned URLs. Changes: - Parse `X-Amz-Expires` and validate within allowed range - Check expiration time for presigned URLs and deny request if past expiration time. - Add an explicit check for all required parameters for presigned URLs Fixes #9599
zubron
force-pushed
the
fix/presigned-urls-ignore-expiration-9599
branch
from
f5fb7ba to
375ec38
Compare
This change ensures that when a request doesn't have the v4 algorithm URL parameter (`x-amz-algorithm`), the chained authenticator tries the next auth method instead of failing. Previously, requests using other signature versions would fail because v4 parsing returned and error that blocked the chain.
zubron
force-pushed
the
fix/presigned-urls-ignore-expiration-9599
branch
from
375ec38 to
07ff93a
Compare
N-o-Z
requested changes
Nov 28, 2025
Member
N-o-Z
left a comment
N-o-Z
left a comment
There was a problem hiding this comment.
Looks good!
Added comment inline.
In addition:
- I think it will be nice to have an esti test that checks the e2e flow with an expired presigned URL request (I think this can be done with a very short expiry) but if you think it's too much work for too less of a value let me know
- These changes fix the behavior for the V4 signer but we also have the V2 which although deprecated is still being used in some cases and is still supported in our code. We might want to consult with product regarding that. If we do need to fix this for V2 let's open a separate issue for that so this PR can converge.
| if err == nil { | ||
| c.chosen = method | ||
| return sigContext, nil | ||
| } else if !errors.Is(err, ErrHeaderMalformed) && !errors.Is(err, ErrBadAuthorizationFormat) { |
Member
There was a problem hiding this comment.
Can you please add a comment here explaining why we are singling out these error types
| return expires, nil | ||
| } | ||
|
|
||
| func isV4PresignedRequest(query url.Values) error { |
Member
There was a problem hiding this comment.
Nice taking this out to a separate method
| return ctx, nil | ||
| } | ||
|
|
||
| // otherwise, see if we have all the required query parameters |
Member
There was a problem hiding this comment.
I think it's important to keep the comment.
I'd modify it to: "Otherwise try to parse request as presigned URL
|
|
||
| func (ctx *verificationCtx) verifyExpiration() error { | ||
| if !ctx.AuthValue.IsPresigned { | ||
| // TODO: we currently don't have handling for this |
Member
There was a problem hiding this comment.
Why is that a TODO?
Is there an expiry mechanism for regular requests?
| timeDiff := now.Sub(requestTime) | ||
|
|
||
| // Check for requests from the future and allow small clock skew | ||
| if timeDiff < 0 && timeDiff.Abs() > 5*time.Minute { |
Member
There was a problem hiding this comment.
Why are we checking for 5min?
What's the behavior in AWS for presigned requests that are less than 5 min in the future?
Please document
| "github.com/treeverse/lakefs/pkg/gateway/errors" | ||
| ) | ||
|
|
||
| func TestVerifyExpiration(t *testing.T) { |
Member
There was a problem hiding this comment.
Nice tests - do we want to also add tests for non-presigned with expiry and check our behavior is consistent with S3?
| } | ||
| } | ||
|
|
||
| func TestParseExpires(t *testing.T) { |
Member
There was a problem hiding this comment.
We should test for nil expires as well (no header)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #9599
Change Description
Bug Fix
This change adds validation for the
X-Amz-Expiresparameter in presigned URLs.Changes:
X-Amz-Expiresand validate within allowed rangeTesting Details
Unit tests for the verification logic were added.
Also tested using the python test in this gist
Breaking Change?
This could be considered a breaking change if users were unaware of this issue and have been relying on presigned URLs not expiring. However, this is a security issue and should be fixed.