Skip to content

Commit 5bf61d6

Browse files
authored
feat(aws-lambda): support plan protocol v2 (#2789)
* feat(aws-lambda): support plan protocol v2 * fix(aws-lambda): align SAM v2 terminal errors
1 parent c6fdd9c commit 5bf61d6

46 files changed

Lines changed: 4685 additions & 112 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.fallowrc.jsonc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -397,6 +397,12 @@
397397
// require intrusive middleware changes beyond this PR's scope.
398398
"minLines": 6,
399399
"ignore": [
400+
// AWS Lambda and GCP Cloud Run deliberately mirror the same distributed
401+
// rendering lifecycle while retaining provider-specific SDK, storage, and
402+
// retry semantics. The Plan v2 AWS adapter extends that existing symmetry;
403+
// extracting a shared cloud abstraction would couple independent packages.
404+
"packages/aws-lambda/src/handler.ts",
405+
"packages/aws-lambda/src/s3Transport.ts",
400406
// sourcePatcher.ts: pre-existing internal clones between the inline-style
401407
// and attribute tag-patchers; only the PatchOperation type gained two
402408
// optional fields here, but the line shift makes fallow re-flag them.

bun.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

examples/aws-lambda/README.md

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ aws stepfunctions start-execution \
7575
"ProjectS3Uri": "s3://${RENDER_BUCKET}/projects/my-project.tar.gz",
7676
"PlanOutputS3Prefix": "s3://${RENDER_BUCKET}/renders/$(date +%s)/",
7777
"OutputS3Uri": "s3://${RENDER_BUCKET}/output.mp4",
78+
"PlanProtocol": "v1",
7879
"Config": {
7980
"fps": 30,
8081
"width": 1920,
@@ -91,6 +92,9 @@ EOF
9192

9293
The Step Functions execution kicks off Plan, fans out RenderChunk via
9394
the Map state, and finally Assemble. Final mp4 lands at `OutputS3Uri`.
95+
`PlanProtocol` may be `"v1"` or `"v2"`; absent defaults to v1. V2 uses
96+
separate manifest and content-addressed artifact locators throughout the
97+
workflow and never places a v2 object in `PlanS3Uri`.
9498

9599
## Local invocation
96100

@@ -119,13 +123,14 @@ the architecture works on a deployed Lambda — use the local smoke
119123
script:
120124

121125
```bash
122-
# All defaults (mp4-h264-sdr fixture, chunk counts 2/4/8, PSNR >= 40 dB).
126+
# Defaults use the fixture's meta.json minPsnr (30 dB for mp4-h264-sdr).
123127
./scripts/smoke.sh
124128

125129
# Customised:
126130
./scripts/smoke.sh \
127131
--fixture mp4-h264-sdr \
128132
--chunk-counts 2,4,8,16 \
133+
--plan-protocol both \
129134
--psnr-threshold 40 \
130135
--reserved-concurrency 8
131136

@@ -141,7 +146,21 @@ per-run stack name, renders the fixture at each chunk count via the
141146
Step Functions state machine, PSNR-compares against the in-process
142147
baseline (which is git-LFS tracked under
143148
`packages/producer/tests/distributed/<fixture>/output/`), captures
144-
per-execution Step Functions history, and tears the stack down.
149+
per-execution Step Functions history, and tears the stack down. Use
150+
`--plan-protocol both` to run v1 and v2 through the same deployed Lambda
151+
package and baseline. Each v1/v2 pair is also gated directly on per-chunk
152+
hashes from Step Functions history, normalized decoded RGBA frame hashes,
153+
decoded 48 kHz stereo s16le PCM hashes and byte counts, normalized stream
154+
metadata, and duration. Encoded MP4 SHA equality is reported but is
155+
informational unless `--require-encoded-sha-equal` is set. The script
156+
assigns unique function/state-machine names, uses a
157+
dedicated temporary SAM artifact bucket, and removes render objects,
158+
retained buckets, the implicit Lambda log group, and deployment artifacts
159+
on teardown. Suspended-version buckets are purged in 1,000-entry batches,
160+
including concrete versions, null versions, and delete markers. It then
161+
verifies that the stack, both buckets, Lambda, state-machine, and both log
162+
groups are absent; an otherwise-successful run fails if cleanup cannot be
163+
proven.
145164

146165
**Wall-clock methodology caveat (`eval.sh` only).** `eval.sh` reports a
147166
local-vs-Lambda "speedup" column. The local timing includes `bun` +
@@ -162,9 +181,11 @@ spend is roughly $0.10-$0.20 per pass before S3 transfer. Lower
162181

163182
Outputs land under `<repo-root>/lambda-smoke-artifacts/`:
164183

165-
- `results.json``chunkCount × wallClockMs × psnrAvgDb`
166-
- `renders/N<N>-output.mp4` — each rendered chunk count
167-
- `renders/N<N>-history.json` — full Step Functions execution history
184+
- `results.json``planProtocol × chunkCount × wallClockMs × psnrAvgDb`
185+
- `semantic-comparisons.json` — direct v1/v2 semantic gate results
186+
- `renders/<protocol>-N<N>-output.mp4` — each rendered variant
187+
- `renders/<protocol>-N<N>-history.json` — full Step Functions execution history
188+
- `renders/v1-v2-N<N>.*` — normalized frame hashes, ffprobe metadata, and comparison JSON
168189

169190
Prerequisites: `aws` (v2), `sam` (≥ 1.100), `bun` (≥ 1.3), `ffmpeg`,
170191
`jq`, `zip`. AWS credentials come from the standard resolution chain
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"Action": "assemble",
3+
"PlanProtocol": "v2",
4+
"PlanV2ManifestS3Uri": "s3://example-bucket/renders/sample/v2/manifest.json",
5+
"PlanV2ArtifactS3Prefix": "s3://example-bucket/renders/sample/v2/artifacts/sha256",
6+
"PlanHash": "0000000000000000000000000000000000000000000000000000000000000000",
7+
"ChunkS3Uris": ["s3://example-bucket/renders/sample/chunks/0000.mp4"],
8+
"AudioS3Uri": null,
9+
"OutputS3Uri": "s3://example-bucket/renders/sample/output.mp4",
10+
"Format": "mp4"
11+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"Action": "plan",
3+
"PlanProtocol": "v2",
4+
"ProjectS3Uri": "s3://example-bucket/projects/sample.tar.gz",
5+
"PlanOutputS3Prefix": "s3://example-bucket/renders/sample/",
6+
"Config": {
7+
"fps": 30,
8+
"width": 1920,
9+
"height": 1080,
10+
"format": "mp4",
11+
"chunkSize": 240,
12+
"maxParallelChunks": 8,
13+
"runtimeCap": "lambda"
14+
}
15+
}
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"Action": "renderChunk",
3+
"PlanProtocol": "v2",
4+
"PlanV2ManifestS3Uri": "s3://example-bucket/renders/sample/v2/manifest.json",
5+
"PlanV2ArtifactS3Prefix": "s3://example-bucket/renders/sample/v2/artifacts/sha256",
6+
"PlanHash": "0000000000000000000000000000000000000000000000000000000000000000",
7+
"ChunkIndex": 0,
8+
"ChunkOutputS3Prefix": "s3://example-bucket/renders/sample/",
9+
"Format": "mp4"
10+
}
Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
#!/usr/bin/env bash
2+
# AWS resource-name isolation and failed-deploy discovery helpers.
3+
4+
hf_new_smoke_run_id() {
5+
local seconds seed digest
6+
seconds=$(date +%s)
7+
seed="${seconds}:$$:${RANDOM}:${BASHPID:-$$}"
8+
digest=$(printf '%s' "$seed" | sha256sum | awk '{print substr($1,1,16)}')
9+
printf '%s-%s\n' "$seconds" "$digest"
10+
}
11+
12+
hf_sam_deploy_bucket_name() {
13+
local account_id="$1" region="$2" run_id="$3" digest
14+
digest=$(printf '%s' "$run_id" | sha256sum | awk '{print substr($1,1,20)}')
15+
printf 'hf-sam-%s-%s-%s\n' "$account_id" "$region" "$digest"
16+
}
17+
18+
hf_derive_project_name() {
19+
local stack_name="$1" prefix digest
20+
prefix=$(printf '%s' "$stack_name" |
21+
tr -c '[:alnum:]-' '-' |
22+
sed -E 's/^-+//; s/-+$//' |
23+
cut -c1-36)
24+
[ -n "$prefix" ] || prefix="hf-smoke"
25+
digest=$(printf '%s' "$stack_name" | sha256sum | awk '{print substr($1,1,12)}')
26+
printf '%s-%s\n' "$prefix" "$digest"
27+
}
28+
29+
hf_known_absent() {
30+
local pattern="$1" output_file="$2"
31+
grep -Eiq "$pattern" "$output_file"
32+
}
33+
34+
hf_assert_command_absent() {
35+
local label="$1" absent_pattern="$2"
36+
shift 2
37+
local output_file status detail
38+
output_file=$(mktemp)
39+
if "$@" >"$output_file" 2>&1; then
40+
echo "ERROR: destructive-isolation collision: $label already exists" >&2
41+
rm -f "$output_file"
42+
return 1
43+
else
44+
status=$?
45+
fi
46+
if ! hf_known_absent "$absent_pattern" "$output_file"; then
47+
detail=$(tr '\n' ' ' < "$output_file" | cut -c1-240)
48+
echo "ERROR: could not prove $label absent (exit=$status): $detail" >&2
49+
rm -f "$output_file"
50+
return 2
51+
fi
52+
rm -f "$output_file"
53+
}
54+
55+
hf_assert_named_list_absent() {
56+
local label="$1"
57+
shift
58+
local output_file output status detail
59+
output_file=$(mktemp)
60+
if output=$("$@" 2>"$output_file"); then
61+
if [ -n "$output" ]; then
62+
echo "ERROR: destructive-isolation collision: $label already exists ($output)" >&2
63+
rm -f "$output_file"
64+
return 1
65+
fi
66+
else
67+
status=$?
68+
detail=$(tr '\n' ' ' < "$output_file" | cut -c1-240)
69+
echo "ERROR: could not verify $label absence (exit=$status): $detail" >&2
70+
rm -f "$output_file"
71+
return 2
72+
fi
73+
rm -f "$output_file"
74+
}
75+
76+
# Fail closed unless every exact name this smoke run can destructively clean
77+
# is absent. Call before arming cleanup or creating any AWS resource.
78+
hf_assert_deploy_isolation() {
79+
local stack_name="$1" project_name="$2"
80+
local function_name="${project_name}-render"
81+
local lambda_log="/aws/lambda/${function_name}"
82+
local states_log="/aws/states/${function_name}"
83+
84+
hf_assert_command_absent "CloudFormation stack $stack_name" "does not exist" \
85+
aws cloudformation describe-stacks --stack-name "$stack_name" &&
86+
hf_assert_command_absent "Lambda function $function_name" \
87+
"ResourceNotFoundException|Function not found" \
88+
aws lambda get-function --function-name "$function_name" &&
89+
hf_assert_named_list_absent "Step Functions state machine $function_name" \
90+
aws stepfunctions list-state-machines \
91+
--query "stateMachines[?name=='$function_name'].stateMachineArn" --output text &&
92+
hf_assert_named_list_absent "log group $lambda_log" \
93+
aws logs describe-log-groups --log-group-name-prefix "$lambda_log" \
94+
--query "logGroups[?logGroupName=='$lambda_log'].logGroupName" --output text &&
95+
hf_assert_named_list_absent "log group $states_log" \
96+
aws logs describe-log-groups --log-group-name-prefix "$states_log" \
97+
--query "logGroups[?logGroupName=='$states_log'].logGroupName" --output text
98+
}
99+
100+
# Atomically reserve the exact stack name before SAM can create or update it.
101+
# CloudFormation's create-stack call is the compare-and-set: only one concurrent
102+
# smoke run can acquire a name that both preflight checks observed as absent.
103+
hf_reserve_smoke_stack() {
104+
local stack_name="$1" run_id="$2"
105+
aws cloudformation create-stack \
106+
--stack-name "$stack_name" \
107+
--template-body \
108+
'{"Resources":{"SmokeOwnershipHandle":{"Type":"AWS::CloudFormation::WaitConditionHandle"}}}' \
109+
--tags "Key=HyperframesSmokeRun,Value=$run_id" >/dev/null &&
110+
aws cloudformation wait stack-create-complete --stack-name "$stack_name"
111+
}
112+
113+
# Print "owned" when the stack has this run's ownership tag and
114+
# "absent" when there is no stack. Any foreign/missing tag or AWS API error
115+
# fails closed so a cleanup trap cannot delete a concurrent run's resources.
116+
hf_stack_ownership_status() {
117+
local stack_name="$1" run_id="$2" output_file error_file owner status detail
118+
output_file=$(mktemp)
119+
error_file=$(mktemp)
120+
if aws cloudformation describe-stacks \
121+
--stack-name "$stack_name" \
122+
--query "Stacks[0].Tags[?Key=='HyperframesSmokeRun'].Value | [0]" \
123+
--output text >"$output_file" 2>"$error_file"; then
124+
owner=$(tr -d '\r\n' <"$output_file")
125+
rm -f "$output_file" "$error_file"
126+
if [ "$owner" != "$run_id" ]; then
127+
echo "ERROR: refusing cleanup: stack ownership is '${owner:-missing}', expected '$run_id'" >&2
128+
return 3
129+
fi
130+
printf 'owned\n'
131+
return
132+
else
133+
status=$?
134+
fi
135+
if hf_known_absent "does not exist" "$error_file"; then
136+
rm -f "$output_file" "$error_file"
137+
printf 'absent\n'
138+
return
139+
fi
140+
detail=$(tr '\n' ' ' <"$error_file" | cut -c1-240)
141+
echo "ERROR: could not verify stack ownership (exit=$status): $detail" >&2
142+
rm -f "$output_file" "$error_file"
143+
return 2
144+
}
145+
146+
# Return a JSON object with any physical resources CloudFormation managed to
147+
# create, even when stack outputs were never populated. A genuinely absent
148+
# stack is an empty result; auth/network/query failures are errors.
149+
hf_discover_stack_resources() {
150+
local stack_name="$1" output_file error_file status detail
151+
output_file=$(mktemp)
152+
error_file=$(mktemp)
153+
if aws cloudformation list-stack-resources \
154+
--stack-name "$stack_name" --output json >"$output_file" 2>"$error_file"; then
155+
jq '{
156+
renderBucket: (
157+
[.StackResourceSummaries[]?
158+
| select(.LogicalResourceId == "RenderBucket")
159+
| .PhysicalResourceId][0] // ""
160+
),
161+
stateMachineArn: (
162+
[.StackResourceSummaries[]?
163+
| select(.LogicalResourceId == "RenderStateMachine")
164+
| .PhysicalResourceId][0] // ""
165+
)
166+
}' "$output_file"
167+
rm -f "$output_file" "$error_file"
168+
return
169+
else
170+
status=$?
171+
fi
172+
if hf_known_absent "does not exist" "$error_file"; then
173+
printf '{"renderBucket":"","stateMachineArn":""}\n'
174+
rm -f "$output_file" "$error_file"
175+
return
176+
fi
177+
detail=$(tr '\n' ' ' < "$error_file" | cut -c1-240)
178+
echo "ERROR: failed to discover physical stack resources (exit=$status): $detail" >&2
179+
rm -f "$output_file" "$error_file"
180+
return 2
181+
}
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/usr/bin/env bash
2+
# S3 bucket cleanup helpers. Sourcing this file has no side effects.
3+
4+
# Delete every concrete object version and delete marker from a bucket.
5+
#
6+
# We intentionally re-list the first 1,000 entries after every delete batch
7+
# instead of advancing markers through a mutating result set. This handles
8+
# arbitrary pagination depth while avoiding skipped keys when the page being
9+
# used as a cursor has just been removed. It is also required for buckets with
10+
# versioning Suspended: `aws s3 rm` only creates null-version delete markers
11+
# and leaves the historical/null versions behind.
12+
hf_purge_s3_bucket_versions() {
13+
local bucket="$1" work page delete_request delete_response count errors rounds=0
14+
work=$(mktemp -d)
15+
page="$work/page.json"
16+
delete_request="$work/delete.json"
17+
delete_response="$work/delete-response.json"
18+
19+
while true; do
20+
rounds=$((rounds + 1))
21+
if [ "$rounds" -gt 100000 ]; then
22+
echo "ERROR: S3 purge exceeded 100000 batches for s3://$bucket" >&2
23+
rm -rf "$work"
24+
return 1
25+
fi
26+
27+
if ! aws s3api list-object-versions \
28+
--bucket "$bucket" \
29+
--max-keys 1000 \
30+
--no-paginate \
31+
--output json > "$page"; then
32+
echo "ERROR: failed to list object versions for s3://$bucket" >&2
33+
rm -rf "$work"
34+
return 1
35+
fi
36+
37+
jq '{
38+
Objects: [
39+
(.Versions // [])[],
40+
(.DeleteMarkers // [])[]
41+
] | map({Key, VersionId}),
42+
Quiet: true
43+
}' "$page" > "$delete_request"
44+
count=$(jq '.Objects | length' "$delete_request")
45+
if [ "$count" -eq 0 ]; then
46+
break
47+
fi
48+
49+
if ! aws s3api delete-objects \
50+
--bucket "$bucket" \
51+
--delete "file://$delete_request" \
52+
--output json > "$delete_response"; then
53+
echo "ERROR: failed to delete a version batch from s3://$bucket" >&2
54+
rm -rf "$work"
55+
return 1
56+
fi
57+
# Successful Quiet=true deletes may produce a zero-byte response body.
58+
# Slurp mode treats that as an empty input set and therefore zero errors,
59+
# while still counting per-object Errors when AWS returns a JSON object.
60+
errors=$(jq -s '[.[] | (.Errors // [])[]] | length' "$delete_response")
61+
if [ "$errors" -ne 0 ]; then
62+
echo "ERROR: S3 returned per-object deletion errors for s3://$bucket:" >&2
63+
jq -c '.Errors[]' "$delete_response" >&2
64+
rm -rf "$work"
65+
return 1
66+
fi
67+
echo " purged $count object versions/delete markers from s3://$bucket"
68+
done
69+
70+
rm -rf "$work"
71+
}
72+
73+
hf_delete_s3_bucket_completely() {
74+
local bucket="$1"
75+
hf_purge_s3_bucket_versions "$bucket" &&
76+
aws s3api delete-bucket --bucket "$bucket"
77+
}

0 commit comments

Comments
 (0)