Hey! Ran into this on @convex-dev/r2@0.10.1 and figured I'd flag it
since I couldn't find an existing issue.
Every time my app does an R2 upload, the upload itself works fine
(presigned URL, file lands in the bucket, image URL works), but right
after the component's syncMetadata action crashes server-side and
the file metadata never gets stored in Convex. Terminal logs this:
[CONVEX A(lib:syncMetadata)] Uncaught ReferenceError: Buffer is not defined
at fromString (@smithy/core/.../util-buffer-from/buffer-from.js:7)
at fromUtf8 (@smithy/core/.../util-utf8/fromUtf8.js:3)
at toUint8Array (@smithy/core/.../util-utf8/toUint8Array.js:1)
at createStringToSign (@smithy/signature-v4/.../SignatureV4Base.js:33)
at getSignature (@smithy/signature-v4/.../SignatureV4.js:133)
at signRequest (@smithy/signature-v4/.../SignatureV4.js:124)
at async sign (@aws-sdk/core/.../AwsSdkSigV4Signer.js:42)
After poking around a bit, I think this is what's going on:
src/component/lib.ts exports syncMetadata as an action without a
"use node" directive, so it runs in Convex's default V8 runtime.
That runtime doesn't have a global Buffer, but the AWS SDK's SigV4
signing path inside @smithy/signature-v4 calls Buffer.from(...) to
encode the string-to-sign and the signing key — which explodes the
moment r2.send(new HeadObjectCommand(...)) runs inside syncMetadata.
I couldn't just slap "use node" on the file because it also has
mutations and queries in it, and per Convex docs those can't share a
file with the Node runtime.
Funny wrinkle while debugging: the bug is actually intermittent in
production. Because the default V8 runtime reuses isolates and
globalThis mutations persist across function calls, if any other
function in the convex deployment has polyfilled Buffer (some apps
already do this because their own code touches the AWS SDK), then
syncMetadata happens to work for that isolate's lifetime. So you may
see metadata rows for some uploads and crashes for others, depending
on which function happened to warm the isolate first. Took me a
minute to figure out why my Convex dashboard had metadata rows from
days ago but the terminal was still spitting Buffer errors today.
Worth flagging that in any repro guide since it might explain why
this isn't 100% reproducible for everyone.
A couple of ways this could be fixed upstream:
- Move
syncMetadata (and any other actions that hit AWS SDK) into
its own file with "use node" at the top, keeping the mutations
and queries in the V8-runtime lib.ts.
- Or just polyfill Buffer at the top of
lib.ts from the pure-JS
buffer npm package — something like:
import { Buffer as BufferPolyfill } from "buffer"
if (typeof globalThis.Buffer === "undefined") {
;(globalThis as any).Buffer = BufferPolyfill
}
Until then, I've patched my own copy via pnpm patch @convex-dev/r2
with option 2 above and uploads + metadata are now working
end-to-end. Happy to open a PR if it'd help — let me know what you'd
prefer.
Thanks for the component, by the way — saved me a ton of S3 wiring
work even with this hiccup.
Label it `bug` and submit. Done.
Hey! Ran into this on
@convex-dev/r2@0.10.1and figured I'd flag itsince I couldn't find an existing issue.
Every time my app does an R2 upload, the upload itself works fine
(presigned URL, file lands in the bucket, image URL works), but right
after the component's
syncMetadataaction crashes server-side andthe file metadata never gets stored in Convex. Terminal logs this:
[CONVEX A(lib:syncMetadata)] Uncaught ReferenceError: Buffer is not defined
at fromString (@smithy/core/.../util-buffer-from/buffer-from.js:7)
at fromUtf8 (@smithy/core/.../util-utf8/fromUtf8.js:3)
at toUint8Array (@smithy/core/.../util-utf8/toUint8Array.js:1)
at createStringToSign (@smithy/signature-v4/.../SignatureV4Base.js:33)
at getSignature (@smithy/signature-v4/.../SignatureV4.js:133)
at signRequest (@smithy/signature-v4/.../SignatureV4.js:124)
at async sign (@aws-sdk/core/.../AwsSdkSigV4Signer.js:42)
After poking around a bit, I think this is what's going on:
src/component/lib.tsexportssyncMetadataas an action without a"use node"directive, so it runs in Convex's default V8 runtime.That runtime doesn't have a global
Buffer, but the AWS SDK's SigV4signing path inside
@smithy/signature-v4callsBuffer.from(...)toencode the string-to-sign and the signing key — which explodes the
moment
r2.send(new HeadObjectCommand(...))runs insidesyncMetadata.I couldn't just slap
"use node"on the file because it also hasmutations and queries in it, and per Convex docs those can't share a
file with the Node runtime.
Funny wrinkle while debugging: the bug is actually intermittent in
production. Because the default V8 runtime reuses isolates and
globalThismutations persist across function calls, if any otherfunction in the convex deployment has polyfilled Buffer (some apps
already do this because their own code touches the AWS SDK), then
syncMetadata happens to work for that isolate's lifetime. So you may
see metadata rows for some uploads and crashes for others, depending
on which function happened to warm the isolate first. Took me a
minute to figure out why my Convex dashboard had metadata rows from
days ago but the terminal was still spitting Buffer errors today.
Worth flagging that in any repro guide since it might explain why
this isn't 100% reproducible for everyone.
A couple of ways this could be fixed upstream:
syncMetadata(and any other actions that hit AWS SDK) intoits own file with
"use node"at the top, keeping the mutationsand queries in the V8-runtime
lib.ts.lib.tsfrom the pure-JSbuffernpm package — something like: