Skip to content

Buffer is not defined in lib:syncMetadata when running in V8 runtime #62

Description

@ParwarYasinQadr

Hey! Ran into this on @convex-dev/r2@0.10.1 and figured I'd flag it
since I couldn't find an existing issue.

Every time my app does an R2 upload, the upload itself works fine
(presigned URL, file lands in the bucket, image URL works), but right
after the component's syncMetadata action crashes server-side and
the file metadata never gets stored in Convex. Terminal logs this:

[CONVEX A(lib:syncMetadata)] Uncaught ReferenceError: Buffer is not defined
at fromString (@smithy/core/.../util-buffer-from/buffer-from.js:7)
at fromUtf8 (@smithy/core/.../util-utf8/fromUtf8.js:3)
at toUint8Array (@smithy/core/.../util-utf8/toUint8Array.js:1)
at createStringToSign (@smithy/signature-v4/.../SignatureV4Base.js:33)
at getSignature (@smithy/signature-v4/.../SignatureV4.js:133)
at signRequest (@smithy/signature-v4/.../SignatureV4.js:124)
at async sign (@aws-sdk/core/.../AwsSdkSigV4Signer.js:42)

After poking around a bit, I think this is what's going on:

src/component/lib.ts exports syncMetadata as an action without a
"use node" directive, so it runs in Convex's default V8 runtime.
That runtime doesn't have a global Buffer, but the AWS SDK's SigV4
signing path inside @smithy/signature-v4 calls Buffer.from(...) to
encode the string-to-sign and the signing key — which explodes the
moment r2.send(new HeadObjectCommand(...)) runs inside syncMetadata.

I couldn't just slap "use node" on the file because it also has
mutations and queries in it, and per Convex docs those can't share a
file with the Node runtime.

Funny wrinkle while debugging: the bug is actually intermittent in
production. Because the default V8 runtime reuses isolates and
globalThis mutations persist across function calls, if any other
function in the convex deployment has polyfilled Buffer (some apps
already do this because their own code touches the AWS SDK), then
syncMetadata happens to work for that isolate's lifetime. So you may
see metadata rows for some uploads and crashes for others, depending
on which function happened to warm the isolate first. Took me a
minute to figure out why my Convex dashboard had metadata rows from
days ago but the terminal was still spitting Buffer errors today.
Worth flagging that in any repro guide since it might explain why
this isn't 100% reproducible for everyone.

A couple of ways this could be fixed upstream:

  1. Move syncMetadata (and any other actions that hit AWS SDK) into
    its own file with "use node" at the top, keeping the mutations
    and queries in the V8-runtime lib.ts.
  2. Or just polyfill Buffer at the top of lib.ts from the pure-JS
    buffer npm package — something like:
import { Buffer as BufferPolyfill } from "buffer"
if (typeof globalThis.Buffer === "undefined") {
  ;(globalThis as any).Buffer = BufferPolyfill
}

Until then, I've patched my own copy via pnpm patch @convex-dev/r2
with option 2 above and uploads + metadata are now working
end-to-end. Happy to open a PR if it'd help — let me know what you'd
prefer.

Thanks for the component, by the way — saved me a ton of S3 wiring
work even with this hiccup.

Label it `bug` and submit. Done.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions