Skip to content

[wrangler] Improve asset upload performance with single-file uploads#14305

Draft
jbwcloudflare wants to merge 1 commit into
cloudflare:mainfrom
jbwcloudflare:wrangler-edge-kv-asset-upload
Draft

[wrangler] Improve asset upload performance with single-file uploads#14305
jbwcloudflare wants to merge 1 commit into
cloudflare:mainfrom
jbwcloudflare:wrangler-edge-kv-asset-upload

Conversation

@jbwcloudflare

Copy link
Copy Markdown
Contributor

Fixes #WC-4107

Begin onboarding wrangler to use the improved API endpoint for uploading Workers Assets.
This is being staged as a gradual rollout:

  • By default, users will continue to use the existing wrangler codepath and API endpoint.
  • Users that are opted-in to the new feature (controlled on the server side) will use the new endpoint.

This allows us to gradually roll out and validate the new feature without a hard cutover during a new wrangler release, or the need for emergency wrangler patch releases if something goes wrong.

  • Tests
    • Tests included/updated
    • Automated tests not possible - manual testing has been completed as follows:
    • Additional testing not necessary because:
  • Public documentation
    • Cloudflare docs PR(s):
    • Documentation not necessary because: not a new feature just an implementation detail change

A picture of a cute animal (not mandatory, but encouraged)

@changeset-bot

changeset-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c48cd7c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
wrangler Patch
@cloudflare/vite-plugin Patch
@cloudflare/vitest-pool-workers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Jun 15, 2026

Copy link
Copy Markdown
create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@14305

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/@cloudflare/deploy-helpers@14305

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@14305

miniflare

npm i https://pkg.pr.new/miniflare@14305

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@14305

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@14305

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@14305

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@14305

@cloudflare/workers-auth

npm i https://pkg.pr.new/@cloudflare/workers-auth@14305

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@14305

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@14305

wrangler

npm i https://pkg.pr.new/wrangler@14305

commit: c48cd7c

@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from 5b5e896 to c549f98 Compare June 15, 2026 13:53
@jbwcloudflare jbwcloudflare marked this pull request as ready for review June 15, 2026 14:09
@workers-devprod workers-devprod requested review from a team and james-elicx and removed request for a team June 15, 2026 14:09
@workers-devprod

Copy link
Copy Markdown
Contributor

Codeowners approval required for this PR:

  • @cloudflare/wrangler
Show detailed file reviewers
  • .changeset/edge-kv-asset-upload.md: [@cloudflare/wrangler]
  • packages/deploy-helpers/src/deploy/helpers/assets.ts: [@cloudflare/wrangler]
  • packages/deploy-helpers/src/deploy/helpers/jwt.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/tests/deploy/assets.test.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/tests/deploy/helpers.ts: [@cloudflare/wrangler]
  • packages/wrangler/src/pages/upload.ts: [@cloudflare/wrangler]

devin-ai-integration[bot]

This comment was marked as resolved.

@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from c549f98 to ba647bb Compare June 15, 2026 18:23
devin-ai-integration[bot]

This comment was marked as resolved.

@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from ba647bb to 13f5a9f Compare June 15, 2026 19:21
devin-ai-integration[bot]

This comment was marked as resolved.

@jbwcloudflare jbwcloudflare marked this pull request as draft June 15, 2026 19:36
@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from 13f5a9f to 6676947 Compare June 15, 2026 19:42
@jbwcloudflare jbwcloudflare marked this pull request as ready for review June 15, 2026 19:43
@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from 6676947 to 5d74650 Compare June 15, 2026 20:10

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

export function getEdgeKvUploadConcurrency(jwt: string): number {
try {
const value = Number(decodeJwtPayload(jwt).edge_kv_upload_concurrency);
return value > 0 ? Math.floor(value) : EDGE_KV_UPLOAD_CONCURRENCY;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 getEdgeKvUploadConcurrency can return 0 for fractional JWT values between 0 and 1, breaking PQueue

When edge_kv_upload_concurrency in the JWT is a fractional value in the range (0, 1) — e.g. 0.5 — the check value > 0 passes but Math.floor(value) produces 0. PQueue requires concurrency ≥ 1; setting it to 0 will cause the queue to either throw or hang indefinitely, preventing any uploads from completing. While the value is server-controlled (making this unlikely in practice), a simple guard like Math.max(1, Math.floor(value)) or checking value >= 1 would make this robust.

Suggested change
return value > 0 ? Math.floor(value) : EDGE_KV_UPLOAD_CONCURRENCY;
return value >= 1 ? Math.floor(value) : EDGE_KV_UPLOAD_CONCURRENCY;
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@jbwcloudflare jbwcloudflare Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine lol we will not set this to a fractional value between 0 and 1 on the server side

@james-elicx james-elicx marked this pull request as draft June 15, 2026 20:44
@jbwcloudflare jbwcloudflare changed the title [wrangler] Support Edge KV single-file asset uploads [wrangler] Improve asset upload performance with single-file uploads Jun 16, 2026
@jbwcloudflare jbwcloudflare force-pushed the wrangler-edge-kv-asset-upload branch from 5d74650 to c48cd7c Compare June 17, 2026 19:04
},
body: payload,
let res: UploadResponse;
if (useEdgeKvUpload) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the new codepath: if the gate is on, use the new single-file upload endpoint

}
);
} else {
// Populate the payload only when actually uploading (this is limited to 3 concurrent uploads at 50 MiB per bucket meaning we'd only load in a max of ~150 MiB)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the old codepath: just moved down into the else

const contentType = getContentType(absFilePath);
res = await fetchResult<UploadResponse>(
complianceConfig,
`/accounts/${accountId}/workers/assets/upload/${manifestEntry[1].hash}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't realize we were using the same endpoint but with different semantics, i kinda figured we'd have a new endpoint? not an issue though

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the old bulk one is:
/accounts/${accountId}/workers/assets/upload,

and the new single file-one is:
/accounts/${accountId}/workers/assets/upload/:hash

so they are different, unless i'm misunderstanding the question?


const queue = new PQueue({
concurrency: useEdgeKvUpload
? getEdgeKvUploadConcurrency(initializeAssetsResponse.jwt)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need concurrency limits for single-file uploads? i guess probably if there's any level of buffering, and definitely better start there and increase it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure i'm following: edgeKvUpload is the single-file mode and this will currently set the concurrency limit to 25 by default (it's actually a setting in the JWT so we can increase it from EWC later if we want)

is that what you meant? or some kind of server side concurrency limit maybe?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Untriaged

Development

Successfully merging this pull request may close these issues.

3 participants