From eb0e979cdf9908d1da3ad286ea19724b0bc062d7 Mon Sep 17 00:00:00 2001 From: Noam Rosenthal Date: Wed, 9 Apr 2025 10:10:13 +0100 Subject: [PATCH 1/2] Deferred fetching Add a JS-exposed function to request a deferred fetch, called `fetchLater`. A deferred fetch would be invoked in one of two scenarios: - The document is destroyed (the fetch group is terminated). - A given period of time has passed. A few constraints: - Request body streams are not allowed. - This is only allowed in documents, and only reporting to potentially-trustworthy URLs. - Deferred fetch requests are limited to 64KB per origin. Exceeding this would immediately throw. The quota algorithm is a bit intricate, but its default should be somewhat reasonable for all but advanced cases. - A top level document has 640kb of quota for deferred fetching. This is important to avoid wasting high bandwidth after a tab has been closed. This quota is shared, by default, with the top-level's document same-origin same-agent descendants. The same-agent restriction is important for avoiding race conditions, as same-agent frames are guaranteed to call `fetchLater` in sequence. - By default, 128kb out of the 640kb quota is reserved for cross-origin or cross-agent iframes. Permissions policy (`deferred-fetch-minimal`) controls that, and the top-level document can disable that allocated quota by disabling that permissions policy. - Any document can delegate 64kb out of its reserved quota for cross-origin or cross-agent subframes, by explicitly enabling the `deferred-fetch` permissions policy. - Reserving some of the quota to a cross-origin or cross-agent subframe happens when the frame is being navigated by the container, e.g. setting `src` on an iframe. It is not guaranteed that the subframe would actually be able to use that quota, as it might end up navigating to a same-origin URL or disable the feature in its own permissions policy. However, the container's document only cares about the initial reserved value for subframes it doesn't have direct access to. See https://github.com/WICG/pending-beacon/issues/70. --- fetch.bs | 673 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 651 insertions(+), 22 deletions(-) diff --git a/fetch.bs b/fetch.bs index f1ed1e46f..670802cf4 100755 --- a/fetch.bs +++ b/fetch.bs @@ -42,6 +42,7 @@ urlPrefix:https://httpwg.org/specs/rfc9112.html#;type:dfn;spec:http1 url:status.line;text:reason-phrase url:https://w3c.github.io/resource-timing/#dfn-mark-resource-timing;text:mark resource timing;type:dfn;spec:resource-timing +url:https://w3c.github.io/webappsec-permissions-policy/#algo-define-inherited-policy-in-container;text:define an inherited policy for feature in container;type:dfn urlPrefix:https://w3c.github.io/hr-time/#;spec:hr-time type:dfn @@ -1855,7 +1856,8 @@ not always relevant and might require different behavior. connect-src navigator.sendBeacon(), {{EventSource}}, HTML's <a ping=""> and <area ping="">, - fetch(), {{XMLHttpRequest}}, {{WebSocket}}, Cache API + fetch(), fetchLater(), {{XMLHttpRequest}}, + {{WebSocket}}, Cache API "object" object-src @@ -2776,28 +2778,63 @@ functionality.

Fetch groups

Each environment settings object has an associated -fetch group. +fetch group, which holds a fetch group. -

A fetch group holds an ordered list of -fetch records. +

A fetch group holds information about fetches. -

A fetch record has an associated -request (a -request). +

A fetch group has associated: -

A fetch record has an associated -controller (a -fetch controller or null). +

+
fetch records +
A list of fetch records. + +
deferred fetch records +
A list of deferred fetch records. +
+ +

A fetch record is a struct with the following +items: + +

+
request +
A request. + +
controller +
A fetch controller or null. +
+
+ +

A deferred fetch record is a struct used to maintain state needed to +invoke a fetch at a later time, e.g., when a document is unloaded or becomes not +fully active. It has the following items: + +

+
request +
A request. + +
notify invoked +
An algorithm accepting no arguments. + +
invoke state (default "pending") +
"pending", "sent", or "aborted". +
-

When a fetch group is -terminated, for each associated -fetch record whose fetch record's -controller is non-null, and whose request's -done flag is unset or keepalive is false, -terminate the fetch record's -controller. +

When a fetch group fetchGroup is +terminated: + +

    +

  1. For each fetch record record of + fetchGroup's fetch records, if record's + controller is non-null and record's + request's done flag is unset and keepalive is + false, terminate record's + controller. + +

  2. Process deferred fetches for fetchGroup. +

+

Resolving domains

@@ -4593,15 +4630,16 @@ the response. [[!HTTP-CACHING]] dispatch and processing of HTTP/1 fetches. [[!RFC9218]]
  • -

    If request is a subresource request, then: +

    If request is a subresource request:

    1. Let record be a new fetch record whose request is request and controller is fetchParams's controller. -

    2. Append record to request's client's - fetch group list of fetch records. +

    3. Append record to request's + client's fetch group's + fetch records.

  • Run main fetch given fetchParams. @@ -5788,7 +5826,7 @@ run these steps:

  • Let inflightKeepaliveBytes be 0.

  • Let group be httpRequest's client's - fetch group. + fetch group.

  • Let inflightRecords be the set of fetch records in group whose request's keepalive is true @@ -6914,6 +6952,444 @@ agent's CORS-preflight cache for which there is a cache entry match +

    Deferred fetching

    + +

    Deferred fetching allows callers to request that a fetch is invoked at the latest possible +moment, i.e., when a fetch group is terminated, or after a +timeout. + +

    The deferred fetch task source is a task source used to update the result of a +deferred fetch. User agents must prioritize tasks in this task source before other task +sources, specifically task sources that can result in running scripts such as the +DOM manipulation task source, to reflect the most recent state of a +fetchLater() call before running any scripts that might depend on it. + +

    +

    To queue a deferred fetch given a request request, a null or +{{DOMHighResTimeStamp}} activateAfter, and onActivatedWithoutTermination, +which is an algorithm that takes no arguments: + +

      +

    1. Populate request from client given request. + +

    2. Set request's service-workers mode to "none". + +

    3. Set request's keepalive to true. + +

    4. Let deferredRecord be a new deferred fetch record whose + request is request, and whose + notify invoked is + onActivatedWithoutTermination. + +

    5. Append deferredRecord to request's + client's fetch group's + deferred fetch records. + +

    6. +

      If activateAfter is non-null, then run the following steps in parallel: + +

        +
      1. +

        The user agent should wait until any of the following conditions is met: + +

          +

        • At least activateAfter milliseconds have passed. + +

        • The user agent has a reason to believe that it is about to lose the opportunity to + execute scripts, e.g., when the browser is moved to the background, or when + request's client's + global object is a {{Window}} object whose + associated document had a "hidden" visibility state for + a long period of time. +

        + +

      2. Process deferredRecord. +

      + +

    7. Return deferredRecord. +

    +
    + +
    +

    To compute the total request length of a request request: + +

      +

    1. Let totalRequestLength be the length of request's + URL, serialized with + exclude fragment set to true. + +

    2. Increment totalRequestLength by the length of + request's referrer, serialized. + +

    3. For each (name, value) of request's + header list, increment totalRequestLength by name's + length + value's length. + +

    4. Increment totalRequestLength by request's body's + length. + +

    5. Return totalRequestLength. +

    +
    + +
    +

    To process deferred fetches given a fetch group fetchGroup: + +

      +

    1. For each deferred fetch record + deferredRecord of fetchGroup's + deferred fetch records, process a deferred fetch + deferredRecord. +

    +
    + +
    +

    To process a deferred fetch deferredRecord: +

      +

    1. If deferredRecord's invoke state is not + "pending", then return. + +

    2. Set deferredRecord's invoke state to + "sent". + +

    3. Fetch deferredRecord's request. + +

    4. Queue a global task on the deferred fetch task source with + deferredRecord's request's + client's global object to run + deferredRecord's notify invoked. +

    +
    + +

    Deferred fetching quota

    + +

    This section is non-normative. + +

    The deferred-fetch quota is allocated to a top-level traversable (a "tab"), +amounting to 640 kibibytes. The top-level document and its same-origin directly nested documents can +use this quota to queue deferred fetches, or delegate some of it to cross-origin nested documents, +using permissions policy. + +

    By default, 128 kibibytes out of these 640 kibibytes are allocated to delegating the quota to +cross-origin nested documents, each reserving 8 kibibytes. + +

    The top-level document, and subsequently its nested documents, can control how much +of their quota is delegates to cross-origin child documents, using permissions policy. By default, +the "{{PermissionsPolicy/deferred-fetch-minimal}}" policy is enabled for any origin, while +"{{PermissionsPolicy/deferred-fetch}}" is enabled for the top-level document's origin only. By +relaxing the "{{PermissionsPolicy/deferred-fetch}}" policy for particular origins and nested +documents, the top-level document can allocate 64 kibibytes to those nested documents. Similarly, by +restricting the "{{PermissionsPolicy/deferred-fetch-minimal}}" policy for a particular origin or +nested document, the document can prevent the document from reserving the 8 kibibytes it would +receive by default. By disabling the "{{PermissionsPolicy/deferred-fetch-minimal}}" policy for the +top-level document itself, the entire 128 kibibytes delegated quota is collected back into the main +pool of 640 kibibytes. + +

    Out of the allocated quota for a document, only 64 kibibytes can be used +concurrently for the same reporting origin (the request's URL's +origin). This prevents a situation where particular third-party libraries would reserve +quota opportunistically, before they have data to send. + +

    +

    Any of the following calls to fetchLater() would throw due to + the request itself exceeding the 64 kibibytes quota allocated to a reporting origin. Note that the + size of the request includes the URL itself, the body, the + header list, and the referrer. + + 

    
+fetchLater(a_72_kb_url);
+fetchLater("https://origin.example.com", {headers: headers_exceeding_64kb});
+fetchLater(a_32_kb_url, {headers: headers_exceeding_32kb});
+fetchLater("https://origin.example.com", {method: "POST", body: body_exceeding_64_kb});
+fetchLater(a_62_kb_url /* with a 3kb referrer */);
+
    + +

    In the following sequence, the first two requests would succeed, but the third one would throw. + That's because the overall 640 kibibytes quota was not exceeded in the first two calls, however the + 3rd request exceeds the reporting-origin quota for https://a.example.com, and would + throw. + + 

    
+fetchLater("https://a.example.com", {method: "POST", body: a_64kb_body});
+fetchLater("https://b.example.com", {method: "POST", body: a_64kb_body});
+fetchLater("https://a.example.com");
+
    + +

    Same-origin nested documents share the quota of their parent. However, cross-origin or + cross-agent iframes only receive 8kb of quota by default. So in the following example, the first + three calls would succeed and the last one would throw. + + 

    
+// In main page
+fetchLater("https://a.example.com", {method: "POST", body: a_64kb_body});
+
+// In same-origin nested document
+fetchLater("https://b.example.com", {method: "POST", body: a_64kb_body});
+
+// In cross-origin nested document at https://fratop.example.com
+fetchLater("https://a.example.com", {body: a_5kb_body});
+fetchLater("https://a.example.com", {body: a_12kb_body});
+
    + +

    To make the previous example not throw, the top-level document can delegate some of its quota + to https://fratop.example.com, for example by serving the following header: + + 

    Permissions-Policy: deferred-fetch=(self "https://fratop.example.com")
    + +

    Each nested document reserves its own quota. So the following would work, because each frame + reserve 8 kibibytes: + + 

    
+// In cross-origin nested document at https://fratop.example.com/frame-1
+fetchLater("https://a.example.com", {body: a_6kb_body});
+
+// In cross-origin nested document at https://fratop.example.com/frame-2
+fetchLater("https://a.example.com", {body: a_6kb_body});
+
    + +

    The following tree illustrates how quota is distributed to different nested documents in a tree: + +

      +
    • +

      https://top.example.com, with permissions policy set to + Permissions-policy: deferred-fetch=(self "https://ok.example.com") +

        +
      • +

        https://top.example.com/frame: shares quota with the top-level traversable, as + they are same origin. + +

        • https://x.example.com: receives 8 kibibytes.

        + +
      • +

        https://x.example.com: receives 8 kibibytes. +

        • https://top.example.com: 0. Even though it's same origin with the + top-level traversable, it does not automatically share its quota as they are separated by a + cross-origin intermediary.

        + +
      • +

        https://ok.example.com/good: receives 64 kibibytes, granted via the + "{{PermissionsPolicy/deferred-fetch}}" policy. + +

        • https://x.example.com: receives no quota. Only documents with the same + origin as the top-level traversable can grant the 8 kibibytes based on the + "{{PermissionsPolicy/deferred-fetch-minimal}}" policy.

        + +

      • https://ok.example.com/redirect, navigated to + https://x.example.com: receives no quota. The reserved 64 kibibytes for + https://ok.example.com are not available for + https://x.example.com. + +

      • https://ok.example.com/back, navigated to + https://top.example.com: shares quota with the top-level traversable, as they're + same origin. +

      +
    + +

    In the above example, the top-level traversable and its same origin + descendants share a quota of 384 kibibytes. That value is computed as such: +

      +

    • 640 kibibytes are initially granted to the top-level traversable. + +

    • 128 kibibytes are reserved for the "{{PermissionsPolicy/deferred-fetch-minimal}}" policy. + +

    • 64 kibibytes are reserved for the container navigating to + https://ok.example/good. + +

    • 64 kibibytes are reserved for the container navigating to + https://ok.example/redirect, and lost when it navigates away. + +

    • https://ok.example.com/back did not reserve 64 kibibytes, because it navigated + back to top-level traversable's origin. + +

    • 640 − 128 − 64 − 64 = 384 kibibytes. +

    +
    + +

    This specification defines a policy-controlled feature identified by the string +"deferred-fetch". Its +default allowlist is "self". + +

    This specification defines a policy-controlled feature identified by the string +"deferred-fetch-minimal". Its +default allowlist is "*". + +

    The quota reserved for deferred-fetch-minimal is 128 kibibytes. + +

    Each navigable container has an associated number +reserved deferred-fetch quota. Its possible values are +minimal quota, which is 8 kibibytes, +normal quota, which is 64 kibibytes, or 0. Unless +stated otherwise, it is 0. + +

    +

    To get the available deferred-fetch quota given a document +document and an origin-or-null origin: + +

      +

    1. Let controlDocument be document's + deferred-fetch control document. + +

    2. Let navigable be controlDocument's node navigable. + +

    3. Let isTopLevel be true if controlDocument's node navigable is a + top-level traversable; otherwise false. + +

    4. Let deferredFetchAllowed be true if controlDocument is + allowed to use the policy-controlled feature + "{{PermissionsPolicy/deferred-fetch}}"; otherwise false. + +

    5. Let deferredFetchMinimalAllowed be true if controlDocument is + allowed to use the policy-controlled feature + "{{PermissionsPolicy/deferred-fetch-minimal}}"; otherwise false. + +

    6. +

      Let quota be the result of the first matching statement: + +

      +
      isTopLevel is true and deferredFetchAllowed is false +
      0 + +
      isTopLevel is true and deferredFetchMinimalAllowed is false +
      +

      640 kibibytes +

      640kb should be enough for everyone. + +

      isTopLevel is true +
      +

      512 kibibytes +

      The default of 640 kibibytes, decremented By + quota reserved for deferred-fetch-minimal) + +

      deferredFetchAllowed is true, and navigable's + navigable container's reserved deferred-fetch quota is + normal quota +
      normal quota + +
      deferredFetchMinimalAllowed is true, and navigable's + navigable container's reserved deferred-fetch quota is + minimal quota +
      minimal quota + +
      Otherwise +
      0 +
      + +

    7. Let quotaForRequestOrigin be 64 kibibytes. + +

    8. +

      For each navigable in controlDocument's + node navigable's inclusive descendant navigables whose + active document's deferred-fetch control document is + controlDocument: + +

        +

      1. For each container in navigable's + active document's shadow-including inclusive descendants which is a + navigable container, decrement quota by container's + reserved deferred-fetch quota. + +

      2. +

        For each deferred fetch record deferredRecord of + navigable's active document's relevant settings object's + fetch group's + deferred fetch records: + +

          +

        1. Let requestLength be the total request length of + deferredRecord's request. + +

        2. Decrement quota by requestLength. + +

        3. If deferredRecord's request's + URL's origin is same origin with origin, + then decrement quotaForRequestOrigin by requestLength. +

        +
      + +

    9. If quota is equal or less than 0, then return 0. + +

    10. If quota is less than quotaForRequestOrigin, then return + quota. + +

    11. Return quotaForRequestOrigin. +

    +
    + +
    +

    To reserve deferred-fetch quota for a navigable container +container given an origin originToNavigateTo: + +

    This is called on navigation, when the source document of the navigation is the +navigable's parent document. It potentially reserves either 64kb or 8kb of quota for +the container and its navigable, if allowed by permissions policy. It is not observable to the +cotnainer document whether the reserved quota was used in practice. This algorithm assumes that the +container's document might delegate quota to the navigated container, and the reserved quota would +only apply in that case, and would be ignored if it ends up being shared. If quota was reserved and +the document ends up being same origin with its parent, the quota would be +freed. + +

      +

    1. Set container's reserved deferred-fetch quota to 0. + +

    2. Let controlDocument be container's node document's + deferred-fetch control document. + +

    3. If the inherited policy + for "{{PermissionsPolicy/deferred-fetch}}", container and originToNavigateTo + is "Enabled", and the available deferred-fetch quota for + controlDocument is equal or greater than + normal quota, then set container's + reserved deferred-fetch quota to normal quota and + return. + +

    4. +

      If all of the following conditions are true: + +

      + +

      then set container's reserved deferred-fetch quota to + minimal quota. +

    +
    + +
    +

    To potentially free deferred-fetch quota for a document +document, if document's node navigable's container document is +not null, and its origin is same origin with document, then +set document's node navigable's navigable container's +reserved deferred-fetch quota to 0. + +

    This is called when a document is created. It ensures that same-origin +nested documents don't reserve quota, as they anyway share their parent quota. It can only be called +upon document creation, as the origin of the document is only known +after redirects are handled. +

    + +
    +

    To get the deferred-fetch control document of a document +document: + +

      +

    1. If document' node navigable's container document is null or a + document whose origin is not same origin with + document, then return document; otherwise, return the + deferred-fetch control document given document's node navigable's + container document. +

    +

    Fetch API

    @@ -8637,12 +9113,25 @@ otherwise false. -

    Fetch method

    +

    Fetch methods

     partial interface mixin WindowOrWorkerGlobalScope {
   [NewObject] Promise<Response> fetch(RequestInfo input, optional RequestInit init = {});
 };
+
+dictionary DeferredRequestInit : RequestInit {
+  DOMHighResTimeStamp activateAfter;
+};
+
+[Exposed=Window]
+interface FetchLaterResult {
+  readonly attribute boolean activated;
+};
+
+partial interface Window {
+  [NewObject] FetchLaterResult fetchLater(RequestInfo input, optional DeferredRequestInit init = {});
+};
    @@ -8776,6 +9265,146 @@ with a promise, request, responseObject, and an
    +

    A {{FetchLaterResult}} has an associated activated getter steps, +which is an algorithm returning a boolean. + +

    +

    The activated getter steps are to return +the result of running this's activated getter steps. +

    + +
    +

    The fetchLater(input, init) +method steps are: + +

      +

    1. Let requestObject be the result of invoking the initial value of {{Request}} as + constructor with input and init as arguments. + +

    2. If requestObject's signal is aborted, + then throw signal's abort reason. + +

    3. Let request be requestObject's request. + +

    4. Let activateAfter be null. + +

    5. If init is given and init["{{DeferredRequestInit/activateAfter}}"] + exists, then set activateAfter to + init["{{DeferredRequestInit/activateAfter}}"]. + +

    6. If activateAfter is less than 0, then throw a {{RangeError}}. + +

    7. If this's relevant global object's associated document is not + fully active, then throw a {{TypeError}}. + +

    8. If request's URL's scheme is not an + HTTP(S) scheme, then throw a {{TypeError}}. + +

    9. If request's URL is not a potentially trustworthy URL, + then throw a {{TypeError}}. + +

    10. +

      If request's body is not null, and request's + body length is null, then throw a {{TypeError}}. + +

      Requests whose body is a {{ReadableStream}} object cannot be + deferred. + +

    11. If the available deferred-fetch quota given request's + client and request's URL's + origin is less than request's total request length, then throw a + "{{QuotaExceededError}}" {{DOMException}}. + +

    12. Let activated be false. + +

    13. Let deferredRecord be the result of calling queue a deferred fetch given + request, activateAfter, and the following step: set activated to + true. + +

    14. Add the following abort steps to requestObject's + signal: Set deferredRecord's + invoke state to "aborted". + +

    15. Return a new {{FetchLaterResult}} whose + activated getter steps are to return activated. +

    +
    + +
    +

    The following call would queue a request to be fetched when the document is terminated: + + 

    
+fetchLater("https://report.example.com", {
+  method: "POST",
+  body: JSON.stringify(myReport),
+  headers: { "Content-Type": "application/json" }
+})
+
    + +

    The following call would also queue this request after 5 seconds, and the returned value would + allow callers to observe if it was indeed activated. Note that the request is guaranteed to be + invoked, even in cases where the user agent throttles timers. + + 

    
+const result = fetchLater("https://report.example.com", {
+  method: "POST",
+  body: JSON.stringify(myReport),
+  headers: { "Content-Type": "application/json" },
+  activateAfter: 5000
+});
+
+function check_if_fetched() {
+  return result.activated;
+}
+
    + +

    The {{FetchLaterResult}} object can be used together with an {{AbortSignal}}. For example: + + 

    
+let accumulated_events = [];
+let previous_result = null;
+const abort_signal = new AbortSignal();
+function accumulate_event(event) {
+  if (previous_result) {
+    if (previous_result.activated) {
+      // The request is already activated, we can start from scratch.
+      accumulated_events = [];
+    } else {
+      // Abort this request, and start a new one with all the events.
+      signal.abort();
+    }
+  }
+
+  accumulated_events.push(event);
+  result = fetchLater("https://report.example.com", {
+    method: "POST",
+    body: JSON.stringify(accumulated_events),
+    headers: { "Content-Type": "application/json" },
+    activateAfter: 5000,
+    abort_signal
+  });
+}
+
    + +

    Any of the following calls to fetchLater() would throw: + + 

    
+// Only potentially trustworthy URLs are supported.
+fetchLater("http://untrusted.example.com");
+
+// The length of the deferred request has to be known when.
+fetchLater("https://origin.example.com", {body: someDynamicStream});
+
+// Deferred fetching only works on active windows.
+const detachedWindow = iframe.contentWindow;
+iframe.remove();
+detachedWindow.fetchLater("https://origin.example.com");
+
    + +

    See deferred fetch quota examples for examples + portraying how the deferred-fetch quota works. +

    +

    Garbage collection

    From 26b5c4907809719c14965c562370b69157c608ae Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Tue, 29 Jul 2025 16:49:55 +0200 Subject: [PATCH 2/2] nit --- fetch.bs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fetch.bs b/fetch.bs index 670802cf4..38f61e8b2 100755 --- a/fetch.bs +++ b/fetch.bs @@ -7217,8 +7217,8 @@ fetchLater("https://a.example.com", {body: a_6kb_body});

    Each navigable container has an associated number reserved deferred-fetch quota. Its possible values are -minimal quota, which is 8 kibibytes, -normal quota, which is 64 kibibytes, or 0. Unless +minimal quota, which is 8 kibibytes, and +normal quota, which is 0 or 64 kibibytes. Unless stated otherwise, it is 0.