Cannot handle large files with multipart-parser

[season-goodbye]

Reproduction

1. Upload a file via multipart/form-data that exceeds the available memory capacity of the environment.
2. Convert the request into FormData using parseFormData (@remix-run/form-data-parser).

cf. https://reactrouter.com/how-to/file-uploads

System Info

N/A

Used Package Manager

npm

Expected Behavior

The request should be correctly converted into FormData.

Actual Behavior

The process crashes because it attempts to allocate memory equal to the file size.

This seems to be caused by MultipartParser (@remix-run/multipart-parser) creating a buffer for each part.
cf. https://github.com/mjackson/remix-the-web/blob/88b3716a6eb0b0dcd8ef1e4b5051ccd00593d819/packages/multipart-parser/CHANGELOG.md

Is this a deliberate limitation in react-router to only handle small files?

[season-goodbye]

I don’t fully understand exactly when and how the deadlock is occurring, but I was wondering if an approach like the following—handling streams separately from MultipartPart—could avoid the issue?
(Sorry if I’m completely off the mark here.)

1. Add streamHandler to the constructor:

// type StreamHandler = (
//   fieldName: string,
//   fileName: string,
//   mediaType: string,
//   body: ReadableStream<Uint8Array>
// ) => Promise<void | null | string | Blob>;

constructor(boundary: string, options: MultipartParserOptions, private streamHandler: StreamHandler) {

2. Modify the Header handling in *write:

        this.#currentPart = new MultipartPart(chunk.subarray(index, headerEndIndex), []);
        this.#contentLength = 0;

        // Added below
        if (this.#currentPart.isFile) {
          const {name = "", filename = "", mediaType = ""} = this.#currentPart;

          // #handlerReturnPromise: Promise<void | null | string | Blob> | null = null;
          this.#handlerReturnPromise = this.streamHandler(
            name,
            filename,
            mediaType,
            new ReadableStream<Uint8Array>({
              start: (controller) => {
                // #bodyController: ReadableStreamDefaultController<Uint8Array> | null = null;
                this.#bodyController = controller;
              },
            })
          );
        }

3. Modify the #append logic:

    if (this.#contentLength + chunk.length > this.maxFileSize) {
      const error = new MaxFileSizeExceededError(this.maxFileSize);
      // Added below
      if (this.#bodyController) {
        this.#bodyController.error(error);
      }
      throw error;
    }

    // Added below
    if (this.#bodyController) {
      this.#bodyController.enqueue(chunk);
    } else if (this.#currentPart!.content instanceof Array) {
      this.#currentPart!.content.push(chunk);
    }

4. Add #closeBody:

  #closeBody() {
    this.#bodyController?.close();
    this.#bodyController = null;
  }

5. Modify the Body handling in *write:

        this.#append(chunk.subarray(index, boundaryIndex));
        // Added below
        this.#closeBody();

        // Added below
        if (this.#handlerReturnPromise) {
          const returned = await this.#handlerReturnPromise;
          if (returned) {
            const content = typeof returned === "string" ? encoder.encode(returned) : new Uint8Array(await returned.arrayBuffer());
            this.#currentPart?.content.push(content);
          }
          this.#handlerReturnPromise = null;
        }

        yield this.#currentPart!;

-- This code has not been validated.