SaveAs can cause excessive heap usage when saving workbooks with many large worksheet XML parts

[1436761858]

Description

Description

We are seeing very high heap usage when calling SaveAs on workbooks created from large templates.

In our production workload, we use XLSX templates that contain many worksheets. The service opens the template, writes report data into it, and then calls SaveAs to generate the final workbook.

For some large templates, memory usage stays relatively low while opening the file and writing rows, but increases sharply during SaveAs. In one case, heap usage rose to around 1.5 GB during save. This can cause OOM in smaller production containers, for example 2 CPU / 2 GB memory containers.

The issue becomes more visible when the workbook contains many large worksheet XML parts. Those worksheet XML files can be spilled to temporary files when opening the workbook, but they are loaded back into memory during save.

Root Cause

In writeToZip, temp-file-backed entries are currently written with:

if n, err = fi.Write(f.readBytes(path)); int64(n) > math.MaxUint32 {
    f.zip64Entries = append(f.zip64Entries, path)
}

I will create a pr, could help check it?

[1436761858]

Steps to Reproduce

1. Prepare an XLSX template with many worksheets.

   • The workbook should contain many large worksheet XML parts.
   • In our case, the worksheets contain a large number of formulas.
   • The compressed XLSX file size is around 20 MB, while the uncompressed worksheet XML data is much larger.

2. Open the workbook with Excelize.

3. Modify the workbook, for example by writing report data into one or more worksheets.

4. Start memory profiling with pprof, or monitor heap usage with another memory profiling tool.

5. Call SaveAs to write the modified workbook to a new XLSX file.

6. Observe heap usage during SaveAs.

In our case, memory usage stayed relatively low before SaveAs, but increased sharply during SaveAs, reaching around 1.5 GB heap usage. In memory-constrained containers, such as 2 CPU / 2 GB memory containers, this can lead to OOM.

[nick133]

I'm not sure it's related to this exact issue, but I have similar behaviour doing Save() or SaveAs(...) with existing opened file after editing cell(s) - program freezes and CPU usage goes to 100%. The opened file is not huge though, 4 sheets and just text/numeric data, no charts, no images.

[xuri]

Thanks for your feedback, stream writer's performance depend on many factors: the number of rows / columns, cell styles, and text length. Additionally, since I don't know your CPU hardware specifications and no reproduce demo provided, I can't determine if the performance you're seeing is in expected. The following benchmark report compares memory usage before and after the changes in this PR. The benchmarks were conducted by creating a workbook generated by xuri/excelize-benchmark containing 512K cells based on version 09f0656.

Environment:
Windows 11 Enterprise, 13th Gen Intel® Core™ i9-13900 (2.00 GHz), 64 GB, 500GB SSD, Go version go1.25.5 windows/amd64

Workbook generated using the following command:

v2.11.0-master -func=StreamWriter -rows=102400 -cols=50 -n=6

Memory usage:

Func: StreamWriter_r102400xc50.xlsx  RSS = 182 MB  Alloc = 134 MB  TotalAlloc = 1470 MB  Sys = 189 MB  NumGC = 71  Cost = 3.1179301s

Please also reference the performance benchmark report, if the performance of generating workbooks at a comparable scale of cells aligns with the trends in the table, it is operating within expectations.