fix: ocr gpu memory usage by aeoform · Pull Request #24996 · immich-app/immich

aeoform · 2026-01-03T14:23:46Z

Description

This PR fixes excessive GPU memory usage in the OCR pipeline.

In text recognition, extremely wide text crops could cause batch-level padding to grow unbounded, leading to steadily increasing GPU memory usage. The fix processes these extreme crops separately while keeping normal recognition behavior unchanged.
In text detection, a missing image shape normalization step was added to ensure consistent input dimensions.

These changes stabilize GPU memory usage while preserving OCR accuracy.

Fixes #23462

How Has This Been Tested?

OCR on images with a large number of detected text boxes.
OCR on images containing very long horizontal text.
Verified stable GPU memory usage compared to previous behavior.

Tested locally using the default Immich OCR configuration with ONNX Runtime.

Checklist:

I have performed a self-review of my own code
I have made corresponding changes to the documentation if applicable
I have no unrelated changes in the PR.
I have confirmed that any new dependencies are strictly necessary.
I have written tests for new code (if applicable)
I have followed naming conventions/patterns in the surrounding code

Please describe to which degree, if any, an LLM was used in creating this pull request.

An LLM was used as a general coding and reasoning assistant during development.

feat(ocr): 优化OCR模型性能和添加子项目依赖 - 添加RapidOCR子项目作为依赖 - 在文本检测器中添加图像尺寸日志记录 - 优化文本检测器的图像预处理，添加固定形状填充以支持批处理推理 - 限制图像调整大小后的尺寸不超过目标尺寸 - 重构文本识别器，添加宽高比计算函数 - 实现极端宽高比文本的单独处理以节省显存 - 将文本识别分为正常文本和极端文本两部分处理 - 添加结果占位和后处理逻辑 - 优化内存使用和推理性能 ```

immich-push-o-matic · 2026-01-03T14:23:58Z

Label error. Requires exactly 1 of: changelog:.*. Found: 🧠machine-learning. A maintainer will add the required label.

bo0tzz · 2026-01-03T14:29:35Z

Please fill out the PR description.

bo0tzz · 2026-01-03T14:34:24Z

The description is the first post, at the top of the page. You can edit it by clicking the three dots.

aeoform · 2026-01-03T14:48:12Z

OK，I did.

mertalev

Thanks for the PR! It’s a step in the right direction, but I think the strategy needs refinement and the code quality could be improved as well.

mertalev · 2026-01-03T16:41:41Z

        resize_h = int(round(resize_h / 32) * 32)
        resize_w = int(round(resize_w / 32) * 32)
+
+        # Calculate target size (fixed shape for padding)


Instead of using a square, I think a boxing strategy for different aspect ratios would be better to avoid excessive padding. Also, the logic here is flawed. It clamps to target resolution even if it isn't a multiple of 32.

I think this is not ideal because using multiple shapes would further increase the memory usage. Currently, a graphics card with at least 5GB of VRAM is required to run it. If this continues, the requirements are too high, and many small servers might not be able to run it. Alternatively, we could consider using an environment variable to define this.

There would only be a handful of variations. I think the main ones would be 16:9, 1:1 and 9:16 vs just 1:1. The behavior on main allows any number of different box dimensions, so having a few shapes is still much better than current.

Yes, theoretically, this would improve the accuracy rate, but how should we verify it through practical testing? If possible, I'll test it this weekend.

The best way would be to see if/how the results change with different input images, comparing main and the PR.

Once you can confirm they're the same (or very close), you can check the README for testing with Locust. It simulates concurrent requests to the ML server, which lets us see both the latency/throughput impact of the change and how memory usage is affected. The locustfile.py only creates a random image though, so you might need to make it use a real image with text for the test. It depends on how far you want to go with it, but you can also make it use different images to test the aspect ratio handling as well (otherwise, it will only be testing a static input dimension).

I mean that the recognition rate currently appears normal, and changing the logic shouldn't have much impact.

The amount of padding affects speed (the less similar the image is to a square, the more compute is wasted on padding). On CPU, it would increase memory usage. I'm not too worried about accuracy since padding shouldn't change much there (assuming the padding value is actually correct).

From my testing. The current performance is normal. With an i5-10400F and a 1660 SUPER, it processed 14,107 photos in less than two hours.CPU usage is at 9%.The graphics card is running at full power.

mertalev · 2026-01-03T16:55:31Z

+        normal_indices = []
+        extreme_indices = []
+
+        EXTREME_WH_RATIO = 12.0


Similarly, it could box into aspect ratios in a cleaner way without hardcoded “normal” and “extreme” paths. It should probably also be padded to fixed aspect ratios rather than padding to any max, but it’s a bit tricky since RapidOCR handles part of the processing here.

Since the RapidOCR project was not primarily designed with ONNX in mind, some of its code is not suitable. If you don't want this to be the case, you might need to modify the RapidOCR library.The main problem here is each time you use RapidOCR, it only aligns with the maximum value of that instance. When it is called next time, it does not have the previous parameters, and the shape will change. Therefore, we must ensure that the input shape is the same each time.

savely-krasovsky · 2026-01-10T02:46:35Z

I tried this, but unfortunately memory still leaks during 10 minutes of running with Intel Arc Pro. In my case I think this is some ONNX Runtime leak though unfortunately.

aeoform · 2026-01-13T16:52:49Z

I tried this, but unfortunately memory still leaks during 10 minutes of running with Intel Arc Pro. In my case I think this is some ONNX Runtime leak though unfortunately.

Perhaps ONNX also has issues, as I encountered an unusual memory usage of about 500M during testing, but it was extremely rare, and I couldn't find a way to reproduce it. Could you share the problematic photo?

savely-krasovsky · 2026-01-13T16:57:32Z

I believe it's another issue which is even worsened by #23462. It just monotonically increases until both graphics and system memory will fill up and comes OOM errors from ONNX. Memory usage increases exponentially, slowly at first and then it explodes.

savely-krasovsky · 2026-01-14T11:57:31Z

@bb159

This is look like that.

aeoform · 2026-01-14T14:01:52Z

I see this as a steady growth, it should not be a problem with the image, but I still hope We can try using the Nvidia environment to see if my modifications only work for the CUDA environment. I only have a CUDA environment and cannot reproduce your issue.

savely-krasovsky · 2026-02-01T23:07:06Z

@bb159 FYI I attempted to fix myself recently: #25604 and it completely fixed issue in my case. Processed 150k images with constant memory load within about 570-620MB of VRAM usage.

bb159 and others added 2 commits January 3, 2026 21:26

无代码更改，提交空白

40e0bfc

aeoform requested a review from mertalev as a code owner January 3, 2026 14:23

immich-push-o-matic Bot added the 🧠machine-learning label Jan 3, 2026

aeoform mentioned this pull request Jan 3, 2026

ML OCR memory leaks? #23462

Closed

2 of 4 tasks

Merge branch 'main' into fix-ocr-gpu

dfa934d

mertalev requested changes Jan 3, 2026

View reviewed changes

bwees changed the title ~~Fix ocr gpu~~ fix: ocr gpu memory usage Feb 2, 2026

aeoform closed this Apr 3, 2026

Conversation

aeoform commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Checklist:

Please describe to which degree, if any, an LLM was used in creating this pull request.

Uh oh!

immich-push-o-matic Bot commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bo0tzz commented Jan 3, 2026

Uh oh!

bo0tzz commented Jan 3, 2026

Uh oh!

aeoform commented Jan 3, 2026

Uh oh!

mertalev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aeoform Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

savely-krasovsky commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aeoform commented Jan 13, 2026

Uh oh!

savely-krasovsky commented Jan 13, 2026

Uh oh!

savely-krasovsky commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aeoform commented Jan 14, 2026

Uh oh!

savely-krasovsky commented Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aeoform commented Jan 3, 2026 •

edited

Loading

immich-push-o-matic Bot commented Jan 3, 2026 •

edited

Loading

aeoform Jan 4, 2026 •

edited

Loading

savely-krasovsky commented Jan 10, 2026 •

edited

Loading

savely-krasovsky commented Jan 14, 2026 •

edited

Loading

savely-krasovsky commented Feb 1, 2026 •

edited

Loading