Duplicate Finder gets stuck on Step 2

[Baine]

Greetings :)

I'm using LRReader 1.8.10 with Lanraragi Server 0.8.90 on Windows 11 (22621.2134).

While executing the Duplicate Finder the progress always gets stuck around halfway through (see image below)

Server is hosted remotely on a Ubuntu 23.04 host via docker, all files are directly accessible for the Server Process through the filesystem (no NFS/CIFS etc).

Is there any way for me to get this to continue/finish?

Thanks in advance :)

[Guerra24]

Hello!

This is a difficult one... Since no errors are shown, I will guess LANraragi is locking up due to the large amount of requests, try increasing the delay per thread to something like 50 or 100.

Another thing that may be happening is that you don't have thumbnails for all archives and that is slowing down everything.

Does it always stop at 29115?

If you wait, does it continue in any way?

[Baine]

it first started to stop around 17k
I've recreated the library and ran a force generate thumbnail,
and after that it's now stuck at that point (and always stops at that count when retrying)

I'll increase the delay and report back.

thanks a lot already!

Edit: Made some runs with varying delay (and also enabled skipping missing thumbnails), and the point were processing stopped has changed for each run now. 29117, 29119, now sitting at 29124 (50ms delay).
interestingly enough cpu, memory and network are changing, so it doesn't look like it's frozen.

If it helps, the image cache size displayed in LRReader is 1,436.25MB and didn't change during the runs,
while the thumbs dir on the server has 2,6G (used du -sch).

Is there some log that I can check if it gets stuck on the same file or something along that line?

[Baine]

halved the threads, now stuck at ~7k

[Guerra24]

Sorry for taking so long to respond, I've added logging to it, and it's on the nightly build you can download from here.

The logs should be here:
C:\Users\<user>\AppData\Local\Packages\Guerra24.LRReader_3fr0p4qst6948\LocalCache\Logs

[Baine]

thanks for adding that :)

I've ran the duplicate finder on a smaller sample, and it was working fine. my suspicion here is, that lanraragi fails to process some archives (for whatever reason), hence generates no thumbnail (or invalid? have to dig more) and fails to get processed by your tool. I've tried setting the option to ignore missing thumbnails, but still got stuck.

I'll report back once I've got more reliable information.