Huge memory usage

[PyXiion]

I've been testing libcoro with the latest updates, but I've encountered a huge memory usage.

Results based on tzcnt/runtime-benchmarks#11 (peak memory usage tracking PR)

Benchmark	libcoro	libfork	TooManyCooks	tbb	taskflow	cppcoro	coros	concurrencpp
skynet	20.62 GB	15.97 MB	15.97 MB	15.87 MB	15.87 MB	72.15 MB	15.87 MB	15.87 MB
nqueens	5.37 GB	15.97 MB	15.97 MB	15.87 MB	15.87 MB	72.25 MB	15.87 MB	15.87 MB
fib(39)	17.02 GB	15.97 MB	15.97 MB	15.87 MB	15.87 MB	72.3 MB	15.87 MB	15.87 MB
matmul(2048)	60.93 MB	59.66 MB	57.09 MB	64.79 MB	57.71 MB	130.18 MB	56.79 MB	56.59 MB
io_socket_st	15.97 MB	N/A	16.34 MB	N/A	N/A	15.87 MB	N/A	N/A

Environment:

• OS: Manjaro Linux x86_64
• Kernel: Linux 6.12.64-1-MANJARO
• CPU: AMD Ryzen 7 8845H (16) @ 5.10 GHz
• Memory: 27.21 GiB

heaptrack graph shows that coroutine creating consumes the most memory

[jbaldwin]

woof, those don't look great 😅 I think this likely gets back to how tasks are passed into thread_pool and likely needs a complete re-work. I also wonder if the task class needs to possibly be more aggressive on cleaning up the coroutine frames, not just in its destructor? I'm wondering how the other libs do it (I haven't really checked).

edit: I'm wondering if tasks in the other projects basically are always task_self_deleting, that could explain at least the coroutine frame memory explosion. I think getting of the stdlib constructs is another issue too.

[jbaldwin]

@PyXiion would like like to open a PR that adds these programs as examples so we can run them consistently and work towards getting libcoro inline with the other projects?

[tzcnt]

Hi, sorry that you were bothered about this twice. It's the same benchmark suite that I pinged you about prior: #404

The fundamental issue is the effect that a FIFO task queue has on a recursive fork-join benchmark like this. With FIFO it results in a BFS of the task graph, so for skynet(8) all 100,000,000 leaf node tasks would be created before any can be completed. This is why most work stealing architectures treat the local thread queue as a LIFO stack. This does a DFS on the task graph, which for skynet(8) limits the depth of the local thread queue to 72.

While threads process their local queue as a LIFO stack, when stealing, they should steal from the other end (as a FIFO queue) which in this benchmark means that they are not stealing leaf node tasks, but stealing parents higher up the tree, representing a larger piece of overall work.

This is the primary root cause of both the poor performance, and the memory blowup. That's why I suggested previously a simple fix of using mutex-locked thread-local deques, which will resolve the algorithmic inefficiency issue. Optionally transitioning after that to a lock-free data structure would only be an improvement to the constant-time part.

[PyXiion]

Sorry for inconvenience, I wanted to make it a separate issue, because last updates improved libcoro performance, but not its memory usage 😅