Suggestion: store decoded dataset images in RAM one time only instead of before every epoch

[LeMinhNgan]

Search before asking

• I have searched the RF-DETR issues and found no similar feature requests.

Description

Hello all,

I checked and understood that the dataset images are read, decoded at every training epochs. This pipeline causes long training time and may deteriorate HDD as well (i guess my case).

Improvement: if we have sufficient RAM as my case, we should load the dataset one time only, decode and store them in RAM for the training process --> make the training faster

My suggestion:

1. Make a new training parameter called use_ram_cache:bool to let the user decide if they want to use RAM as a cache to load and store the dataset onetime only (set True) or keep reading dataset before every epoch as the current way (set False).
2. Live monitor free memory of RAM in rich progress bar

src/rfdetr/training/callbacks/gpu_memory_progress_bar.py:

import psutil                                #for RAM calculation

def get_metrics(self, trainer: Trainer, pl_module: LightningModule) -> _Metrics:
    ...
        free_ram= f"{(psutil.virtual_memory().available / 1024**3):.2f}/{(psutil.virtual_memory().total / 1024**3):.2f} GB_RAM"
    ...

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[atikulmunna]

interested in implementing this if you want it. a few questions first, since the answers change the shape quite a bit.

where the cache sits. decoding happens in torchvision's CocoDetection._load_image, which the rfdetr subclass reaches through super(). overriding _load_image keeps the whole thing at the dataset layer and touches nothing else. is that where you'd want it?

what gets cached. decoded arrays are fastest but expensive: a 640x640 rgb image is ~1.2 mb in ram against maybe 100 kb on disk, so 50k images needs ~60 gb. caching the encoded bytes instead kills the disk reads, which is the hdd wear this issue is about, at roughly a tenth of the memory, and still pays decode each epoch. fast one, cheap one, or both behind the same flag?

workers. with num_workers > 0 each worker gets its own copy, so memory multiplies by worker count unless the cache is populated before fork or lives in shared memory. worth settling early because it decides whether a "do i have enough ram" check means anything.

the flag. is use_ram_cache: bool the shape you want, or would you rather it enables itself when the dataset fits in some fraction of free ram?

on the ram readout in the progress bar, psutil isn't a dependency today. happy to leave that out of a first pr so this doesn't pull one in, unless you want it.

can run before and after numbers on a real training run if that helps you decide.

[LeMinhNgan]

@atikulmunna very appreciate that you are interested in my suggestion. I'd like to confirm my points based on your questions

1. where the cache sits: Yes, keep the cache at the dataset layer by overriding _load_image.
2. what gets cached: i prefer decoded cache, it's expensive but fast and gives a considerable difference (at least my case, i have 192GB RAM). Since this use_ram_cache:bool is a configurable, users can set it False and back to the current pipelines if they find they RAM is not enough.
3. workers: i feel num_workers=0 has less bugs and prefer it.
4. the flag. is use_ram_cache: bool. Could we make the option use_ram_cache: Literal[True, False, "auto"], with False as the default? True would force caching, False would disable it, and "auto" would enable it only when the estimated cache size fits within a conservative RAM budget.
5. psutil: it is not a big deal, i can handle it myself or leave it for other suggestion, you can ignore it.

Again, thank your kind attention to my suggestion