[WIP] Add Deepcache

[rmatif]

This PR is currently in progress and far from complete. It adds DeepCache, a method that can be applied to U-Net architectures to skip certain blocks and reuse them in later steps in order to save compute time.

I have been inspired by this ComfyUI implementation.

It adds --deepcache interval,depth,start,stop arguments.

Currently, it's not working well and I can't figure out why or how to achieve better results. I have been debugging the cache step and counter logic for a week, but the issue seems to be more subtle than that.

Command example:

./build/bin/sd -m ../models/realisticVisionV60B1_v51HyperVAE.safetensors -v -p "cute cat" --cfg-scale 2.5 --steps 8 --deepcache 2,3,0,8

w/o deepcache	--deepcache 2,3,0,8	--deepcache 3,3,0,8

If someone could help by taking a look or continue the work, I would be grateful. Otherwise, I don't think I'll spend more time on it.

[FSSRepo]

I'm very interested in this PR; I wish I had time to test DeepCache in Comfy UI and compare the results with your PR.

[rmatif]

@FSSRepo Thanks for your interest! Here's a comparison with ComfyUI, using the same model and parameters as above.

w/o deepcache	interval = 2, depth = 3, start = 0, stop = 8	interval = 3, depth = 3, start = 0, stop = 8

The results are so much better in ComfyUI compared to what I’m getting. My implementation doesn’t seem to work without CFG, which is really odd since DeepCache is supposed to be CFG-agnostic. I’m definitely doing something wrong. I'd love to continue working on this, but I’ve run out of ideas. It would be great if you could take a look and share any feedback!

[stduhpf]

My implementation doesn’t seem to work without CFG, which is really odd since DeepCache is supposed to be CFG-agnostic.

My guess is that it's sharing the same cache between uncond and conditioned pass and it's probably not supposed to.

[rmatif]

My implementation doesn’t seem to work without CFG, which is really odd since DeepCache is supposed to be CFG-agnostic.

My guess is that it's sharing the same cache between uncond and conditioned pass and it's probably not supposed to.

I tried to create a separate cache for conditional and unconditional passes, but it broke things even more. In any case, I think we should fix things with CFG first before addressing the CFG-free issue, don't think those are related