Dreamer 4

[nshoman]

Hafner and colleagues have released Dreamer 4. This seem to be quite a departure from Dreamer V3 as D4 is demonstrated on learning strictly from an offline dataset with a world model based on diffusion, among other significant changes.

Opening this issues as a point of discussion and to help scaffold an eventual implementation.

[nshoman]

Reading through the paper, the intended application of DV4 is quite different than prior models, so it might actually not be appropriate for this repo.

[belerico]

Hi @nshoman, I've just heard the news yesterday! I would love to implement it. Do you already have some ideas on how to do it? Especially since it's an offline algo, which was never supported in the first place here on sheeprl.

[belerico]

Moreover, I would definitely update the implementation of dv3 to the latest one also

[nshoman]

I'm not sure that DV4 belongs here in sheeprl -- I was surprised to see it was a fairly significant departure from DV3. I might have some support this year to help with the DV3 updates, but won't know for a few weeks. I've glanced at a few of the DV3-v2 changes and some should be fairly straightforward to implement (in theory).