Combine CatCMAwM and LRA-CMA

[rrraaaccc]

Thanks again for you great work, just another question here.

Is it possible to combine the advantages of CatCMAwM and LRA-CMA?

As far as is see, this is currently not supported by the library, but from my point of view it would be a great enhancement.

Best regards, Nils

[nomuramasahir0]

Hi Nils, thank for you the question.

At the moment, we do not have a clear answer.

Both LRA-CMA and CatCMAwM already use learning rate adaptation, although in different contexts, so it is useful to clarify this first.
In LRA-CMA, learning rate adaptation is introduced as a mechanism to adaptively obtain smaller learning rates than the default ones, mainly to handle multimodal problems and noisy optimization.
In contrast, in CatCMAwM, learning rate adaptation is used to align the update speeds between the continuous/integer part (modeled by a multivariate Gaussian distribution) and the categorical part (modeled by a categorical distribution).

From a technical point of view, it may be possible to introduce learning rate adaptation in the sense of LRA-CMA into CatCMAwM.
However, we have not yet investigated how this would interact with the learning rate adaptation that is already built into CatCMAwM, nor what kind of effects it would have in practice.

Overall, combining these two mechanisms would likely require careful analysis and design, and we believe this would constitute a new research topic rather than a straightforward extension.