HPO config standardization

[Jasha10]

We need a separate issue tracking HPO config standardization.

Originally posted by @jieru-hu in #1886 (comment)

Status

• BasicSweeper — Support defining basic sweep through config file #1886
• Optuna — add hydra.sweeper.params to optuna #2031
• Nevergrad — Standardize Nevergrad sweeper search space config #3311
• Ax — Standardize Ax Sweeper parameters and remove the Ax upper bound #3313

[jieru-hu]

thanks @Jasha10 !

[jieru-hu]

More context on this issue:
In Hydra 1.2 we will standardize the syntax to configure the search space across all Hydra sweepers. All search space will be defined under hydra.sweeper.params like how it is working already with BasicSweeper

• BasicSweeper Support defining basic sweep through config file #1886
• Optuna add hydra.sweeper.params to optuna #2031
• Nevergrad
• Ax