Added optional hyperparameter sweep support using Optuna. - #40
Added optional hyperparameter sweep support using Optuna.#40TahaZahid05 wants to merge 5 commits into
Conversation
|
@TahaZahid05 can you tell me how generated code will look like? |
| return self | ||
|
|
||
|
|
||
| class PruningConfig(BaseModel): |
There was a problem hiding this comment.
This class is super verbose, I think instead of us managing and handling every case, we could simply use a default pruning technique instead. If user wants he can edit the code.
|
|
||
| class ParameterRange(BaseModel): | ||
| model_config = ConfigDict(validate_assignment=True, extra="forbid") | ||
| type: Literal["int", "float"] |
There was a problem hiding this comment.
categorical can also be present
| n_trials: int | None = None | ||
| storage: str | None = None | ||
| pruning: PruningConfig = PruningConfig() | ||
| parameters: dict[str, list[Any] | ParameterRange] |
There was a problem hiding this comment.
A dict config will be bad here I think because for every parameter, there will be an extra nested arguments.
I think we should use a list here, with target here being consistent with other configs.
| - `trainer.py`: Defines the training and evaluation logic. You can override `train_step` or `eval_step` here. | ||
| - `config.py`: Contains Pydantic models for configuration validation. If you add new parameters to `config.yaml`, update the models here. | ||
| - `utils.py`: Shared utilities for configuration and logging. | ||
| {{sweep_project_structure}} |
There was a problem hiding this comment.
Sweeping is a completly extra feature, so I think we should generate a separate readme for it entierely.
|
Also about the metric_name, and other parameters, they will be unique to use case to use case right? hmm, I am not sure how would we generate them for different setup. I think you should consult victor about it, if we should even support sweeping or not as it was more of an prototype to check if it should be added or not. |
|
@aaishwarymishra should |
yes, I think the main functionality the sweeper should provide is the easy way to run multiple experiments with different values. we should not try to handle every possible combination of parameter like strategy, we provide default, user can change it however they want. This keeps code and yaml clean. |
29b6131 to
25ef6d9
Compare
2 participants
What's New
sweep.pyandsweep_utils.pysupportinggrid,random, andtpesearch strategies.--sweepflag totrainite initto generate a boilerplatesweep.yamlconfiguration, runner files, and addoptunadependency topyproject.toml.SweepConfigandParameterRangePydantic schemas to validate target parameters and their search spaces.tests/config/base_test.pyandtests/cli_test.py.