Skip to content

Add suite creation and jedi config render tests#751

Merged
mranst merged 52 commits intodevelopfrom
feature/mranst/code_tests
May 8, 2026
Merged

Add suite creation and jedi config render tests#751
mranst merged 52 commits intodevelopfrom
feature/mranst/code_tests

Conversation

@mranst
Copy link
Copy Markdown
Collaborator

@mranst mranst commented Mar 26, 2026

This PR adds two code tests discussed in #736. The first simply creates all suites to ensure they are configured correctly, and do not contain defer_to_platform or defer_to_model.

The second test renders the jedi config yaml for all JEDI suites and evaluates differences to a set of stored files. These are constructed in a dry-run mode, where observations are not fetched and all filepath prefixes are replaced by placeholders. The idea behind this test is to ensure all configs are able to be successfully constructed, and also can be used to evaluate changes to jedi configs, as any change will be reflected in the file diff as part of a PR. Any change to a jedi yaml is expected to trigger a failure in this test. To make it easy to account for changes, I have introduced a script that can be run using swell utility CreateMockConfigs that will automatically regenerate all of the configs used for comparison in the source code.

I have implemented these tests as part of the regular code testing, as they add only about 20 seconds to the runtime of the code tests, but we can potentially look at implementing them in a different way, such as part of a new workflow

Comment thread src/swell/deployment/create_experiment.py Outdated
Comment thread src/swell/tasks/run_jedi_variational_executable.py Outdated
Comment thread docs/code_tests/code_tests.md
Comment thread src/swell/test/code_tests/jedi_config_test.py Outdated
Comment thread docs/code_tests/code_tests.md Outdated
Comment thread src/swell/test/code_tests/code_tests.py Outdated
Comment thread src/swell/utilities/scripts/create_mock_configs.py
Comment thread src/swell/utilities/scripts/create_mock_configs.py
Comment thread pyproject.toml
@mer-a-o
Copy link
Copy Markdown
Contributor

mer-a-o commented Apr 9, 2026

@mranst I'm testing this branch, I'm getting this:
SwellCreateExperiment: Creating experiment: 'swell-ufo_testing' in '/gpfsm/dnb33/tdirs/login/discover36.174920.mabdiosk/tmpeqil1jlx' in the output. I also see directories and files generated in test_cache_location: /discover/nobackup/mabdiosk/SwellExperiments/swell-test-cache which is iset in my ~/.swell/swell-test.yaml

@mranst
Copy link
Copy Markdown
Collaborator Author

mranst commented Apr 9, 2026

@mranst I'm testing this branch, I'm getting this: SwellCreateExperiment: Creating experiment: 'swell-ufo_testing' in '/gpfsm/dnb33/tdirs/login/discover36.174920.mabdiosk/tmpeqil1jlx' in the output. I also see directories and files generated in test_cache_location: /discover/nobackup/mabdiosk/SwellExperiments/swell-test-cache which is iset in my ~/.swell/swell-test.yaml

Thanks, they should both now go to the test cache

mer-a-o
mer-a-o previously approved these changes Apr 10, 2026
Copy link
Copy Markdown
Contributor

@mer-a-o mer-a-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's working correctly now. Thanks @mranst !

Comment thread src/swell/test/jedi_configs/jedi_3dvar_cf_config.yaml
@mranst mranst merged commit 9a6d2af into develop May 8, 2026
2 checks passed
@mranst mranst deleted the feature/mranst/code_tests branch May 8, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants