-
Notifications
You must be signed in to change notification settings - Fork 393
Refactor on DoMINO utils.py
to improve code quality
#985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Note: this PR should ideally be merged after #973; however, the changes are largely orthogonal between these two, so there should be minimal merge issues. |
…emoves outdated examples.
…arameter for adjustable sampling bias; updates docstring with detailed explanation and examples.
…mples in calculate_center_of_mass, standardize, nd_interpolator, pad, and pad_inp functions; adjusts k-nearest neighbors parameter in nd_interpolator for flexibility; corrects boolean checks in pad and pad_inp examples.
/blossom-ci |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for cleaning this up so extensively, @peterdsharpe . I have a few nits that will make our lives easier long term, but overall this looks great.
- You have introduced a hard dependency on CUPY, which may be fine or maybe we want to avoid. If we do it, let's put it in requirements explicitly. (Its already there in the cuml dep)
- Cupy has a built in array selection function - let's use that if we're requiring cupy and not have a wrapper.
- You have made unit tests in the docstrings. That's 98% of the way to unit test in the test suite too, can we do that? They will both actually get tested, but it will help raise the test coverage bar a little.
- For PyVista and VTK, what about factoring this utils.py file into two? One for pure-array and pure-python functions, and one which has an additional dependency on PV and VTK? This will allow more usage without those deps, which I think aren't needed once you have a preprocessed training set. As it stands, I think if someone handed me a curator-processed data set and I wanted to train DoMINO, I'd need VTK just to import some of these functions.
This commit introduces a new test file `test_domino_utils.py` that includes comprehensive unit tests for various functions in the domino utils module. Each test verifies the functionality of the corresponding utility function using examples from the documentation, ensuring correctness and reliability.
Tests added: daac91a |
…imize area_weighted_shuffle_array for consistent array handling. Remove redundant test for array_type.
CuML hard-dependency removed: d2b11c7 |
… avoid unnecessary dependency loading.
/blossom-ci |
PhysicsNeMo Pull Request
Major changes:
pathlib.Path
overos.path
for file path management (preferred since Python 3.4) [1, 2]from {package} import *
, which is not necessary here and should essentially-never be used.requirements.txt
, to facilitate reproducible traininglist
instead offrom typing import List
; use|
rather thanUnion
; preferType | None
toOptional
)black
formatting toutils.py
, to bring it into compliance with repository-wide pre-commit hooks requirements. (Note: all future PRs should comply with this per CI, and we should not allow PRs to merge that bypass this check.)Changes not yet implemented in this PR; these should be implemented in a future PR:
utils.py
itself has no unit test coverage (and is only indirectly tested by the DoMINO forward pass test).pv.wrap()
, which would reduce verbosity and help long-term maintainability.model.py
, fixing instances that dotorch.cat()
on each iteration of Python-levelfor
loops to incrementally append to tensors (we should instead build up alist[torch.Tensor]
tocat
all-at-once, or vectorize)Description
Checklist
Dependencies