Avoid floating point comparison issues#950
Conversation
... to avoid floating point comparison issues
Luismiguelvillar
left a comment
There was a problem hiding this comment.
It looks good to me. I just have a question regarding the chosen precision. Is there a specific reason for rounding to 6 decimal places?
deco_dst.E = np.round(deco_dst.E, 6)
|
Not a fundamental reason, but a practical one. Rounding to more digits produced discrepancies between my computer and GHA. 6 digits seems enough to maintain precision while avoiding these issues. |
|
|
||
|
|
||
| @given(bunch_of_hits) | ||
| def test_round_hits_positions_in_place(hits): |
There was a problem hiding this comment.
Should we also test how the function behaves with edge cases such as NaNs, infinities, or empty hit lists?
Makes sense! But maybe it will be a good idea to let the user change this precision, similar to the xyz in hits? |
Luismiguelvillar
left a comment
There was a problem hiding this comment.
Everything looks fine!
|
@mcidlaso please merge |
Some tests that check for an "exact result" have been problematic because they rely on floating point comparisons. This is machine-dependent, which means a test may pass locally, but fail on GHA or viceversa. With direct comparisons, this is usually addressed by adding a tolerance, which we already do. However there are other cases where this isn't enough
In this PR, we bypass all these issues by rounding values to a safe number while maintaining enough precision.