Add global random seed for deterministic tests #2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a dynamic random seed configuration in
conftest.pythat gets set at the beginning of each test session. This approach ensures controlled randomness while still allowing for diverse test scenarios across different test runs.Current Issue (Before this PR):
The tests currently use random without any seed management, which could potentially lead to:
a. Flaky Tests: Tests might exhibit different behaviors across runs due to uncontrolled random values
b. Hard to Reproduce: When a test fails, it's difficult to reproduce the exact conditions
c. Inconsistent CI/CD: Different CI runs may have different outcomes
d. Time Wasted: Developers might spend time debugging issues that are hard to replicate
Solution
conftest.pythat:Benefits
a. Controlled Randomness: Each test session uses a consistent set of random values
b. Reproducible Debugging: The printed seed value allows developers to reproduce any failed test session
c. Session-level Consistency: All tests within a single session use the same random seed
d. Transparency: The seed value is logged, making it clear which seed was used in any given run
e. Balance: Maintains randomness across different test runs while ensuring consistency within each session
Usage
When a test fails, developers can view the logged seed value:
Random seed used in this test session: {X}This value can be used to reproduce the exact test conditions if needed.