Skip to content

Conversation

@allrob23
Copy link

This PR introduces a dynamic random seed configuration in conftest.py that gets set at the beginning of each test session. This approach ensures controlled randomness while still allowing for diverse test scenarios across different test runs.

Current Issue (Before this PR):

The tests currently use random without any seed management, which could potentially lead to:
a. Flaky Tests: Tests might exhibit different behaviors across runs due to uncontrolled random values
b. Hard to Reproduce: When a test fails, it's difficult to reproduce the exact conditions
c. Inconsistent CI/CD: Different CI runs may have different outcomes
d. Time Wasted: Developers might spend time debugging issues that are hard to replicate

Solution

  • Added a session-scoped fixture in conftest.py that:
    • Generates a random seed (between 0-1000) at the start of each test session
    • Sets this seed as the global random seed
    • Prints the seed value for reference and reproducibility

Benefits

a. Controlled Randomness: Each test session uses a consistent set of random values
b. Reproducible Debugging: The printed seed value allows developers to reproduce any failed test session
c. Session-level Consistency: All tests within a single session use the same random seed
d. Transparency: The seed value is logged, making it clear which seed was used in any given run
e. Balance: Maintains randomness across different test runs while ensuring consistency within each session

Usage

When a test fails, developers can view the logged seed value:
Random seed used in this test session: {X}
This value can be used to reproduce the exact test conditions if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant