Skip to content

Jmafoster1/388 fix large scenarios#395

Open
jmafoster1 wants to merge 8 commits into
mainfrom
jmafoster1/388-fix-large-scenarios
Open

Jmafoster1/388 fix large scenarios#395
jmafoster1 wants to merge 8 commits into
mainfrom
jmafoster1/388-fix-large-scenarios

Conversation

@jmafoster1

Copy link
Copy Markdown
Collaborator

Closes #388.
This has turned into a bit of a monster PR. Main changes as follows:

  • Estimators no longer store the dataframe. This dramatically reduces the amount of memory taken up causal test cases.
  • The CausalTestingFramework class is now a "first class citizen". It is designed and intended to be used as a standalone API rather than purely as part of the commandline interface.
  • CausalTestCases now have some extra instance attributes:
    • They now store the result of execute_test as an instance variable instead of returning it. This means you don't have to iterate a zip object of the tests and their associated results.
    • They now store the name and skip attributes from the JSON frontend. This means that there's now a 1:1 mapping between tests and their representation in JSON, meaning that we don't need to keep the parsed input around for later use as part of output.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Elapsed time
⚠️ PYTHON black 38 1 1.05s
✅ PYTHON pylint 38 0 5.06s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by OX Security

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.73262% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.18%. Comparing base (8f04b99) to head (3e6fd99).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
causal_testing/__main__.py 98.30% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
+ Coverage   98.15%   98.18%   +0.02%     
==========================================
  Files          31       32       +1     
  Lines        1841     1704     -137     
==========================================
- Hits         1807     1673     -134     
+ Misses         34       31       -3     
Files with missing lines Coverage Δ
causal_testing/causal_testing_framework.py 100.00% <100.00%> (ø)
causal_testing/discovery/abstract_discovery.py 100.00% <100.00%> (ø)
causal_testing/estimation/abstract_estimator.py 100.00% <ø> (ø)
...esting/estimation/abstract_regression_estimator.py 96.29% <100.00%> (-0.48%) ⬇️
...ausal_testing/estimation/cubic_spline_estimator.py 97.29% <100.00%> (-0.08%) ⬇️
...ting/estimation/instrumental_variable_estimator.py 100.00% <100.00%> (ø)
causal_testing/estimation/ipcw_estimator.py 99.26% <100.00%> (ø)
..._testing/estimation/linear_regression_estimator.py 100.00% <100.00%> (ø)
...esting/estimation/logistic_regression_estimator.py 100.00% <100.00%> (ø)
...ing/estimation/multinomial_regression_estimator.py 100.00% <100.00%> (ø)
... and 8 more

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b5b62a...3e6fd99. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jmafoster1
jmafoster1 marked this pull request as ready for review July 17, 2026 07:37
@jmafoster1
jmafoster1 requested a review from jwhogg July 20, 2026 13:45
Comment thread causal_testing/__main__.py Outdated
Comment on lines +196 to +197
estimator_map = {ff.name: ff for ff in entry_points(group="estimators")}
effect_map = {ff.name: ff for ff in entry_points(group="causal_effects")}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How big is this list comp? is it efficient to be using a dict here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're both pretty small, like 5 elements. The entrypoints are defined in the corresponding entries in pyproject.toml. What would you propose to use instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large DAGs/data take a prohibitive amount of memory

2 participants