Skip to content

Jmafoster1/dag evaluation#401

Open
jmafoster1 wants to merge 8 commits into
mainfrom
jmafoster1/dag-evaluation
Open

Jmafoster1/dag evaluation#401
jmafoster1 wants to merge 8 commits into
mainfrom
jmafoster1/dag-evaluation

Conversation

@jmafoster1

@jmafoster1 jmafoster1 commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #393 #399 #391 #387.

  • Removes the MetamorphicRelation class in favour of generating causal tests directly.
  • Improved to_dict methods for easier serialisation.
  • Improved "generate and run tests" pipeline from DAG
  • Added EVALUATE command to evaluate a causal DAG against a dataset.

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Elapsed time
⚠️ PYTHON black 37 1 1.32s
✅ PYTHON pylint 37 0 7.25s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by OX Security

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.21339% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.46%. Comparing base (3e6fd99) to head (423f117).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
causal_testing/__main__.py 70.37% 8 Missing ⚠️
causal_testing/estimation/abstract_estimator.py 66.66% 3 Missing ⚠️
causal_testing/specification/causal_dag.py 96.87% 2 Missing ⚠️
causal_testing/testing/causal_effect.py 95.91% 2 Missing ⚠️
causal_testing/causal_testing_framework.py 96.00% 1 Missing ⚠️
...esting/estimation/abstract_regression_estimator.py 85.71% 1 Missing ⚠️
...ting/estimation/instrumental_variable_estimator.py 50.00% 1 Missing ⚠️
causal_testing/testing/causal_test_case.py 95.45% 1 Missing ⚠️
causal_testing/testing/causal_test_result.py 90.90% 1 Missing ⚠️
causal_testing/testing/data_adequacy.py 83.33% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #401      +/-   ##
==========================================
- Coverage   98.18%   96.46%   -1.72%     
==========================================
  Files          32       31       -1     
  Lines        1704     1754      +50     
==========================================
+ Hits         1673     1692      +19     
- Misses         31       62      +31     
Files with missing lines Coverage Δ
causal_testing/discovery/abstract_discovery.py 94.01% <100.00%> (-5.99%) ⬇️
causal_testing/discovery/hill_climber_discovery.py 98.24% <100.00%> (+0.03%) ⬆️
..._testing/estimation/linear_regression_estimator.py 100.00% <ø> (ø)
causal_testing/testing/base_test_case.py 100.00% <100.00%> (ø)
causal_testing/causal_testing_framework.py 99.26% <96.00%> (-0.74%) ⬇️
...esting/estimation/abstract_regression_estimator.py 95.08% <85.71%> (-1.22%) ⬇️
...ting/estimation/instrumental_variable_estimator.py 96.42% <50.00%> (-3.58%) ⬇️
causal_testing/testing/causal_test_case.py 96.77% <95.45%> (-3.23%) ⬇️
causal_testing/testing/causal_test_result.py 94.44% <90.90%> (-5.56%) ⬇️
causal_testing/testing/data_adequacy.py 92.85% <83.33%> (-7.15%) ⬇️
... and 4 more

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 355f04d...423f117. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jmafoster1
jmafoster1 force-pushed the jmafoster1/dag-evaluation branch from 024e592 to 77d0629 Compare July 21, 2026 15:58
@jmafoster1
jmafoster1 force-pushed the jmafoster1/dag-evaluation branch from 77d0629 to f8e7307 Compare July 22, 2026 06:59
@jmafoster1
jmafoster1 marked this pull request as ready for review July 22, 2026 07:21
@jmafoster1

jmafoster1 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

Codecov isn't happy here because the codebase has got smaller. I have removed quite a lot of code, and thus the tests that previously covered that code. If you try to inspect the difference in coverage, it says "No Files covered by tests were changed". Hopefully this will go away once #395 is in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate causal test cases directly from DAG

1 participant