Skip to content

Fix failing and slow testing suite#1380

Merged
ubdbra001 merged 5 commits into
AFM-SPM:mainfrom
ubdbra001:i1377-fix-testing-suite
Jul 16, 2026
Merged

Fix failing and slow testing suite#1380
ubdbra001 merged 5 commits into
AFM-SPM:mainfrom
ubdbra001:i1377-fix-testing-suite

Conversation

@ubdbra001

Copy link
Copy Markdown
Collaborator

Pin dependencies to fix failing and slow test suite.

  • matplotlib 3.11 produces different images to the baseline. Updating the baseline breaks tests for Python 3.10, so pinning this for now.
  • syrupy >= 5.5 slows down running the tests so reverting to a previous version that deosn't cause this issue

Both of these are plaster fixes, longer term solutions are required:

  • Baseline images should be updated, probably a job for when python 3.10 is dropped
  • The underlying issue for syrupy is likely the large snapshot for the nodestats tests, this should be investigated.

@ubdbra001
ubdbra001 requested a review from SylviaWhittle July 10, 2026 15:29
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.11%. Comparing base (86492cf) to head (98bc988).
⚠️ Report is 25 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1380   +/-   ##
=======================================
  Coverage   84.11%   84.11%           
=======================================
  Files          31       31           
  Lines        6447     6448    +1     
=======================================
+ Hits         5423     5424    +1     
  Misses       1024     1024           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ubdbra001 ubdbra001 linked an issue Jul 10, 2026 that may be closed by this pull request
8 tasks
@ns-rse

ns-rse commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

I've only had a cursory look at what tests are failing (spent the morning trying to recover what looks like a failed harddrive so on a temporary computer that isn't ideally configured).

You could try adding a marks=pytest.mark.skipif() clause to skip tests where images are changing so they don't run if Python != 3.10 for the time being, then update the images with newere Python/matplotlib/numpy. Or update the images and skip if on 3.10. You should be able to get this from environment variables in some manner.

I'd probably want to check what is changing in the images though but I'm afraid I don't have time to investigate that myself right now (perhaps add them to #1377 if they are readily available they will be under somewhere like /tmp/pytest/. You can also run specific tests and use the flag --mpl-generate-summary=html which is a useful way of getting an insight to what is changing.

@ns-rse ns-rse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps make it easier for people to find why the versions have been pinned.

Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
@ubdbra001

Copy link
Copy Markdown
Collaborator Author

To do:

  1. Skip testing for 3.10, and update figure snapshots
  2. Pin syrupy and add issue outlining why

@ubdbra001
ubdbra001 force-pushed the i1377-fix-testing-suite branch from 98bc988 to cd36f1f Compare July 15, 2026 15:27
ubdbra001 and others added 5 commits July 16, 2026 08:58
adds skip on 3.10 and updates baseline images
adds skip on 3.10 and updates baseline images
adds skip on 3.10 and updates baseline images
@ubdbra001
ubdbra001 force-pushed the i1377-fix-testing-suite branch from 8d7d986 to 9154da6 Compare July 16, 2026 07:59
@ubdbra001

ubdbra001 commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

Test are failing due to a bug introduced in #1376, looks like it is caught and fixed in #1382

Edit: Tests passing on my fork where I haven't added #1376 yet

@ubdbra001
ubdbra001 requested a review from tobyallwood July 16, 2026 08:46

@SylviaWhittle SylviaWhittle left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting that this changed too since it is only supposed to be a font change from what we were talking about. But I am not concerned.

@tobyallwood tobyallwood left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me I think

@ubdbra001
ubdbra001 added this pull request to the merge queue Jul 16, 2026
Merged via the queue into AFM-SPM:main with commit 5efa5e8 Jul 16, 2026
1 of 7 checks passed
@ubdbra001
ubdbra001 deleted the i1377-fix-testing-suite branch July 16, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]: TopoStats testing suite failing without apparent cause

4 participants