Fix sklearn MDS API Compatibility Issues

[ainaadekunle]

Fix sklearn MDS API Compatibility Issues

Issue Type: Technical Debt / Maintenance

Problem

Our tests are failing due to frequent sklearn MDS API changes across versions:

• sklearn <1.3: metric="euclidean" parameter
• sklearn 1.3: dissimilarity="euclidean" parameter
• sklearn 1.4-1.9: metric=True/False parameter (deprecated dissimilarity)
• sklearn 1.10+: metric_mds=True/False parameter (deprecated metric=True/False)

Our current workaround uses dynamic API detection and warning suppression, which is fragile.

Current Workarounds

1. Exhaustive warning filters in pytest.ini
2. Dynamic parameter testing in src/fastmdanalysis/analysis/mds.py
3. Environment variable PYTHONWARNINGS=ignore in CI

Required Fixes

Short-term (High Priority):

• Pin sklearn to stable version in pyproject.toml
• Separate warning handling from error handling in analysis code
• Document compatible sklearn versions in README

Medium-term:

• Refactor MDS to use stable interface (adapter pattern)
• Add sklearn version matrix testing in CI
• Consider alternative to MDS (PCA, t-SNE, UMAP)

Long-term:

• Implement proper dependency management (pip-tools/poetry)
• Create version compatibility layer for sklearn
• Evaluate dropping MDS if API remains unstable

Technical Details

Problematic files:

• src/fastmdanalysis/analysis/mds.py - Complex API detection logic
• pytest.ini - Excessive warning filters (100+ lines)
• .github/workflows/test.yml - Warning suppression via env vars

Recommended sklearn version: scikit-learn==1.3.0 (last stable MDS API)

Testing

• Test with pinned sklearn version
• Remove warning filters and ensure clean test output
• Add sklearn version compatibility tests

Related Issues

• Pyparsing deprecation warnings also being caught as errors
• Error handling converts warnings to AnalysisError exceptions

Notes

• Temporary fix implemented in commit: [COMMIT_HASH]
• All Python versions currently passing with workarounds
• Risk: Workarounds may break with future sklearn releases