Skip to content

build(deps): extend Python support to 3.14 and relax numpy to >=2,<3 - #189

Open
laserkelvin wants to merge 26 commits into
NVIDIA:mainfrom
laserkelvin:exp-numpy2-python315
Open

build(deps): extend Python support to 3.14 and relax numpy to >=2,<3#189
laserkelvin wants to merge 26 commits into
NVIDIA:mainfrom
laserkelvin:exp-numpy2-python315

Conversation

@laserkelvin

@laserkelvin laserkelvin commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

ALCHEMI Toolkit Pull Request

Updated after #170 merged into main (squash 8ee5190): this branch
merged main in, so the diff below now shows only this PR's own changes
(pyproject.toml, uv.lock, CHANGELOG.md, docs/dev/python-315-internal.md).
The stacked history is preserved via the uv-lock-bump merge commit.

Description

Extends supported Python versions and relaxes numpy for nvalchemi-toolkit:

  • requires-python goes from >=3.11,<3.14 to >=3.11,<3.15: Python 3.14 is now fully supported (core package, cu12/cu13 CUDA extras, uma/uma-cu12/uma-cu13 extras). Python 3.15 is deliberately not publicly supported yet (upstream wheels missing); internal 3.15 development is enabled by the recipe in docs/dev/python-315-internal.md.
  • numpy<2.4 becomes numpy>=2,<3 — no code changes needed; the lock resolves numpy 2.5.3 for the main envs (2.4.6 where fairchem's numpy<2.5 cap applies in the uma fork). Downstream users are free to use any numpy 2.x.
  • The cu12/cu13 extras are slimmed to PhysicsNeMo core (CUDA torch + nvalchemi-toolkit-ops + cuequivariance-ops-torch + physicsnemo core): the RAPIDS stack (cuml, cupy, pylibraft, DALI) is removed — nothing in nvalchemi imports it, and its upstream pins made pure-pip installs unresolvable on 3.14. Users needing RAPIDS install it directly. This is the PR's one breaking change (install surface only), documented in the CHANGELOG.
  • Python 3.14 support in the CUDA extras is enabled by raising the RAPIDS floors (cuml-cuXX>=26.4.0, the cp311-abi3 line with official 3.14 support, RSN 58) and overriding nvidia-physicsnemo[cuXX]'s cupy-cuda1Xx<14 pin (cupy <14 has no cp314 wheel and no sdist).
  • The uma extras work on 3.14 via a per-Python lmdb override pair (fairchem pins lmdb<=1.7.3, which has no cp314 wheel; nothing in our fairchem usage touches lmdb). Below 3.14 fairchem's original pin is re-stated explicitly so the override doesn't drop the dependency.
  • Caps plotext<6: plotext 6 removed clf(), which hooks/reporting and training/cli call — any fresh resolve would have installed 6.x and crashed the Rich dashboards and training CLI. The lockfile only kept 5.3.2 by resolution stickiness, so this was a latent breakage on every Python version.

Type of Change

  • Bug fix (non-breaking change that fixes an issue) — the plotext cap
  • New feature (non-breaking change that adds functionality) — Python 3.14 support
  • Breaking change — cu12/cu13 extras drop the RAPIDS stack (install surface; no API changes)
  • Performance improvement
  • Documentation update — internal 3.15 dev recipe
  • Refactoring (no functional changes)
  • CI/CD or infrastructure change

Related Issues

None — exploratory dependency-bounds work. Follow-ups tracked on the team board: nvalchemi-toolkit-ops metadata + lazy torch.compile, cu13 runtime testing, cuequivariance-ops floor bump.

Changes Made

  • Extend requires-python to >=3.11,<3.15; relax numpy to >=2,<3
  • Slim cu12/cu13 extras to PhysicsNeMo core; drop the RAPIDS stack and the (now-unnecessary) cupy overrides (breaking, CHANGELOG'd)
  • Keep the lmdb per-Python override pair for fairchem on 3.14
  • Gate unused dev-dep rdkit to python_version < '3.15' (no cp315 x86_64 wheel; zero imports in the codebase)
  • Refresh lockfile: numpy 2.5.3 (main envs), RAPIDS removed, 382 → 324 packages — re-locked on top of fix: update vulnerable dependencies and isolate UMA setuptools #170
  • Cap plotext<6 (6.x removed clf(), used by hooks/reporting and training/cli)
  • Add docs/dev/python-315-internal.md: validated internal 3.15 dev recipe
  • CHANGELOG entries for the above

Testing

  • Unit tests pass locally — full suite (4,140 passed / 0 failed each) on both Python 3.13.11 and Python 3.14.7, cu12 stack on an L4, on the final slimmed state of this PR (torch 2.14.0+cu126, numpy 2.5.3, RAPIDS-free — verified with and without RAPIDS in an isolated 3.14 env)
  • Linting passes — pre-commit hooks (ruff, license, markdownlint, whitespace) passed on all commits; full make lint not run locally
  • New tests added — N/A: dependency-bounds change; the existing suite is the regression net. Isolated-venv validation covered the uma extra on 3.14 (33 UMA tests green with the lmdb override)

Also verified by dry-run resolution on the merged state: cu12/cu13 extras resolve on 3.11–3.14; uma resolves on 3.13 (lmdb 1.7.3, fairchem's pin preserved) and 3.14 (lmdb 2.3.0); the new uma-cu12 extra resolves on 3.14 (fairchem 2.22.0 + lmdb 2.3.0 + torch 2.13.0+cu126).

Checklist

  • I have read and understand the Contributing Guidelines
  • I have updated the CHANGELOG.md
  • I have performed a self-review of my code
  • I have added docstrings to new functions/classes — N/A, no new code
  • I have updated the documentation (if applicable) — docs/dev/python-315-internal.md

Additional Notes

  • Pure-pip installs now work on 3.11–3.14: the RAPIDS removal (and physicsnemo core instead of physicsnemo[cuXX]) eliminates the cupy<14 pin that made pip install nvalchemi-toolkit[cu12] unresolvable on 3.14 — verified: pip resolves the slimmed cu12 set on 3.14 in ~90s. [tool.uv] override-dependencies entries that remain (lmdb) are uv-only and only tighten uv's lock; pip users are unaffected.
  • The uv overrides are pairs where upstream pins must survive on older Pythons: a marker-scoped override alone replaces the requirement outright and would silently drop the dependency below the marker (this bit us with lmdb during development).
  • Known gaps, deliberately left for follow-ups: cu13 runtime is resolution-verified but untested on CUDA-13 hardware; nvalchemi-toolkit-ops still declares requires-python <3.15 (metadata-only cap; a sibling release is queued to relax it); CUDA extras cannot install on 3.15 until upstream (cupy/RAPIDS/cuequivariance) ships cp315 wheels.
  • fix: update vulnerable dependencies and isolate UMA setuptools #170's dev-group cleanup (hypothesis/hypothesis-torch removal) absorbs cleanly into this branch — full suite collects and passes on the merged state.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Gate the RAPIDS stack (nvidia-physicsnemo[cuXX] extra, cuml-cuXX) to
python_version < '3.14' in both cu12 and cu13 extras: RAPIDS publishes
no cp314/cp315 wheels (cuml-cu12 per-interpreter wheels; cuml-cu13 abi3
but its cupy-cuda13x dep lacks cp314), which otherwise blocks universal
resolution with the 3.15 ceiling. Core physicsnemo remains on 3.14+.

Resolution status: cu12/cu13 resolve on 3.11-3.14 (numpy pins to 2.2.6
under the 3.11 floor; 2.3+ needs >=3.12 upstream). 3.15 blocked by dev
dep rdkit (cp314 max) until new wheels ship.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
…verride

RAPIDS supports Python 3.14 from 26.4.0 via cp311-abi3 wheels (RSN 58),
so the python_version gates were wrong. The real 3.14 blocker was
nvidia-physicsnemo[cuXX]'s cupy-cuda1Xx<14 pin (cupy <14 has no cp314
wheel and no sdist); override it to >=14.0.1,!=14.1.0, matching cuml
26.8's own requirement. Both extras now resolve their full stack on
3.11-3.14; on 3.14 the cu12 fork picks torch 2.14.0+cu126.

rdkit stays gated <3.15 (2026.3.6 ships cp315 only for aarch64/mac) and
is unused in the codebase. 3.15 remains blocked upstream by tensordict
(no cp315 wheel or sdist in any release), triton, and cupy.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
The 2.2.6 pin was lockfile stickiness from the old numpy<2.4 spec, not a
constraint of the 3.11 floor. 2.4.6 supports >=3.11 with cp311-cp314
wheels, so 3.14 now gets a wheel instead of an sdist build. Full suite
green on 3.13 and 3.14 with 2.4.6.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
fairchem-core pins lmdb<=1.7.3, which has no cp314 wheel. lmdb>=2.0.0
ships cp314 wheels and nothing in our fairchem usage touches lmdb, so
override the pin from 3.14 on. Below 3.14, re-state fairchem's original
pin explicitly: a marker-scoped override alone replaces the requirement
outright, which would drop lmdb entirely on 3.13 and silently break the
uma env. Verified in isolation on 3.14.7: fairchem imports, 33 UMA
wrapper/equivalence tests pass with lmdb 2.3.0.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
hooks/reporting/layouts/base.py and training/cli.py call plt.clf(),
removed in plotext 6. The lock kept 5.3.2 only by resolution stickiness;
any fresh resolve (uv lock --upgrade, pip) would install 6.x and crash
the Rich dashboard and training CLI. Found during the Python 3.15
exploration (plotext 6.1.0 broke the same paths there).

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
Validated on 3.15.0rc1 (L4, cu126): core stack runs — ~3,554 tests and
a GPU dynamics smoke pass with tensordict built from source, torch
2.14.0+cu126 cp315 wheels, and an eager torch.compile shim. CUDA extras
stay 3.14-max until upstream ships cp315 wheels.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
…on315

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>

# Conflicts:
#	pyproject.toml
#	uv.lock
Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>

# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
#	uv.lock
@laserkelvin
laserkelvin marked this pull request as ready for review September 11, 2026 19:55
@laserkelvin

Copy link
Copy Markdown
Collaborator Author

/ok to test eb282e8

@greptile-apps

greptile-apps Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Retrigger

The PR appears safe to merge; no outstanding blocking or non-blocking code issue was established.

Summary

  • Replaces dm-tree’s single mapping use with a local recursive mapper.
  • Makes the standard CUDA extras pip-resolvable by removing their unused RAPIDS and PhysicsNeMo CUDA-extra dependencies.
  • Adds Python-specific LMDB overrides for UMA and caps plotext below its incompatible 6.x release.
  • Refreshes installation and model documentation plus the lockfile.

Reviews (4) · Last reviewed commit: "build(deps)!: slim cu12/cu13 extras to P..."

Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
dm-tree served a single call site (DataMixin.apply's map_structure over
model_dump output) and nothing else in the lock required it; tensordict
dropped it as a dependency years ago. The lock pinned 0.1.9, which has no
cp314 wheel, so CI's 3.14 cell tried an sdist build that needs cmake —
absent on the runners — and failed at the install step. Replacing the
call with a small dict/list/tuple mapper removes the compiled dependency
entirely (also a Python 3.15 blocker, no cp315 wheel exists).

test/data: 823 passed each on 3.13 and 3.14.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
3.15 is not publicly supported yet: tensordict, cupy, and triton have no
cp315 artifacts, so ordinary installs cannot resolve. Keep the published
range honest at >=3.11,<3.15 (also avoids <=3.15's trap of excluding
3.15.1+ patch releases); internal 3.15 development continues via the
recipe in docs/dev/python-315-internal.md. Addresses Greptile P1 on
unsupported 3.15 metadata.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
@laserkelvin

Copy link
Copy Markdown
Collaborator Author

/ok to test 53aafd7

…stack

Nothing in nvalchemi imports cuml, cupy, pylibraft, or DALI; the RAPIDS
stack entered via physicsnemo's [cuXX] extras and our duplicate cuml
lines, whose upstream pins (cupy<14, pandas<2.4, numba caps) made
'pip install nvalchemi-toolkit[cuXX]' unresolvable on Python 3.14 and
required uv-only cupy overrides. The extras now provide the CUDA torch
build, toolkit-ops, cuequivariance-ops-torch, and physicsnemo core;
users needing RAPIDS install it directly. Verified in an isolated 3.14
env: full import surface plus test/models + test/data green with and
without RAPIDS; pure-pip resolution of the slimmed cu12 set succeeds.
Lock: 382 -> 324 packages (numpy 2.5.3 main / 2.4.6 under fairchem's
numpy<2.5 cap in the uma fork).

Also aligns the install/UMA docs with the slimmed extras, documents the
breaking change in the CHANGELOG, removes the accidentally committed
docs/dev internal notes, and fences the pre-existing install-matrix
HTML that trips MD013.

BREAKING CHANGE: cu12/cu13 extras no longer include the RAPIDS stack.

Signed-off-by: Kelvin Lee <kinlongkelvi@nvidia.com>
@laserkelvin

Copy link
Copy Markdown
Collaborator Author

/ok to test 707387d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant