Add TorsionNet500CCSDT benchmark visualisation, MAE metric, and docs#682
Add TorsionNet500CCSDT benchmark visualisation, MAE metric, and docs#682ellalbolland wants to merge 4 commits into
Conversation
Adds per-fragment RMSE/MAE scatter plots and per-fragment torsion energy profile curves to the TorsionNet500CCSDT analysis, wired into the app via a table-cell -> scatter -> curve click-through. Renames the raw energy fields to ref_energy/model_energy (and ref_rel_energy/model_rel_energy for the mean-centered profiles) to match existing codebase conventions, and switches to the shared rmse/mae utilities instead of a local reimplementation. Named TorsionNet500CCSDT rather than TorsionNet500 to coexist alongside the existing barrier-height implementation in ddmms#642/ddmms#659, since this benchmark reports RMSE/MAE of the full relative energy profile against CCSD(T)/CBS reference data - a different metric and reference level of theory, not a duplicate.
|
Thanks @ellalbolland, from a brief scan its looking great! For the uma model access you can request it i think here: https://huggingface.co/facebook/UMA
|
Save per-fragment elemental info via get_struct_info so the app can filter TorsionNet500CCSDT by element, and set precision="high" explicitly on the calculator.
|
Here's the input dataset used for the calc: TorsionNet500_xyz (1).zip |
thanks uploaded |
| Computational cost | ||
| ------------------ | ||
|
|
||
| Medium: tests are likely to take hours per model to run on CPU, since each |
There was a problem hiding this comment.
this takes about 10 minutes for me locally on cpu, so lets change to less than an hour on cpu and minutes on GPU
|
Hi @ellalbolland, everything is looking really nice so far! One thing that would be great to add is visualisaiton from the torsion curve plot. you could take a look at some of the NEB curves to see how they use the trajectory mode to visualise data points along the curve. thanks! |
…CSDT.py Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>
…CSDT.py Co-authored-by: Joseph Hart <92541539+joehart2001@users.noreply.github.com>
Pre-review checklist for PR author
Summary
Adds per-fragment RMSE/MAE scatter plots and per-fragment torsion energy profile curves to the TorsionNet500CCSDT analysis, enabling table-cell → scatter → curve click-through navigation.
Named TorsionNet500CCSDT rather than TorsionNet500 to coexist alongside the existing barrier-height implementation in #642/#659, since this benchmark reports RMSE/MAE of the full relative energy profile against CCSD(T)/CBS reference data - a different metric and reference level of theory, not a duplicate.
Linked issue
Resolves #679
Progress
Testing
Tested against 16 of the 21 registered models (full 500/500-fragment recompute completed for each). The uma family (uma-s-1p1-omat, uma-m-1p1-omat, uma-s-1p1-omol, uma-s-1p2-omol, uma-m-1p1-omol) could not be run due to lacking HuggingFace authorisation for the gated UMA model weights.
Confirmed working end-to-end by running the app standalone (
uv run python ml_peg/app/conformers/TorsionNet500CCSDT/app_TorsionNet500CCSDT.py) and clicking through the table -> scatter -> curve interactivity manually.New decorators/callbacks
One new local callback,
_register_curve_callbackinapp_TorsionNet500CCSDT.py.plot_from_scatterpre-loads every possible figure upfront (too expensive for 500 curves/model);struct_from_scatterlazy-loads but only for structures, not Plotly figures. This one lazy-loads a curve JSON from file per click instead. Flagged with aTODOto move it intobuild_callbacks.pyif another benchmark needs the same pattern.