NEB surface reaction

[jungsdao]

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.

Summary

Add NEB benchmark for three surface reactions from OC20NEB dataset comparing barrier height errors.

Linked issue

Resolves #293

Progress

• Calculations
• Analysis
• Application
• Documentation

Testing

Test on GPU with

• mace-mp-0a
• mace-mp-0b3
• mace-mpa-0
• mace-omat-0
• mace-matpes-r2scan
• orb-v3-consv-inf-omat
• pet-mad

New decorators/callbacks

None

[jungsdao]

PET-MAD seems to fail sometimes. It failed in one run but if I run it again with the same setting it runs without problem.

[ElliottKasoar]

Thanks for this, @jungsdao!

Apologies for the slow review.

I'd be inclined to rename the test something slightly more specific e.g. OC20NEB, since this name can't clash with any other test.

I'd also like to explore replacing a lot of the calculation with janus-core's NEB calculation, as it's a lot of duplication, and I think the only thing missing is a way of checking convergence/continuing, both of which should be minor additions.

[jungsdao]

Actually from the last drop-in meeting, there was discussion that we can modify this benchmark similar to bulk-crystal Phonon. So I can add more reactions from OC20NEB and make a scatter plot. (I'm not sure yet how many reactions are feasible yet.)

I'll try to incorporate janus interface as well.

[ElliottKasoar]

Actually from the last drop-in meeting, there was discussion that we can modify this benchmark similar to bulk-crystal Phonon. So I can add more reactions from OC20NEB and make a scatter plot. (I'm not sure yet how many reactions are feasible yet.)

I'll try to incorporate janus interface as well.

Great! Please let us know if there any issues.

I've recently proposed some changes to janus-core somewhat inspired by your use-case here: stfc/janus-core#685, which I hope could prove useful.

Also just to note you may need to rebase/merge the latest main, as there is a conflict due to us merging a different NEB PR.

[ElliottKasoar]

Hi @jungsdao, thanks again for all the work that's going into this!

I hope you don't mind that I've rebased and pushed a few suggested changes:

• Separate model directories for the calc outputs - the Li diffusion NEB is a bad example as generally it makes combining data simpler if we keep things in their own directories
• oc20neb -> OC20NEB for consistency with other tests e.g. OC157, and fixed the calculation file, which was still unchanged
• Update the janus-core dependency and a couple of fixes around this, as a couple of the things you use from the NEB were only added recently
• Added spin/charge defaults, as these are required by Orb-omol
• Updated the analysis to use the new model directories, and (I think) fixed a couple of issues with paths that were wrong
• Re-saved the DFT reference data, so we don't need to refer to the original source. This is some duplication, but it keeps things isolated

I also "removed" the docs in a commit. This was to be consistent with your code changes, but I'm not sure if you'd removed it unintentionally? It would definitely be good to add an updated form back in!

Please feel free to revert/ask about any of these changes. The only other question is whether you're happy with the current visualisation, or if it might be preferable to have some sort of structure visualisation, closer to the Li diffusion, as well as #325 and #326?

I'm not sure if we currently have any examples where we include the NEB of the model and reference together, but I think we probably could, and that would save us loading quite a large image, and allow us to link those to their actual structures. What do you think?

[jungsdao]

Hi @ElliottKasoar, Thanks a lot for helping with improving the code. I'm not disagreeing on your changes and thanks for pointing out the issues I have missed.
Also I think in current S3bucket it has only three reactions but I shared more reactions via slack chat.

I think it would be nice if we can visualize geometry along the NEB trajectory like other nebs benchmarks, but I'm not sure how it can be done.

I'm not sure if we currently have any examples where we include the NEB of the model and reference together, but I think we probably could, and that would save us loading quite a large image, and allow us to link those to their actual structures. What do you think?

I'm not quite understood what you meant with this comment. Do you mean showing RPBE reference NEB profile with model NEB profile? And showing structure altogether?

[joehart2001]

Hi @ElliottKasoar, Thanks a lot for helping with improving the code. I'm not disagreeing on your changes and thanks for pointing out the issues I have missed. Also I think in current S3bucket it has only three reactions but I shared more reactions via slack chat.

I think it would be nice if we can visualize geometry along the NEB trajectory like other nebs benchmarks, but I'm not sure how it can be done.

I'm not sure if we currently have any examples where we include the NEB of the model and reference together, but I think we probably could, and that would save us loading quite a large image, and allow us to link those to their actual structures. What do you think?

I'm not quite understood what you meant with this comment. Do you mean showing RPBE reference NEB profile with model NEB profile? And showing structure altogether?

Hi @jungsdao, (@ElliottKasoar correct me if im misunderstanding), but i think what he means is we use the plotly decorators to show an interactive NEB plot with both the reference NEB profile and the MLIP NEB profile, instead of plotting these before and loading an image. I think the idea here could be to click on the NEB profile to visualise the structures at each step (as we do in the other NEB benchmark), but also be able to view the reference structures too, so one could compare the evolution of structures of the reference and MLIP. I think this would be pretty cool.

[jungsdao]

Sorry for belated reply. I understand your suggestion and it should be definitely cool to show structures together. But I don't know what would be suitable way to implement it. It would be cascade of interactive calling: 1) scatter 2) NEB profile 3) structure for each point in NEB profile. Do you have any suggestion how to make it work? There seems to be many functions in ml_peg/app/utils/build_callbacks.py but should I define a new one or do you think it can be done by using what's already there?

[ElliottKasoar]

Sorry for belated reply. I understand your suggestion and it should be definitely cool to show structures together. But I don't know what would be suitable way to implement it. It would be cascade of interactive calling: 1) scatter 2) NEB profile 3) structure for each point in NEB profile. Do you have any suggestion how to make it work? There seems to be many functions in ml_peg/app/utils/build_callbacks.py but should I define a new one or do you think it can be done by using what's already there?

Hi @jungsdao, apologies for the very delayed reply on our end too! I've been away for all of June, and we've recently introduced a few changes that I wanted to be live before following up too much, to make sure everything is in-sync.

Please can you take a look at our new filtering guidelines: https://ddmms.github.io/ml-peg/developer_guide/filter.html? The principles are relatively simple, but you do have to be a little careful, so again, if anything is unclear, please do ask!

Can you also rebase and make sure the tests/pre-commit etc. is passing?

I'll do another proper review of everything this week.

[jungsdao]

@ElliottKasoar Thanks for reviewing! I'll rebase and push again. Also I think it would be better to change and add some more reactions than what is already in S3 link. How can I share the reactions again?

[ElliottKasoar]

@ElliottKasoar Thanks for reviewing! I'll rebase and push again. Also I think it would be better to change and add some more reactions than what is already in S3 link. How can I share the reactions again?

Great, thanks! However is easiest, with the exception of as files in the PR itself ideally. You can upload the zip in this PR as part of a comment, on Slack, Dropbox/OneDrive, or email me (ek440@cam.ac.uk).

[ElliottKasoar]

Hi @jungsdao, a couple of other quick questions in addition to the above comments:

I know you said you wanted to expand the number of reactions, but what is the reason to limit this number in general?

For the visualisation, I'd think we can simplify things a bit. In general there are already helper functions that will render plots from column/cell clicks (plot_from_table_column/plot_from_table_cell), plots from scatters (plot_from_scatter), and structures from scatters (struct_from_scatter, although perhaps not as readily the dual structures you currently have), so I think we reuse a lot of this, and reduce the custom changes here.

Please do make sure you look at the filtering mentioned above too. We need to write an info.json file during analysis to enable element filtering.

Can you also ensure the CI/pre-commit etc all passes, and the conflicting file is resolved?

[ElliottKasoar]

You may want to consider tightening the max steps for the minimisation as well, just in case a model explodes and doesn't converge. I think janus_core's default is 1000 for geom_opt (which gets called by the NEB minimise), which is much more reasonable than ASE's, but still quite large.

It's fine to leave it too, just a thought

[ElliottKasoar]

Apologies if I've already asked this, but how much have you thought about these thresholds/do you understand what they should represent?

If you're happy with them, great! If not, we're happy to discuss further.

[ElliottKasoar]

@joehart2001 may also be more familiar with this, since he wrote this function, but do we actually want/need (based on the structure visualisation) individual scatter plots for each model?

If we were doing all 932 NEBS from OC20NEB we'd probably want individual plots, but currently it's not even a handful of points.

In terms of the structure visualisation, this also seems to be using data from the calculations, similar to the phonon plotting, but in most cases we want these structures to be saved in the app data, which makes the process more modular.

[ElliottKasoar]

Generally we don't want to be constructing these at the point we run the app, but instead build them ahead of time during the analysis, and just load them as part of the app.

[jungsdao]

@ElliottKasoar Thanks for reviewing! I'll rebase and push again. Also I think it would be better to change and add some more reactions than what is already in S3 link. How can I share the reactions again?

Great, thanks! However is easiest, with the exception of as files in the PR itself ideally. You can upload the zip in this PR as part of a comment, on Slack, Dropbox/OneDrive, or email me (ek440@cam.ac.uk).

Hi Sorry for also delayed reply. This is compressed file for 927 DFT NEB trajectories. If computational resource is available, whole trajectories can be used.

What do you think about having separate interactive_helpers.py? It was inspired by bulk_crystal/phonons and I had no idea how to enable cascade of plotting (scatter -> NEB profile -> Two geometries from MLIP and DFT)
Let me know whether it makes sense to keep current API.

oc20neb.tar.gz

[joehart2001]

Hi, just a note, you seem to have committed changes to many files by accident

[jungsdao]

Hi, just a note, you seem to have committed changes to many files by accident

Sorry for messy commits. I think I made mistake before but hopefully it's cleaned up by now.