input4MIPs-prep

Collection of scripts used to help preparing input4MIPs datasets.

These scripts try to manipulate received files as little as possible, hence there is a large reliance on the low-level netCDF4 library and general avoidance of higher-level libraries like xarray and iris which often make other 'helpful' fixes without asking.

Status

• development: the project is actively being worked on

Installation

We do all our environment management using pixi. To get started, you will need to make sure that pixi is installed (instructions here, we found that using the pixi provided script was best on a Mac).

To create the virtual environment, run

pixi install
pixi run pre-commit install

These steps are also captured in the Makefile so if you want a single command, you can instead simply run make virtual-enviroment.

Having installed your virtual environment, you can now run commands in your virtual environment using

pixi run <command>

For example, to run Python within the virtual environment, run

pixi run python

Development

Install and run instructions are the same as the above (this is a simple repository, without tests etc. so there are no development-only dependencies).

Contributing

This is a very thin repository. There aren't any strict guidelines for contributing, partly because we don't know what we're trying to achieve (we're just exploring). If you would like to contribute, it is best to raise an issue to discuss what you want to do (without a discussion, we can't guarantee that any contribution can actually be used).

Repository structure

The repository is very basic. It imposes no structure on you so you can layout your Python files, notebooks etc. in any way you wish. We do have a basic Makefile which captures key commands in one place (for more thoughts on why this makes sense, see general principles: automation). For an introduction to make, see this introduction from Software Carpentry. Having said this, if you're not interested in make, you can just copy the commands out of the Makefile by hand and you will be 90% as happy for a simple repository like this.

Tools

In this repository, we use the following tools:

• git for version-control (for more on version control, see general principles: version control)
  • for these purposes, git is a great version-control system so we don't complicate things any further. For an introduction to Git, see this introduction from Software Carpentry.
• Pixi for environment management (for more on environment management, see general principles: environment management)
  • there are lots of environment management systems. Pixi works well in our experience and, for projects that need conda, it is the only solution we have tried that worked really well.
  • we track the pixi.lock file so that the environment is completely reproducible on other machines or by other people (e.g. if you want a colleague to take a look at what you've done)
• pre-commit with some very basic settings to get some easy wins in terms of maintenance, specifically:
  • code formatting with ruff
  • basic file checks (removing unneeded whitespace, not committing large files etc.)
  • (for more thoughts on the usefulness of pre-commit, see general principles: automation

Original template

This project was generated from this template: basic python repository. copier is used to manage and distribute this template.