feat(wip): met office deterministic

datasets/met-office/README.md

@@ -0,0 +1,43 @@
+# Met Office Dataset
+
+This directory contains the configuration and scripts for ingesting Met Office weather forecast data into the Planetary Computer.
+
+## Collections
+
+This dataset includes eight collections:
+
+### Global Collections (10km resolution)
+
+- **met-office-global-deterministic-height** - Weather parameters at specific atmospheric height levels
+- **met-office-global-deterministic-near-surface** - Near-surface weather parameters (precipitation, temperature, wind, etc.)
+- **met-office-global-deterministic-pressure** - Weather parameters at specific atmospheric pressure levels
+- **met-office-global-deterministic-whole-atmosphere** - Whole atmosphere parameters (CAPE, cloud amounts, etc.)
+
+### UK Collections (2km resolution)
+
+- **met-office-uk-deterministic-height** - Weather parameters at specific atmospheric height levels
+- **met-office-uk-deterministic-near-surface** - Near-surface weather parameters
+- **met-office-uk-deterministic-pressure** - Weather parameters at specific atmospheric pressure levels
+- **met-office-uk-deterministic-whole-atmosphere** - Whole atmosphere parameters
+
+## Scripts
+
+The `scripts/` directory contains shell scripts for managing the dataset workflow:
+
+- **`ingest-collections.sh`** - Ingest all eight Met Office collections into the STAC catalog:
+
+  ```bash
+  ./scripts/ingest-collections.sh
+  ```
+
+- **`upload-all.sh`** - Process items for all collections using a specified chunkset id:
+
+  ```bash
+  ./scripts/upload-all.sh <chunkset-id>
+  ```
+
+- **`process-items-and-monitor.sh`** - Process items for a specific collection and monitor the workflow progress:
+
+  ```bash
+  ./scripts/process-items-and-monitor.sh <chunkset-id> <collection-id>
+  ```

datasets/met-office/collection/met-office-global-deterministic-height/description.md

@@ -0,0 +1,53 @@
+This collection offers 1 parameter at 33 available height levels (5m to 6000m) from the Met Office global deterministic 10km forecast. This is a numerical weather prediction forecast for the whole globe, with a resolution of approximately 0.09 degrees i.e. 10km (2,560 x 1,920 grid points). 
+
+The data is available as NetCDF files. It's offered on a free, unsupported basis, so we don’t recommend using it for any critical business purposes.
+
+## Height Levels
+Available height levels, in metres (m) above ground, are: 
+* 5.0 10.0 20.0 30.0 50.0 75.0 100.0 150.0 200.0 250.0 300.0 400.0 500.0 600.0 700.0 800.0 1000.0 1250.0 1500.0 1750.0 2000.0 2250.0 2500.0 2750.0 3000.0 3250.0 3500.0 3750.0 4000.0 4500.0 5000.0 5500.0 6000.0
+
+## Timesteps
+The following timesteps are available:
+* every hour from 0 to 54 hours (for most parameters, see parameter table for exceptions)
+* every 3 hours from 57 to 144 hours
+* every 6 hours from 150 to 168 hours
+
+## Update Frequency
+The model is run four times each day, with forecast reference times of 00:00, 06:00, 12:00 and 18:00 (UTC).
+
+The runs at 00:00 and 12:00 provide data for the next 168 hours. The runs at 06:00 and 18:00 provide data for the next 67 hours.
+
+The forecast reference time represents the nominal data time or start time of a model forecast run, rather than the time when the data is available.
+
+## Archive length and latency
+As of December 2025, the archive contains data from December 2023 onwards. Forecasts will continue to be available for at least two years from their data date.
+
+The data is typically available 6 hours after the model run time.
+
+## Technical specs
+The data is available as NetCDF files. NetCDF (Network Common Data Form) is an interface for array-orientated data access and a library that supports the interface. It is composed of 3 components:
+* Variables store the data 
+* Dimensions give relevant dimension information for the variables
+* Attributes provide auxiliary information about the variables or dataset itself 
+
+NetCDF is used within the atmospheric and oceanic science communities and is network transparent, allowing for it to be accessed by computers that store integers, characters and floating-point numbers.
+
+Iris supports NetCDF files through reading, writing and handling. Iris implements a model based on the CF conventions, giving a format-agnostic interface for working with data.
+
+[Find further support on using Iris with NetCDF files.](https://scitools-iris.readthedocs.io/en/stable/)
+
+## Help us improve the data services we offer
+[Join the Met Office research panel](https://forms.office.com/Pages/ResponsePage.aspx?id=YYHxF9cgRkeH_VD-PjtmGdxioYGoFbFIkZuB_q8Fb3VUQkoxRVQzTFdUMzNMVzczWVM5VTc3QTY3MC4u) to help us understand how people interact with weather and climate data, uncover challenges and explore opportunities.
+
+## How to cite
+Met Office global deterministic 10km forecast was accessed on DATE from _insert Planetary Computer link_.
+
+## License
+British Crown copyright 2023-2025, the Met Office, is licensed under [CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0/deed.en).
+
+## Providers
+[Met Office](https://www.metoffice.gov.uk/)
+See all datasets managed by [Met Office.](https://registry.opendata.aws/?search=managedBy:met%20office)
+
+## Contact
+[[email protected]](mailto:[email protected]). Service desk is only available Mon – Fri, 09:00 until 17:00 UTC (-1 hour during BST). As a non-operational service we aim to respond to any service support enquiries within 3-5 business days. 

datasets/met-office/collection/met-office-global-deterministic-height/template.json

@@ -0,0 +1,73 @@
+{
+  "id": "met-office-global-deterministic-height",
+  "type": "Collection",
+  "links": [
+    {
+      "rel": "license",
+      "href": "https://creativecommons.org/licenses/by-sa/4.0/deed.en",
+      "type": "text/html",
+      "title": "Creative Commons Attribution-ShareAlike 4.0"
+    },
+
+    {
+      "rel": "describedBy",
+      "href": "https://www.metoffice.gov.uk/services/data/external-data-channels",
+      "title": "Met Office Dataset Documentation"
+    }
+  ],
+  "title": "Met Office Global Deterministic Height Level",
+  "assets": {
+    "thumbnail": {
+      "href": "https://ai4edatasetspublicassets.blob.core.windows.net/assets/pc_thumbnails/met-office-global-deterministic-height.jpg",
+      "type": "image/png",
+      "roles": ["thumbnail"],
+      "title": "Met Office Global Deterministic Height Level Thumbnail"
+    }
+  },
+  "extent": {
+    "spatial": {
+      "bbox": [[-180, -90, 180, 90]]
+    },
+    "temporal": {
+      "interval": [["2023-12-15T00:00:00Z", null]]
+    }
+  },
+  "license": "proprietary",
+  "keywords": ["MetOffice", "Global", "Cloud"],
+  "providers": [
+    {
+      "url": "https://www.metoffice.gov.uk/",
+      "name": "Met Office",
+      "roles": ["producer", "licensor", "processor"]
+    },
+    {
+      "url": "https://planetarycomputer.microsoft.com",
+      "name": "Microsoft",
+      "roles": ["host", "processor"]
+    }
+  ],
+  "description": "The Met Office Global Deterministic Height Level dataset provides a composite of weather parameters generated for specific atmospheric height levels.",
+  "cube:variables": {
+    "cloud_amount_on_height_levels": {
+      "description": "Fraction of horizontal grid square occupied by cloud in layers centred on height levels. The levels are height above ground.",
+      "dimensions": [],
+      "unit": "1",
+      "attrs": {
+        "1H_timesteps": "T+0 to T+54",
+        "3H_timesteps": "T+57 to T+144",
+        "6H_timesteps": "T+150 to T+168",
+        "content_type": "application/netcdf"
+      }
+    }
+  },
+  "stac_version": "1.0.0",
+  "msft:group_id": "metoffice",
+  "msft:container": "staging",
+  "stac_extensions": [
+    "https://stac-extensions.github.io/forecast/v0.2.0/schema.json",
+    "https://stac-extensions.github.io/datacube/v2.0.0/schema.json"
+  ],
+  "msft:storage_account": "ukmoeuwest",
+  "msft:short_description": "Met Office Global Deterministic Height Level",
+  "msft:region": "westeurope"
+}

datasets/met-office/collection/met-office-global-deterministic-near-surface/description.md

@@ -0,0 +1,49 @@
+This collection offers 48 parameters at near-surface level from the Met Office global deterministic 10km forecast. This is a numerical weather prediction forecast for the whole globe, with a resolution of approximately 0.09 degrees i.e. 10km (2,560 x 1,920 grid points).
+
+The data is available as NetCDF files. It's offered on a free, unsupported basis, so we don’t recommend using it for any critical business purposes.
+
+## Timesteps
+The following timesteps are available:
+* every hour from 0 to 54 hours (for most parameters, see parameter table for exceptions)
+* every 3 hours from 57 to 144 hours
+* every 6 hours from 150 to 168 hours
+
+## Update Frequency
+The model is run four times each day, with forecast reference times of 00:00, 06:00, 12:00 and 18:00 (UTC).
+
+The runs at 00:00 and 12:00 provide data for the next 168 hours. The runs at 06:00 and 18:00 provide data for the next 67 hours.
+
+The forecast reference time represents the nominal data time or start time of a model forecast run, rather than the time when the data is available.
+
+## Archive length and latency
+As of December 2025, the archive contains data from December 2023 onwards. Forecasts will continue to be available for at least two years from their data date.
+
+The data is typically available 6 hours after the model run time.
+
+## Technical specs
+The data is available as NetCDF files. NetCDF (Network Common Data Form) is an interface for array-orientated data access and a library that supports the interface. It is composed of 3 components:
+* Variables store the data 
+* Dimensions give relevant dimension information for the variables
+* Attributes provide auxiliary information about the variables or dataset itself 
+
+NetCDF is used within the atmospheric and oceanic science communities and is network transparent, allowing for it to be accessed by computers that store integers, characters and floating-point numbers.
+
+Iris supports NetCDF files through reading, writing and handling. Iris implements a model based on the CF conventions, giving a format-agnostic interface for working with data.
+
+[Find further support on using Iris with NetCDF files.](https://scitools-iris.readthedocs.io/en/stable/)
+
+## Help us improve the data services we offer
+[Join the Met Office research panel](https://forms.office.com/Pages/ResponsePage.aspx?id=YYHxF9cgRkeH_VD-PjtmGdxioYGoFbFIkZuB_q8Fb3VUQkoxRVQzTFdUMzNMVzczWVM5VTc3QTY3MC4u) to help us understand how people interact with weather and climate data, uncover challenges and explore opportunities.
+
+## How to cite
+Met Office global deterministic 10km forecast was accessed on DATE from _insert Planetary Computer link_.
+
+## License
+British Crown copyright 2023-2025, the Met Office, is licensed under [CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0/deed.en).
+
+## Providers
+[Met Office](https://www.metoffice.gov.uk/)
+See all datasets managed by [Met Office.](https://registry.opendata.aws/?search=managedBy:met%20office)
+
+## Contact
+[[email protected]](mailto:[email protected]). Service desk is only available Mon – Fri, 09:00 until 17:00 UTC (-1 hour during BST). As a non-operational service we aim to respond to any service support enquiries within 3-5 business days.