The Sydney Soil Water-Energy Balance (SWEB) Model (work in progress)

Python workflows for generating root-zone soil moisture from gridded precipitation, evapotranspiration, and soil hydraulic properties. The current meteorology pathway is ERA5-Land-based and globally usable; SWB soil inputs now default to Earth Engine OpenLandMap, and the reference SSM path now defaults to gssm1km from users/qianrswaterr/GlobalSSM1km0509. The repository is under active development; interfaces and defaults may change.

The package-first refactor is now the main execution path. The canonical code layout lives under pysweb/, while workflows/ keeps thin CLI entrypoints and convenience wrappers around those package modules. Supported runtime imports should come from pysweb/.

Current repository structure

PySWEB/
├── pysweb/                                # Canonical package code
│   ├── dem/                               # DEM backend dispatch and NASADEM preparation
│   ├── io/                                # Shared I/O helpers
│   ├── met/                               # Meteorology path resolution and source-specific helpers
│   │   ├── era5land/
│   │   └── silo/
│   ├── soil/                              # Canonical soil-source discovery and loading logic
│   ├── ssebop/                            # Package-backed SSEBop prepare/run logic
│   │   ├── api.py
│   │   ├── landsat.py
│   │   └── inputs/                        # Legacy shim path during the transition
│   ├── swb/                               # Package-backed SWB preprocess/calibrate/run logic
│       ├── __init__.py
│       ├── api.py
│       ├── calibrate.py
│       ├── core.py
│       ├── preprocess.py
│       ├── run.py
│       └── solver.py
│   └── visualisation/                     # Canonical plotting modules for notebooks and scripts
│
├── workflows/                             # CLI entrypoints and convenience wrappers
│   ├── 1_ssebop_prepare_inputs.py         # Unified first SSEBop step: Landsat + ERA5-Land + DEM + stacks
│   ├── 2_ssebop_run_model.py              # Thin CLI wrapper over the package-backed SSEBop run workflow
│   ├── 3_sweb_preprocess_inputs.py        # Thin CLI wrapper over the package-backed SWB preprocess workflow
│   ├── 4_sweb_calib_domain.py             # Thin CLI wrapper over the package-backed SWB calibration workflow
│   ├── 5_sweb_run_model.py                # Thin CLI wrapper over the package-backed SWB run workflow
│   ├── 6_plot_results.py                  # Optional post-run plotting wrapper over pysweb.visualisation
│   ├── ssebop_runner_landsat.sh           # Convenience bash wrapper for Steps 1-2
│   └── sweb_domain_runner.sh              # Convenience bash wrapper for Steps 3-5
│
├── notebooks/                             # Example Jupyter notebooks
│   ├── README.md                          # Notebook index and scope
│   ├── 01_run_pysweb.ipynb                # Canonical notebook run example for SSEBop + SWB
│   ├── 02_plot_heatmap.ipynb              # Heatmap plotting walkthrough
│   └── 03_plot_time_series.ipynb          # Time-series plotting walkthrough
│
├── README.md
└── SWEB_logo.png

Runtime outputs are written under the unified prepared-input layout rooted at 1_ssebop_inputs/ plus 2_ssebop_outputs/, 3_sweb_inputs/, and 4_sweb_outputs/.

Workflow overview

1. workflows/1_ssebop_prepare_inputs.py: unified first SSEBop step. It prepares Landsat inputs, downloads ERA5-Land daily GeoTIFFs, prepares NASADEM, and stacks ERA5-Land meteorology products, writing Landsat to out_dir/landsat, DEM to out_dir/dem/nasadem.tif, and ERA5-Land outputs to out_dir/met/era5land/{raw,stack}.
2. workflows/2_ssebop_run_model.py: thin CLI wrapper over the package-backed SSEBop run workflow. It consumes the prepared Landsat directory plus a meteorology stack directory (for example out_dir/met/era5land/stack).
3. workflows/3_sweb_preprocess_inputs.py: thin wrapper over pysweb.swb.preprocess; it aligns ERA5-Land precipitation, SSEBop E/T/ET, OpenLandMap soil properties, and optional gssm1km reference SSM to one grid.
4. workflows/4_sweb_calib_domain.py: thin wrapper over pysweb.swb.calibrate; it calibrates domain-wide SWEB parameters (diff_factor, sm_max_factor, sm_min_factor, root_beta).
5. workflows/5_sweb_run_model.py: thin CLI wrapper over the package-backed SWB run workflow.
6. workflows/6_plot_results.py: optional post-processing wrapper over pysweb.visualisation for heatmaps and time-series plots.

The wrapper handoff follows the same contract: ssebop_runner_landsat.sh prepares meteorology under 1_ssebop_inputs/<run_subdir>/met/era5land/stack, and sweb_domain_runner.sh consumes that location by default.

Quick start

The primary entrypoints are the workflow CLIs in workflows/, which delegate into pysweb/ where that package wiring exists:

python workflows/1_ssebop_prepare_inputs.py \
  --date-range "2024-01-01 to 2024-01-31" \
  --extent "147.20,-35.10,147.30,-35.00" \
  --met-source era5land \
  --gee-project your-gee-project \
  --out-dir /path/to/run_inputs

python workflows/2_ssebop_run_model.py \
  --date-range "2024-01-01 to 2024-01-31" \
  --landsat-dir /path/to/run_inputs/landsat \
  --met-dir /path/to/run_inputs/met/era5land/stack \
  --dem /path/to/run_inputs/dem/nasadem.tif \
  --output-dir /path/to/ssebop_outputs

The unified first step now prepares NASADEM automatically under run_inputs/dem/nasadem.tif, so the SSEBop run step should consume that prepared DEM artifact instead of an unrelated external DEM path.

Convenience wrappers remain available for environment-specific end-to-end runs, but they are wrappers around the workflow CLIs rather than the primary architecture:

# Step A: prepare Landsat + meteorology inputs and run SSEBop
GEE_PROJECT=your-gee-project bash workflows/ssebop_runner_landsat.sh <run_subdir>

# Step B: preprocess, calibrate, and run SWEB against the same run_subdir
bash workflows/sweb_domain_runner.sh <run_subdir>

In that wrapper sequence, GEE_PROJECT must be provided when the SSEBop wrapper runs Step 1. The SWEB wrapper reads precipitation from 1_ssebop_inputs/<run_subdir>/met/era5land/stack, so the handoff works without manually copying ERA5-Land stacks.

For notebook-driven runs, start with notebooks/01_run_pysweb.ipynb. For plotting from Python or the command line, use the Step 6 post-processing wrapper or the canonical modules under pysweb.visualisation:

python workflows/6_plot_results.py time-series \
  --run-subdir <run_subdir> \
  --output /g/data/ym05/sweb_model/figures/<run_subdir>_timeseries.png

python workflows/6_plot_results.py heatmap \
  --run-subdir <run_subdir> \
  --domain-mean \
  --output /g/data/ym05/sweb_model/figures/<run_subdir>_heatmap_domain.png

python -m pysweb.visualisation.plot_time_series \
  --run-subdir <run_subdir> \
  --output /g/data/ym05/sweb_model/figures/<run_subdir>_timeseries.png

python -m pysweb.visualisation.plot_heatmap \
  --run-subdir <run_subdir> \
  --lat <latitude> --lon <longitude> \
  --output /g/data/ym05/sweb_model/figures/<run_subdir>_heatmap.png

python -m pysweb.visualisation.plot_heatmap \
  --run-subdir <run_subdir> \
  --domain-mean \
  --output /g/data/ym05/sweb_model/figures/<run_subdir>_heatmap_domain.png

cd notebooks
jupyter notebook

The canonical plotting modules are pysweb.visualisation.plot_time_series and pysweb.visualisation.plot_heatmap; workflows/6_plot_results.py is the workflow-level plotting entrypoint.

The meteorology path is now ERA5-Land-based and globally usable. SWB soil texture/SOC inputs now default to Earth Engine OpenLandMap, and the reference SSM input now defaults to gssm1km from users/qianrswaterr/GlobalSSM1km0509.

Key outputs

• From the unified first SSEBop step (1_ssebop_prepare_inputs.py): a prepared run directory containing landsat/, dem/nasadem.tif, met/era5land/raw/, and met/era5land/stack/. The stack directory holds precipitation_daily_<start>_<end>.nc, tmax_daily_<start>_<end>.nc, tmin_daily_<start>_<end>.nc, rs_daily_<start>_<end>.nc, ea_daily_<start>_<end>.nc, and et_short_crop_daily_<start>_<end>.nc.
• From SSEBop run (2_ssebop_run_model.py): et_daily_ssebop_<start>_<end>.nc plus intermediate etf/ndvi products, driven by the prepared meteorology stack directory.
• From SWEB preprocess (3_sweb_preprocess_inputs.py): rain_daily_*.nc, effective_precip_daily_*.nc, et_daily_*.nc, t_daily_*.nc, soil_*.nc, and optionally reference_ssm_daily_*.nc. When invoked via sweb_domain_runner.sh, precipitation is sourced from the unified prepared stack.
• From calibration (4_sweb_calib_domain.py): CSV with calibrated domain parameters.
• From SWEB run (5_sweb_run_model.py): consolidated RZSM NetCDF, optionally split into burn-in and post-burn products by sweb_domain_runner.sh.
• From the Step 6 plotting workflow (workflows/6_plot_results.py) and canonical plotting modules (pysweb.visualisation.plot_time_series, pysweb.visualisation.plot_heatmap): PNG plots and optional extracted CSV tables.

Requirements

• Python 3.12+ (recommended)
• Core packages: numpy, pandas, xarray, rioxarray, rasterio, netCDF4, scipy, pyproj, pyyaml, tqdm
• System libs for geospatial reprojection: GDAL/PROJ
• Access to forcing and ancillary datasets (Landsat, ERA5-Land, DEM, SSEBop-ready ET products)
• Google Earth Engine access for OpenLandMap and users/qianrswaterr/GlobalSSM1km0509 (for example with project your-gee-project)

Naming convention

This repository uses:

• pysweb/ for the canonical package code.
• workflows/ for runnable CLI entry points and convenience wrappers around package or workflow-owned implementations.

For soil-source logic, pysweb.soil is now the canonical package location. Keep new soil backend selection and loading changes there, with workflows and wrappers delegating into that package.

Development guidance

Use this rule of thumb while the package-first refactor is still in progress:

• Prefer pysweb/ for new reusable logic and for revisions to already-migrated functionality.
• Keep workflows/ thin. Update them when CLI arguments, orchestration, or wrapper behavior changes.

In practice:

• SSEBop prepare/run changes should usually go into pysweb.ssebop first.
• SWB run changes should usually go into pysweb.swb first.
• SWB preprocess/calibration are now package-backed entry points, so edits should usually go into pysweb.swb.preprocess and pysweb.swb.calibrate first, with workflows/3_sweb_preprocess_inputs.py and workflows/4_sweb_calib_domain.py kept as thin wrappers.

The package test suite includes a guard that fails if a pysweb module imports a top-level core package.

Workflow naming note

For the SSEBop first step, keep workflows/1_ssebop_prepare_inputs.py as the canonical entrypoint.

• prepare_inputs is the current package/API naming.

Development status

This repository is actively evolving. Verify file paths, date ranges, and spatial settings before running large jobs.

pysweb.ssebop.prepare_inputs, pysweb.ssebop.run, pysweb.swb.preprocess, pysweb.swb.calibrate, and pysweb.swb.run are wired today. workflows/2_ssebop_run_model.py, workflows/3_sweb_preprocess_inputs.py, workflows/4_sweb_calib_domain.py, and workflows/5_sweb_run_model.py are thin wrappers over those package modules.

The notebooks/ directory currently contains:

• 01_run_pysweb.ipynb: canonical notebook run example using import pysweb for SSEBop plus SWB preprocess/calibrate/run
• 02_plot_heatmap.ipynb: heatmap plotting walkthrough, with plotting modules exposed through pysweb.visualisation
• 03_plot_time_series.ipynb: SSEBop + SWEB time-series plotting walkthrough, with plotting modules exposed through pysweb.visualisation