Skip to content

Maintaining Spack Buildcache

Jump to bottom
Matthew Thompson edited this page Apr 29, 2026 · 19 revisions

This page details the efforts to create a buildcache for use in Spack CI tests in MAPL.

Useful Links

Building Packages on AWS

Preliminaries

Padded Length

Testing showed we apparently need to increase the length of the padding in the install tree. This is what I ran:

spack config add config:install_tree:padded_length:256

packages.yaml

My current packages.yaml is set as:

packages:

  gcc:
    externals:
    - spec: gcc@13.3.0 languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc
          cxx: /usr/bin/g++
          fortran: /usr/bin/gfortran
        flags: {}
        environment: {}
        extra_rpaths: []
    - spec: gcc@14.2.0 languages='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc-14
          cxx: /usr/bin/g++-14
          fortran: /usr/bin/gfortran-14
        flags: {}
        environment: {}
        extra_rpaths: []
    buildable: false

  all:
    require: target=x86_64_v3
    providers:
      mpi: [openmpi, intel-oneapi-mpi]
      blas: [openblas, intel-oneapi-mkl]
      lapack: [openblas, intel-oneapi-mkl]

  gcc:
    buildable: false
  openmpi:
    require:
    - any_of: ['%gcc']
      message: Only use Open MPI with GCC

  intel-oneapi-mpi:
    require:
    - any_of: ['%oneapi']
      message: Only use Intel MPI with Intel

  hdf5:
    variants: +fortran +szip +hl +threadsafe +mpi
    # Note that cdo requires threadsafe, but hdf5 doesn't
    # seem to want that with parallel. Hmm.
  netcdf-c:
    variants: +dap
  esmf:
    variants: ~pnetcdf ~xerces
  cdo:
    variants: ~proj ~fftw3
    # cdo wanted a lot of extra stuff for proj and fftw3. Turn off for now
  pflogger:
    variants: +mpi
  pfunit:
    variants: +mpi +fhamcrest
  fms:
    require: '@2024.03 ~gfs_phys +pic constants=GEOS precision=32,64 +deprecated_io ~yaml target=x86_64_v3'
    # NOTE: I have to re-add the target here otherwise spack seemed to not build for a generic processor
  mapl:
    variants: +extdata2g +fargparse +pflogger +pfunit ~pnetcdf

repos.yaml

Spack is now split with packages in a separate repo. To be consistent, we manually clone and put it in a place easy to maintain. As such, our repos.yaml is:

repos:
  builtin:
    git: git@github.com:spack/spack-packages.git
    destination: /home/ubuntu/spack-packages
  geosesm: /home/ubuntu/geosesm-spack

Building the Buildcache

Create Spack Environment

It turns out, you can't build or push a buildcache unless you are in a Spack environment. So we need to create one:

spack env create geos-buildcache
spack env activate -p geos-buildcache

Add Packages to Environment

spack spec geosgcm-deps ^esmf@8.9.1

Check to make sure that has the right ESMF version. Then:

spack add geosgcm-deps ^esmf@8.9.1
spack install geosgcm-deps ^esmf@8.9.1

Add mirror for buildcache

spack mirror add --oci-username mathomp4 --oci-password-variable GEOS_BUILDCACHE_TOKEN --unsigned geos-buildcache oci://ghcr.io/GEOS-ESM/geos-buildcache

Pushing the Buildcache

spack buildcache push --update-index oci://ghcr.io/GEOS-ESM/geos-buildcache

ESMF 9

There is now a need to have ESMF 9 in a buildcache. But, testing shows that you can't really have two versions of ESMF in the same Spack environment. You should, but the fact that ESMF v9.0.0b03 is not a "real" release but a git commit makes it tricky.

So, let's make a new Spack environment and buildcache just for ESMF 9.

Add ESMF 9 to Spack

First, I did spack edit esmf and added the 9.0.0b03 version to the package.py file (see below).

diff --git a/repos/spack_repo/builtin/packages/esmf/package.py b/repos/spack_repo/builtin/packages/esmf/package.py
index 295f2c42de..1b20392bdc 100644
--- a/repos/spack_repo/builtin/packages/esmf/package.py
+++ b/repos/spack_repo/builtin/packages/esmf/package.py
@@ -33,6 +33,7 @@ class Esmf(MakefilePackage, PythonExtension):
     # Develop is a special name for spack and is always considered the newest version
     version("develop", branch="develop")
     # generate chksum with 'spack checksum esmf x.y.z'
+    version("9.0.0b11", tag="v9.0.0b11")
     version("8.9.1", sha256="e3fafd0c057bf1c3b4c41c997b392016d621b1f1a7c601355c325a7f58425d78")
     version("8.9.0", sha256="586e0101d76ff9842d9ad43567fae50317ee794d80293430d9f1847dec0eefa5")
     version("8.8.1", sha256="b0acb59d4f000bfbdfddc121a24819bd2a50997c7b257b0db2ceb96f3111b173")

Create New Spack Environment

spack env deactivate
spack env create geos-buildcache-esmf9
spack env activate -p geos-buildcache-esmf9

Add ESMF 9 to Environment and make Buildcache

spack spec geosgcm-deps ^esmf@9.0.0b11
spack add geosgcm-deps ^esmf@9.0.0b11
spack install geosgcm-deps ^esmf@9.0.0b11

spack mirror add --oci-username mathomp4 --oci-password-variable GEOS_BUILDCACHE_TOKEN --unsigned geos-buildcache-esmf9 oci://ghcr.io/GEOS-ESM/geos-buildcache-esmf9
spack buildcache push --update-index oci://ghcr.io/GEOS-ESM/geos-buildcache-esmf9

Clone this wiki locally