Skip to content

Preliminary Analysis framework and GlobalStats group#452

Draft
brian-oneill wants to merge 24 commits into
E3SM-Project:developfrom
brian-oneill:omega/initial-analysis-framework
Draft

Preliminary Analysis framework and GlobalStats group#452
brian-oneill wants to merge 24 commits into
E3SM-Project:developfrom
brian-oneill:omega/initial-analysis-framework

Conversation

@brian-oneill

@brian-oneill brian-oneill commented Jul 1, 2026

Copy link
Copy Markdown

Add the initial analysis module update and the first set of analysis stats to support the GlobalStats group.

Changes include:

  • Core Analysis driver.
  • AnalysisOperator abstract base class and set of derived operators for computing global statistics: SpatialMinOp, SpatialMaxOp, SpatialMeanOp,SpatialStdDevOp, and TimeMeanOp
  • AnalysisOpFactory for registering new operator types without modifying core infrastructure
  • AnalysisGroup abstract base class for bundling related analysis outputs into output streams and derived group GlobalStats
  • Needed Omega infrastructure enhancements including: added TimeInterval::isDivisibleBy function, made IOStream::create public, added reduction functions to support lower-dimensional masks.

Checklist

  • Documentation:
  • Linting
  • Building
    • CMake build does not produce any new warnings from changes in this PR
  • Testing
    • Add a comment to the PR titled Testing with the following:
      • Which machines CTest unit tests
        have been run on and indicate that are all passing.
      • The Polaris omega_pr test suite
        has passed, using the Polaris e3sm_submodules/Omega baseline
      • Document machine(s), compiler(s), and the build path(s) used for -p for both the baseline (Polaris e3sm_submodules/Omega) and the PR build
      • Indicate "All tests passed" or document failing tests
    • New tests:
      • CTest unit tests for new features have been added per the approved design.

Adds the TimeInterval::isDivisible by method to the TimeMgr module. Checks if
one TimeInterval is evenly divisible by another. Has specific logic to handle
calendar-based intervals and variable-length months
Mark non-template functions as inline to prevent multiple definition linker errors
when Reductions.h is included in multiple files
Implement globalMaskedSum functions that compute global sums of array products where a
higher-dimensional value array is multiplied by a lower-dimensional mask/weight array
(1D or 2D).
Implement globalMaskedMin and globalMaskedMax funcitons that compute global minimum and
maximum values of array products where a higher-dimensional value array is multiplied by
a lower-dimensional mask/weight array (1D or 2D).
SpatialMaxOp, SpatialMeanOp, SpatialMinOp, SpatialStdDev, and TimeMeanOp
Add convenience header including all operator classes and implementation of
registerAllBaseAnalysisOperators function
@brian-oneill

Copy link
Copy Markdown
Author

Testing

CTest unit tests:

  • Machine: Frontier
  • Compilers: craygnu, craygnu-mphipcc
  • Build type: Release
  • Result: All tests passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduce an initial in-situ Analysis framework in the Omega component to enable
runtime computation and streaming of derived “GlobalStats” outputs (spatial
reductions and time-averaged statistics), supported by a factory-based operator
registry and bundled analysis groups.

Changes:

  • Add core Analysis orchestration (operator DAG, alarm-driven scheduling, and
    dependency resolution) plus an extensible AnalysisOpFactory.
  • Add first analysis operator set (spatial min/max/mean/stddev and TimeMean)
    and a bundled GlobalStats group that builds operator chains + IOStreams.
  • Extend infrastructure to support analysis needs (time-interval divisibility
    checks, stream creation/alarm access, and masked reductions).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
components/omega/test/CMakeLists.txt Register new analysis operator/system tests with CTest.
components/omega/test/analysis/AnalysisSystemTest.cpp End-to-end/system coverage for DAG wiring, alarms, factory, and integration.
components/omega/test/analysis/AnalysisOperatorTest.cpp Multi-type/operator-correctness unit tests for spatial ops and TimeMean.
components/omega/src/infra/TimeMgr.h Add TimeInterval::isDivisibleBy API and Calendar friendship.
components/omega/src/infra/TimeMgr.cpp Implement TimeInterval::isDivisibleBy.
components/omega/src/infra/OmegaKokkos.h Add scalar→array type traits (Array1D_t, Array2D_t, …).
components/omega/src/infra/IOStream.h Make IOStream::create public; add IOStream::getAlarm.
components/omega/src/infra/IOStream.cpp Implement IOStream::getAlarm.
components/omega/src/CMakeLists.txt Add analysis include path and compile analysis sources into Omega library.
components/omega/src/base/Reductions.h Add masked sum/min/max reductions to support lower-dimensional masks.
components/omega/src/analysis/operators/TimeMeanOp.h New temporal averaging operator (TimeMeanOp).
components/omega/src/analysis/operators/SpatialStdDevOp.h New global spatial stddev operator (mask-aware).
components/omega/src/analysis/operators/SpatialMinOp.h New global spatial min operator (mask-aware).
components/omega/src/analysis/operators/SpatialMeanOp.h New global spatial mean operator (mask-aware).
components/omega/src/analysis/operators/SpatialMaxOp.h New global spatial max operator (mask-aware).
components/omega/src/analysis/operators/Ops.h Convenience include for all bundled operator headers.
components/omega/src/analysis/operators/Ops.cpp Central registration of base operators into AnalysisOpFactory.
components/omega/src/analysis/AnalysisOpFactory.h New operator factory interface + registration macro expansion.
components/omega/src/analysis/AnalysisOpFactory.cpp Factory registry implementation and metadata-based dispatch.
components/omega/src/analysis/AnalysisOperator.h New operator base class + inline Config helpers for operator params.
components/omega/src/analysis/AnalysisOperator.cpp Base operator implementation (cache, init, registry cleanup).
components/omega/src/analysis/analysisGroups/Groups.h Convenience include for bundled analysis groups.
components/omega/src/analysis/analysisGroups/GlobalStats.h New bundled analysis group API for global statistics.
components/omega/src/analysis/analysisGroups/GlobalStats.cpp Build operator chains and stream metadata from config.
components/omega/src/analysis/AnalysisGroup.h New base class for bundled analysis groups + stream param helpers.
components/omega/src/analysis/AnalysisGroup.cpp Create/group IOStreams, validate restart divisibility, attach fields.
components/omega/src/analysis/Analysis.h Core Analysis orchestrator API and OperatorNode graph definition.
components/omega/src/analysis/Analysis.cpp Core Analysis implementation (parsing, DAG build, alarms, compute).
components/omega/configs/Default.yml Add a default-enabled Analysis: GlobalStats configuration block.

Comment thread components/omega/src/analysis/AnalysisGroup.cpp
Comment thread components/omega/src/analysis/Analysis.cpp
Comment thread components/omega/src/analysis/Analysis.cpp Outdated
Comment thread components/omega/src/analysis/operators/TimeMeanOp.h
Comment thread components/omega/src/infra/TimeMgr.cpp
Comment thread components/omega/src/base/Reductions.h
Comment thread components/omega/src/base/Reductions.h
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants