Add 3D gather/scatter capability#154
Conversation
…the same fypp used for COLLAPSE=2
awnawab
left a comment
There was a problem hiding this comment.
Thanks a lot for this development @pmarguinaud 🙏 We don't use the gather-scatter functionality, so we would support replacing the existing gather-scatter with gather-scatter-collapse1. Please proceed however you see fit.
I've left a few comments otherwise. I would prefer to restrict the fypp mapping to the rank-suff matrix, and for further templating moving the loop inside the files themselves. Since this is purely a stylistic consideration, I'll leave the final call up to you. But it would be nice to have a green CI before we merge.
| #ifdef OMPGPU | ||
| !$OMP TARGET | ||
| #else | ||
| !$ACC SERIAL PRESENT (ID_FILTERED) COPYIN (YL_FGS) IF (JPASS == 2) |
There was a problem hiding this comment.
Compilation is failing for this test in the OpenACC builds. That would be nice to fix before this can be merged. My best guess would be a missing CONTIGUOUS attribute on ID_FILTERED.
|
|
||
| CALL FGS%SCATTER () | ||
|
|
||
| DO JBLK = JBLKL, JBLKU |
There was a problem hiding this comment.
We're failing somewhere here in the cray builds. Since we don't use the gather scatter functionality, I would be ok with this test being excluded from the cray CI run (with something like a conditional ctest -E collapse2_lastdim.
| #! granted to it by virtue of its status as an intergovernmental organisation | ||
| #! nor does it submit to any jurisdiction. | ||
|
|
||
| MODULE FIELD_${RANK}$${SUFF}$_GATHER_COLLAPSE${COLLAPSE}$_MODULE |
There was a problem hiding this comment.
Do we really need a separate module per derived type here? If possible, I would prefer to keep the fypp expansion to the rank-suff matrix, and a loop in here for the different values of COLLAPSE. This is effectively what we are already doing in many places for WRAPPER, OWNER types in the same module. You could then simply add the relevant files to the ranksuff_srcs and non_ranksuff_srcs lists.
…ion is larger than NFLEVG. This can happen when processing arrays with extra levels such as : REAL (KIND=JPRB) :: ZFLD (NPROMA, 0:NFLEVG)
Description
Both
FIELD_GATHSCAT(already in the Field API library) andFIELD_GATHSCAT_COLLAPSE2(introduced in this PR) implement high-level gather-scatter patterns:This pattern is essential for algorithms that need to process only a subset of grid points (e.g., convective columns, cloud-covered areas, or land points); this has already proven to improve performance quite a lot.
FIELD_GATHSCATis currently used in the shallow convection algorithm of ARPEGE.FIELD_GATHSCATis used when the selection condition is 2-D and you want to keep all intermediate dimensions (e.g., vertical levels, variables) intact in the gathered array. It works with fields of any rank ≥ 2.The purpose of
FIELD_GATHSCAT_COLLAPSE2is to be used when the selection condition is 3-D and you need to collapse the two leading spatial dimensions of arrays into one. This is typical for physics parametrizations that operate on a subset of atmospheric cells. We will use it in the AROME micro-physics.Several tests have been added to exercise the newly created code.
Furthermore, we also implemented a
FIELD_GATHSCAT_COLLAPSE1whose purpose is identical toFIELD_GATHSCATbut is generated from the same fypp templates used forFIELD_GATHSCAT_COLLAPSE2;FIELD_GATHSCAT_COLLAPSE1could replaceFIELD_GATHSCAT(to be discussed).Tested on A100 with nvhpc/26.5 and MI300X with afar/23.2.0. To be tested in AROME.
Contributor Declaration
By opening this pull request, I affirm the following: