Configuration for Slingshot-11

[MoraruMaxim]

What would be a good/recommended configuration for running MPICH CH4:OFI on a Cray system equipped with Slingshot-11?

I ask because I obtained better performance in my application (Parthenon) when setting MPIR_CVAR_NO_LOCAL=1, which I was not expecting. Could you clarify if this indicates I misconfigured something, or if this is expected behavior?

Here is my mpichversion:

MPICH Version:      4.3.1
MPICH Device:       ch4:ofi
MPICH configure:    --enable-g=no --enable-error-checking=no --with-device=ch4:ofi --enable-threads=multiple --with-ch4-shmmods=posix,xpmem --enable-thread-cs=per-vci --with-libfabric=.../libfabric/1.22.0 --with-xpmem --with-pmi=/cpe/23.12/pmi/default --enable-fast=O3 --with-hwloc=.../hwloc/gcc4x/2.7.1
MPICH CC:           gcc     -O3
MPICH CXX:          g++   -O3
MPICH F77:          gfortran   -O3
MPICH FC:           gfortran   -O3
MPICH features:     threadcomm

[hzhou]

MPIR_CVAR_NO_LOCAL disables MPICH's shared memory so all communication including intranode goes to CXI. CXI's local communication is quite fast, I guess that is why you observed good performance. Just to note that all shared memory features in MPICH such as shared memory in RMA and one-copy IPC algorithms in point-to-ppint, and shared memory-based collective algorithms will all be disabled.