[WIP] NVIDIA/OpenACC hackathon May 2026 work#286
Conversation
…n, also add commented out version to report on all optimization
…FLAGS, and nvtx library to the linker LDFLAGS
…am to a crawl, make sure the leading dash is on the -Minfo flag
…because adding a new compiler nvhpc-prof didn't work as the the build failed at the link step
… nsys profiler, however it's FAILing at the link step of the build, however, just modifying the nvhpc compiler with the changes is working
… nsys profiler, however it's FAILing at the link step of the build, however, just modifying the nvhpc compiler with the changes is working
…er though, because it seems to only know about GPUs
…that symbols will be used by the profiler
…EBUG as it will be empty, use plain -Minstrument since the functions option is the default
…bably the missing piece to get nvhpc-prof working as an option
…pc-prof compiler build
…profiler options are setup right
…e nvhpc for profiling
ekluzek
added
enhancement
There was a problem hiding this comment.
My 2 cents here: Adding an nvhpc-prof option is a little confusing since what you are doing is using the same NVHPC compiler but enabling the Nsight profiling tool (i.e., you are not adding an actually different compiler). I usually only need to override the MPI_RUN_COMMAND with the profiling command and keep the same nvhpc compiler setup.
There was a problem hiding this comment.
It does add some to the compilation and link step so that the profiler can be run at the run phase. So there is a difference in compilation as well.
I mainly don't like the duplication it has in copying everything that nvhpc has with a few small changes. Possibly a better way would be to have an XML variable that turns on profiling? This would cut the duplication and make it easier to use for other compilers.
Since, you have to both change the compilation AND the mpirun I thought it better to have it wrapped up in one.
What do you think of the idea of adding an XML variable for this? I think it would have to go somewhere like in CMEPS/cime_config/config_component_cesm.xml though. Or under CIME. Maybe something like:
BUILD_AND_RUN_PROFILER
?
There was a problem hiding this comment.
Pinging @billsacks @fischer-ncar on the above question. And in general on what's in this PR for what parts might be good to come to main at some point.
There was a problem hiding this comment.
Thanks @ekluzek . I agree with your point here as I also feel uncomfortable with the large duplication of copying every from the existing nvhpc setups. I like your idea about adding an XML variable to turn on profiling and add the extra necessary compilation and link steps. Since it is specific for NVIDIA GPU profiling, maybe add something like ENABLE_NSYS_PROFILER to this section (https://github.com/ESCOMP/CMEPS/blob/main/cime_config/config_component.xml#L806-L852)?
|
Thank you for all of your work on this, @ekluzek . My feeling is that I want to see that this is useful to multiple people long-term before bringing it into main - for now having it on a branch is fine for the hackathon and then we can see how much we want to continue using it, and in what form, beyond that point. |
|
(I should add, though, that I haven't looked in detail at what this adds, so it may be that some of this is stuff we more clearly want??) |
…he Makefile.common in the perf_testing directory of CTSM nvidia-hackathon-26 branch, also remove the timing of the compile for nchpc as it produces a ton of useless output
3 participants
Some changes mostly to the nvhpc compiler (and adding a nvhpc-prof compiler to also run profiling) for the OpenACC/NVIDIA Hackathon in late April and early May 2026.
There will probably be bits of this that we might want to bring into main development. But, it might not be clear until we are finished.
Things that likely would be good to bring in: