How late can we initialize MPI?

[AtlantaPepsi]

Do we have to call MPI_Init() the first thing of any MPI program (or at least one of the first things)? Assuming we have a program launched by mpirun, and assuming no MPI calls (if any) are made before MPI_Init(), is there any risks calling MPI_Init() late in the program or only enclose part of the program?

[hzhou]

You can call MPI_Init any time during your program. A few considerations:

• You only can make MPI calls after MPI_Init, including figure out your job size and process ranks
• MPI_Init is collective and the most expensive step. In the middle of your program, it may cause delays due to uneven arrival time.