Skip to content

Commit 28c40aa

Browse files
pojedapojeda
committed
Parallel: exercises tool chain updated for tetralith
1 parent f324162 commit 28c40aa

File tree

1 file changed

+98
-80
lines changed

1 file changed

+98
-80
lines changed

‎docs/day4/parallel.rst‎

Lines changed: 98 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ Parallel computing with Python
2020

2121
**Prerequisites**
2222

23+
.. warning::
24+
25+
Instructions are updated mainly for Tetralith
26+
2327
- Demo
2428

2529
.. tabs::
@@ -491,7 +495,7 @@ Exercises
491495

492496
.. hint::
493497
:class: dropdown
494-
498+
495499
Use shared arrays of multiprocessing package to store the partial summations from each process:
496500

497501
.. code-block:: python
@@ -1380,6 +1384,24 @@ example,
13801384

13811385
.. tabs::
13821386

1387+
.. tab:: NSC
1388+
1389+
.. code-block:: sh
1390+
1391+
#!/bin/bash -l
1392+
#SBATCH -A naiss202X-XY-XYZ
1393+
#SBATCH -t 00:05:00
1394+
#SBATCH -n 4
1395+
#SBATCH -o output_%j.out # output file
1396+
#SBATCH -e error_%j.err # error messages
1397+
1398+
ml buildenv-gcccuda/12.2.2-gcc11-hpc1
1399+
#ml julia/1.10.2-bdist # if Julia is needed
1400+
1401+
source /path-to-your-project/vpyenv-python-course/bin/activate
1402+
1403+
mpirun -np 4 python integration2d_mpi.py
1404+
13831405
.. tab:: HPC2N
13841406

13851407
.. code-block:: sh
@@ -1419,22 +1441,6 @@ example,
14191441
14201442
mpirun -np 4 python integration2d_mpi.py
14211443
1422-
.. tab:: NSC
1423-
1424-
.. code-block:: sh
1425-
1426-
#!/bin/bash -l
1427-
#SBATCH -A naiss202X-XY-XYZ
1428-
#SBATCH -t 00:05:00
1429-
#SBATCH -n 4
1430-
#SBATCH -o output_%j.out # output file
1431-
#SBATCH -e error_%j.err # error messages
1432-
1433-
#ml julia/1.9.4-bdist # if Julia is needed
1434-
1435-
source /path-to-your-project/vpyenv-python-course/bin/activate
1436-
1437-
mpirun -np 4 python integration2d_mpi.py
14381444
14391445
.. tab:: LUNARC
14401446

@@ -1501,6 +1507,26 @@ It is recommended to use a batch script for Heat scripts:
15011507

15021508
.. tabs::
15031509

1510+
.. tab:: NSC
1511+
1512+
.. code-block:: sh
1513+
1514+
#!/bin/bash -l
1515+
#SBATCH -A naiss202X-XY-XYZ
1516+
#SBATCH -t 00:05:00
1517+
#SBATCH -n 1
1518+
#SBATCH -c 32
1519+
#SBATCH --gpus-per-task=1
1520+
#SBATCH -o output_%j.out # output file
1521+
#SBATCH -e error_%j.err # error messages
1522+
1523+
ml buildenv-gcccuda/12.2.2-gcc11-hpc1
1524+
#ml julia/1.10.2-bdist # if Julia is needed
1525+
1526+
source /path-to-your-project/vpyenv-python-course/bin/activate
1527+
1528+
mpirun --oversubscribe -np 2 python heat_datatypes.py
1529+
15041530
.. tab:: HPC2N
15051531

15061532
.. code-block:: sh
@@ -1521,22 +1547,7 @@ It is recommended to use a batch script for Heat scripts:
15211547
15221548
mpirun -np 2 python heat_datatypes.py
15231549
1524-
.. tab:: NSC
1525-
1526-
.. code-block:: sh
1527-
1528-
#!/bin/bash -l
1529-
#SBATCH -A naiss202X-XY-XYZ
1530-
#SBATCH -t 00:05:00
1531-
#SBATCH -n 2
1532-
#SBATCH -o output_%j.out # output file
1533-
#SBATCH -e error_%j.err # error messages
15341550
1535-
#ml julia/1.9.4-bdist # if Julia is needed
1536-
1537-
source /path-to-your-project/vpyenv-python-course/bin/activate
1538-
1539-
mpirun -np 2 python heat_datatypes.py
15401551
15411552
On Kebnekaise, the ``srun`` command also works:
15421553

@@ -1661,16 +1672,8 @@ Exercises
16611672
.. challenge:: Running a parallel code efficiently
16621673
:class: dropdown
16631674

1664-
In this exercise we will run a parallelized code that performs a 2D integration:
1665-
1666-
.. math::
1667-
\int^{\pi}_{0}\int^{\pi}_{0}\sin(x+y)dxdy = 0
1668-
1669-
One way to perform the integration is by creating a grid in the ``x`` and ``y`` directions.
1670-
More specifically, one divides the integration range in both directions into ``n`` bins.
1671-
1672-
Here is a parallel code using the ``multiprocessing`` module in Python (call it
1673-
``integration2d_multiprocessing.py``):
1675+
In this exercise we will run the parallelized code that performs a 2D integration:
1676+
using the ``multiprocessing`` module in Python that we described above:
16741677

16751678
.. admonition:: integration2d_multiprocessing.py
16761679
:class: dropdown
@@ -1734,13 +1737,31 @@ Exercises
17341737
print("Time spent: %.2f sec" % (endtime-starttime))
17351738
17361739
1737-
Run the code with the following batch script.
1740+
Run the code with the following batch script:
17381741

17391742
.. admonition:: job.sh
17401743
:class: dropdown
17411744

17421745
.. tabs::
17431746

1747+
.. tab:: NSC
1748+
1749+
.. code-block:: sh
1750+
1751+
#!/bin/bash -l
1752+
#SBATCH -A naiss202X-XY-XYZ # your project_ID
1753+
#SBATCH -J job-serial # name of the job
1754+
#SBATCH -N 1
1755+
#SBATCH -c *FIXME* # nr. coresw
1756+
#SBATCH --time=00:20:00 # requested time
1757+
#SBATCH --error=job.%J.err # error file
1758+
#SBATCH --output=job.%J.out # output file
1759+
1760+
# Load any required modules
1761+
ml buildenv-gcccuda/12.2.2-gcc11-hpc1
1762+
1763+
python integration2d_multiprocessing.py
1764+
17441765
.. tab:: UPPMAX
17451766

17461767
.. code-block:: sh
@@ -1795,22 +1816,6 @@ Exercises
17951816
ml GCCcore/12.3.0 Python/3.11.3
17961817
python integration2d_multiprocessing.py
17971818
1798-
.. tab:: NSC
1799-
1800-
.. code-block:: sh
1801-
1802-
#!/bin/bash -l
1803-
#SBATCH -A naiss202X-XY-XYZ # your project_ID
1804-
#SBATCH -J job-serial # name of the job
1805-
#SBATCH -n *FIXME* # nr. tasks/coresw
1806-
#SBATCH --time=00:20:00 # requested time
1807-
#SBATCH --error=job.%J.err # error file
1808-
#SBATCH --output=job.%J.out # output file
1809-
1810-
# Load any modules you need, here for Python 3.11.8 and compatible SciPy-bundle
1811-
ml buildtool-easybuild/4.8.0-hpce082752a2 GCCcore/11.3.0 Python/3.10.4
1812-
python integration2d_multiprocessing.py
1813-
18141819
.. tab:: PDC
18151820

18161821
.. code-block:: sh
@@ -1851,13 +1856,24 @@ Exercises
18511856
and goes in steps of 2 (3, 5, 7, ...). The following codes contain parallelized workflows
18521857
whose goal is to compute the average of the whole feature **Value** using some number of
18531858
workers. Substitute the **FIXME** strings in the following codes to perform the tasks given
1854-
in the comments.
1859+
in the comments. Call the script for instance ``script-df.py``.
1860+
1861+
1862+
.. warning::
1863+
1864+
For Tetralith you will need to install ``pandas``:
1865+
1866+
.. code-block:: sh
18551867
1856-
*The main idea for all languages is to divide the workload across all workers*.
1857-
You can run the codes as suggested for each language.
1868+
ml buildenv-gcccuda/12.2.2-gcc11-hpc1
1869+
source vpyenv-python-course/bin/activate
1870+
pip install pandas
18581871
1859-
Pandas is available in the following combo ``ml GCC/12.3.0 SciPy-bundle/2023.07`` (HPC2N) and
1860-
``ml python/3.11.8`` (UPPMAX). Call the script ``script-df.py``.
1872+
1873+
Pandas is available in the following combo ``ml GCC/12.3.0 SciPy-bundle/2023.07`` (HPC2N) and
1874+
``ml python/3.11.8`` (UPPMAX).
1875+
1876+
18611877

18621878
.. code-block:: python
18631879
@@ -1901,6 +1917,24 @@ Exercises
19011917

19021918
.. tabs::
19031919

1920+
.. tab:: NSC
1921+
1922+
.. code-block:: sh
1923+
1924+
#!/bin/bash -l
1925+
#SBATCH -A naiss202X-XY-XYZ # your project_ID
1926+
#SBATCH -J job-serial # name of the job
1927+
#SBATCH -n 4 # nr. tasks/coresw
1928+
#SBATCH --time=00:20:00 # requested time
1929+
#SBATCH --error=job.%J.err # error file
1930+
#SBATCH --output=job.%J.out # output file
1931+
1932+
# Load any required modules
1933+
ml buildenv-gcccuda/12.2.2-gcc11-hpc1
1934+
source vpyenv-python-course/bin/activate
1935+
1936+
python script-df.py
1937+
19041938
.. tab:: UPPMAX
19051939

19061940
.. code-block:: sh
@@ -1951,22 +1985,6 @@ Exercises
19511985
ml GCCcore/12.3.0 Python/3.11.3 SciPy-bundle/2023.07
19521986
python script-df.py
19531987
1954-
.. tab:: NSC
1955-
1956-
.. code-block:: sh
1957-
1958-
#!/bin/bash -l
1959-
#SBATCH -A naiss202X-XY-XYZ # your project_ID
1960-
#SBATCH -J job-serial # name of the job
1961-
#SBATCH -n 4 # nr. tasks/coresw
1962-
#SBATCH --time=00:20:00 # requested time
1963-
#SBATCH --error=job.%J.err # error file
1964-
#SBATCH --output=job.%J.out # output file
1965-
1966-
# Load any modules you need, here for Python 3.11.8 and compatible SciPy-bundle
1967-
ml buildtool-easybuild/4.8.0-hpce082752a2 GCCcore/11.3.0 Python/3.10.4
1968-
python script-df.py
1969-
19701988
19711989
.. tab:: PDC
19721990

0 commit comments

Comments
 (0)