update publish

docs/README.md

@@ -1,3 +1,3 @@
-# For future use
+# Deprecated
 
-May contain html version of the content at some point
+Don't edit this repository. Work continued in https://github.com/csc-training/csc-env-eff

hands-on/allas/allas-exercises.md

@@ -1,41 +1,55 @@
-# Using Allas in CSC HPC environmnest
+# Using Allas in CSC HPC environments
+
+Before the actual exercise, open the view to the Allas service in your browser using the cPouta WWW-interface.
+
+Open: https://pouta.csc.fi
+
+And login with your account.
+
+From the upper left corner, you find a project selection pop-up menu. If you have several projects available, select the
+training project: **project_2002389**
+
+Then from the menu in left side of the interface, select:
+
+**Object Store -> Containers**
+
+During the exercises, you can use this interface to get another view to the buckets and objects in Allas.
+Note that you need to **reload** the view in order to see the changes.
+
+=======
+
 
 ## A. Log in Puhti and use scratch
 
 1. Login to puhti.csc.fi and move to scratch:
 
 **Linux/mac**
 ```text
-ssh [email protected]   (replace XXXX with your user account)
+ssh [email protected]   (replace XXXX with your csc user account)
 ```
 
 **Windows/PuTTY**
 
    **host:* puhti.csc.fi
 
-   **login as:** XXXX  (replace XXXX with your account number)
+   **login as:** XXXX  (replace XXXX with your csc user account)
 
 
-In Puhti check you environment with command:
+In Puhti check your environment with command:
 ```text
 csc-workspaces
 ```
 Switch to the scratch directory of your project 
 ```text
-cd /scratch/project_2002389
+cd /scratch/project_2002389  # note! replace the text here (and below) with your project
 ```
-And create your own sub-directory, named after you training account:
+And create your own sub-directory, named after your training account (if this directory does not yet exist):
 ```text
 mkdir XXXX 
 ```
-(relace XXXX with your user account)
+(replace XXXX with your user account)
 
-Make the directory permissions such, that other group members can only read the contents but
-not modify it
-```text
-chmod g-wx XXXX 
-```
-move to the new directory.
+move to the directory.
 ```text
 cd XXXX
 ```
@@ -51,16 +65,18 @@ ls -ltr
 tree pythium
 ```
 
-## Using Allas
+# Using Allas
 
 Open connection to Allas:
 ```text
 module load allas
 allas-conf 
 ```
+If you have several Allas projects available, select the training project we are currently using.
+
 ### Upload case 1.  rclone
 
-Upload the data from Puhti to Allas with rclone. Use the command below (replace XXXX with your user account):
+Upload the data from Puhti to Allas with `rclone`. Use the command below (replace XXXX with your user account):
 ```text
 rclone -P copyto pythium allas:xxxx-genomes-rc/
 ```
@@ -78,12 +94,12 @@ rclone lsf allas:xxxx-genomes-rc/
 
 Check how this looks like in the Pouta web interface. Open browser and go to: [https://pouta.csc.fi/](https://pouta.csc.fi/)
 
-In Pouta interface, go to _object store_ section, list the buckets (that are here called as “Containers”).
+In Pouta interface, go to _object store_ section, list the buckets (which are here called as “Containers”).
 Locate your own _xxxx-genomes-rc_ directory and download one of the uploaded fasta files to your local computer.
 
 ### Upload case 2. a-put 
 
-Upload the pyhium directory from to Allas using following commands
+Upload the pythium directory from Puhti to Allas using following commands
 (replace XXXX with your user account)
 
 A-put case 1: Store everything in one object:
@@ -139,7 +155,7 @@ Try opening the public link that a-flip produced, with your browser.
 ## Upload case 3. Allas-backup
 Run commands:
 ```test
-allas-backup –help
+allas-backup -help
 allas-backup pythium
 allas-backup list
 ```
@@ -152,7 +168,8 @@ The data in pythium directory is now stored in many ways to Allas so we can remo
 rm -r pythium
 exit
 ```
-# C. Downloading data from Allas to Puhti
+# Downloading data from Allas to Puhti
+
 
 1. Login to puhti.csc.fi and move to scratch:
 
@@ -174,7 +191,7 @@ csc-workspaces
 ```
 Go to your personal scratch directory of your project. 
 ```text
-cd /scratch/project_yourprojectnumber/trng_xxxx
+cd /scratch/project_yourprojectnumber/xxxx
 ```
 Set up Allas connection
 ```text
@@ -215,9 +232,9 @@ mkdir vexans
 rclone copyto allas:xxxx-genomes-rc/pythium_vexans vexans/
 ls -l vexans
 ```
-example 3: copy just one object
+### example 3: copy just one object
 ```text
-rclone copyto allas:trng_xxxx-genomes-rc/pythium_vexans/pythium_vexans.fasta \ ./vexans.fasta
+rclone copyto allas:trng_xxxx-genomes-rc/pythium_vexans/pythium_vexans.fasta  ./vexans.fasta
 ls -l
 ```
 

hands-on/allas/allas-mini-tutorial.md

@@ -0,0 +1,108 @@
+# Allas Mini tutorial
+
+Open the view to the Allas service in your browser using the cPouta WWW-interface.
+
+Open: https://pouta.csc.fi
+
+And login with your account.
+
+From the upper left corner, you find a project selection pop-up menu. If you have several projects available, select the
+training project: **project_2002389**
+
+Then from the menu in left side of the interface, select:
+
+**Object Store -> Containers**
+
+And create new container by pressin button: **+Container**
+
+Keep the container _Not public_ and name it as 2002389_xxxx ( replace xxxx with your user account).
+
+Open the new bucket (that is here calles as container) and upload one file from your computer. 
+Any file should do, but prefer a file that you can open in Puhti.
+
+During the exercises, you can use this interface to get another view to the buckets and objects in Allas.
+Note that you need to **reload** the view in order to see the changes.
+
+
+## Log in Puhti and use scratch
+
+1. Login to puhti.csc.fi and move to scratch:
+
+**Linux/mac**
+```text
+ssh [email protected]   (replace XXXX with your csc user account)
+```
+
+**Windows/PuTTY**
+
+   **host:* puhti.csc.fi
+
+   **login as:** XXXX  (replace XXXX with your csc user account)
+
+
+In Puhti check your environment with command:
+```text
+csc-workspaces
+```
+Switch to the scratch directory of your project 
+```text
+cd /scratch/project_2002389  # note! replace the text here (and below) with your project
+```
+And create your own sub-directory, named after your training account (if this directory does not yet exist):
+```text
+mkdir XXXX 
+```
+(replace XXXX with your user account)
+
+move to the directory.
+```text
+cd XXXX
+```
+
+## Using Allas
+
+Open connection to Allas. 
+```text
+module load allas
+allas-conf 
+```
+If you have several Allas projects available, select the training project we are currently using.
+
+Study what you have in allas with commands
+```text
+a-list
+rclone lsd allas:
+
+a-list 2002389_xxxx
+rclone ls allas:2002389_xxxx
+```
+
+Download the file you just uploaded to Allas from your local computer.
+You can do that in two ways (replace your-file-name with the name of the file you uploaded):
+```text
+a-get 2002389_xxxx/your-file-name
+```
+or
+```
+rclone copy allas:2002389_xxxx/your-file-name ./
+```
+
+Upload the file back to Allas.
+
+Try commands:
+
+```text
+a-put your-file-name
+a-put --nc -b 2002389_xxxx 
+```
+Use use _a-put -h_ to fugure out the difference between the two commands above.
+
+Then do the upload with rclone:
+```text
+rclone copy your-file-name allas:2002389_xxxx/
+```
+Locate the files you just uploaded in Pouta www-interface.
+
+
+
+

hands-on/batch_jobs/README.md

@@ -1,9 +1,13 @@
 # Batch jobs
 
 ## Tutorials
-* [Batch job tutorial](batch_jobs_tutorial.md) on serial and parallel jobs.
-* [Hands-on batch jobs in Puhti tutorial](https://docs.csc.fi/support/tutorials/cmdline-handson/)
+* [Serial batch job tutorial](serial.md) start with this.
+* [Parallel batch job tutorial](parallel.md) using MPI and/or OpenMP.
+* [Interactive batch job tutorial](interactive.md) on Puhti.
+* [Hands-on batch jobs in Puhti tutorial](https://docs.csc.fi/support/tutorials/cmdline-handson/) A longer set of jobs to run.
 
 ## Exercises
+* [Retrieving bio data from repository](exercise_retrieving-bio-data.md) as an interactive job. (BIO)
+* [Serial, array and parallel jobs with R + contours calculation from DEM with raster package (GIS) ](https://github.com/csc-training/geocomputing/tree/master/R/puhti)
+* [Serial, array and parallel jobs with Python + NDVI calculation rasterio package (GIS) ](https://github.com/csc-training/geocomputing/tree/master/python/puhti)
 * ...
-

hands-on/batch_jobs/interactive.md

@@ -0,0 +1,31 @@
+# Batch job tutorial - Interactive jobs
+
+- In this tutorial we'll get familiar with the basic usage of the Slurm batch queue system at CSC
+- The goal is to learn how to request resources that **match** the needs of a job
+- A job consists of two parts: resource requests and the job step(s)
+- Examples are done on Puhti
+
+## Interactive jobs
+
+- In an interactive batch job, an interactive shell session is launced on a computing node. For heavy interactive tasks one can request specific resources (time, memory, cores, disk). 
+
+- You can also use tools with graphical user interfaces in an interactive shell session. For such usage the [NoMachine](https://docs.csc.fi/support/tutorials/nomachine-usage/) remote desktop often provides an improved experience.  
+
+### A simple interactive job 
+
+- To start an interactive job using one core for ten minutes
+```text
+sinteractive --account myprojectname --time 00:10:00
+```
+- You should see that the command prompt (what's shown to the left to your cursor) has changed from _puhti-login1_ (or _puhti-login2_) to e.g. _r07c51_
+- Once on the compute node, you can run commands directly from the command line without `srun`, e.g. launch the (default) Python interpreter:
+```
+module load python-env
+python3
+```
+- Quit the Python interpreter with `quit()`
+- This way you can work interactively for extended period, using lots of memory without creating load on the login nodes, which is forbidden in [the Usage Policy](https://docs.csc.fi/computing/overview/#usage-policy).
+- Quit the interactive batch job with `exit`. Note, that above you asked only for 10 minutes of time. Once that is up, you will be automatically logged out from the compute node. From the command line prompt you can see whether you're in the compute node (e.g. _r07c51_) or back to the login node (e.g. _puhti-login2_). Giving `exit` in the login node, will log you out from Puhti.
+- See the documetation at docs.csc.fi of [Interactive usage](https://docs.csc.fi/computing/running/interactive-usage/), for further information
+
+## Additional material [FAQ on CSC batch jobs ](https://docs.csc.fi/support/faq/#batch-jobs) in Docs CSC