Skip to content

Commit a10fef8

Browse files
blokhinblokhin
authored
Merge pull request #145 from knopki/feature/own-vm-image-example
Bring your own VM image example
2 parents fde1d80 + 231c61c commit a10fef8

12 files changed

+334
-0
lines changed

‎CLOUD.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Cloud Providers
22

3+
You can pre-build an image with all the engines you need. This can be faster
4+
than uploading an engine each time/creating an engine when configuring a node.
5+
There is an example of building such an image at
6+
[examples/own-vm-image/](examples/own-vm-image/README.md)`.
7+
38
## Azure
49

510
### Setup

‎examples/own-vm-images/README.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Build Your Own OS Image
2+
3+
You can pre-build an image with all the engines you need. This can be faster
4+
than uploading an engine each time/creating an engine when configuring a node.
5+
6+
The [packer](https://www.hashicorp.com/en/products/packer) utility is used
7+
to build images. It must be installed beforehand.
8+
See [documentation](https://developer.hashicorp.com/packer/install).
9+
10+
The details of creation will vary for different targets,
11+
but the essence will be common.
12+
13+
The main configuration file for `packer` is the `*.pkr.hcl` file.
14+
It specifies which plugin to use in the `packer` section,
15+
what to use as OS image source in the `source` section and
16+
how to build it in the `build` section.
17+
The contents of this file will be different for different OS and clouds.
18+
The common parts are in the scripts listed in the `source` section.
19+
20+
## FLEUR Engine on Debian 12 at Hetzner Cloud
21+
22+
In Hetzner Cloud, the OS image is called a "snapshot".
23+
The snapshot can be used as a base for creating a "server".
24+
25+
You need to create a project in HCloud and create a read and write API key
26+
in it. Then specify this API key in the `HCLOUD_TOKEN` environment variable.
27+
For example, `export HCLOUD_TOKEN=“xxxx”`.
28+
29+
The main configuration file for the build is `hcloud-debian-12-fleur.pkr.hcl`.
30+
Most likely you need to change the `location` and `server_type` in the
31+
`source` section to match your HCloud project's settings.
32+
33+
There is a list of provision shell scripts in the `build` section.
34+
The engine build script is located in the `install-fleur.sh` file.
35+
The rest of the scripts perform common operations to prepare an OS image
36+
and will not be significantly different for other OS/clouds/engines.
37+
38+
Let's build an image. To do this, install required `packer` plugins:
39+
40+
```sh
41+
packer init ./hcloud-debian-12-fleur.pkr.hcl
42+
```
43+
44+
Then run build script:
45+
46+
```sh
47+
packer build ./hcloud-debian-12-fleur.pkr.hcl
48+
```
49+
50+
This can take 15 to 20 minutes.
51+
If everything is successful, the last line will show the image ID. Remember this ID - you will need to specify it in `yascheduler.conf`.
52+
53+
```
54+
--> hcloud.debian: A snapshot was created: 'fleur-debian-xxxxx' (ID: 123123123)
55+
```
56+
57+
As a result, we have an OS image (snapshot) with the usual
58+
Debian 12 with `inpgen` and `fleur` already installed.
59+
60+
Let's configure `yascheduler` to use our OS image.
61+
Setup `db`, `local` and `remote` sections in `yascheduler.conf` as usual. Add `clouds` and engines sections:
62+
63+
```ini
64+
[engine.inpgen]
65+
spawn = inpgen -explicit -inc +all -f aiida.in > shell.out 2> out.error
66+
check_cmd = ps -eocomm= | grep -q inpgen
67+
input_files = aiida.in
68+
output_files = aiida.in inp.xml default.econfig shell.out out out.error scratch struct.xsf
69+
70+
[engine.fleur]
71+
spawn = fleur -minimalOutput -wtime 360 > shell.out 2> out.error
72+
check_cmd = ps -eocomm= | grep -q fleur
73+
input_files = inp.xml
74+
output_files = inp.xml kpts.xml sym.xml relax.xml shell.out out.error out out.xml FleurInputSchema.xsd FleurOutputSchema.xsd juDFT_times.json cdn1 usage.json
75+
76+
[clouds]
77+
; Your API key
78+
hetzner_token = xxx
79+
; Your preffered server type.
80+
; The disk should be at least as large as the snapshot.
81+
hetzner_server_type = cpx21
82+
; It's best to use the same location as the snapshot.
83+
; This way servers will be created faster.
84+
hetzner_location = fsn1
85+
; OS image ID that you should have memorized earlier.
86+
; You can always look under Snapshots in HCloud Dashboard.
87+
hetzner_image_name = 237472643
88+
```
89+
90+
With this configuration, `yascheduler` will build servers from our
91+
OS image with pre-installed engines. This way, after spending time once,
92+
we don't waste time building/loading the engine every time.

‎examples/own-vm-images/clean-apt-files.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env bash
2+
# remove apt files
3+
4+
export DEBIAN_FRONTEND=noninteractive
5+
6+
apt-get -y autopurge
7+
apt-get -y clean
8+
9+
rm -rf /var/lib/apt/lists/*

‎examples/own-vm-images/clean-logs.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env bash
2+
# remove logs
3+
4+
journalctl --flush
5+
journalctl --rotate --vacuum-time=0
6+
7+
find /var/log -type f -exec truncate --size 0 {} \; # truncate system logs
8+
find /var/log -type f -name '*.[1-9]' -delete # remove archived logs
9+
find /var/log -type f -name '*.gz' -delete # remove compressed archived logs

‎examples/own-vm-images/clean-root.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
# remove user files
3+
4+
unset HISTFILE
5+
6+
rm -rf /root/.cache
7+
rm -rf /root/.ssh
8+
rm -f /root/.bash_history
9+
rm -f /root/.lesshst
10+
rm -f /root/.viminfo

‎examples/own-vm-images/docker-debian-12-fleur.pkr.hcl‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
packer {
2+
required_plugins {
3+
docker = {
4+
source = "github.com/hashicorp/docker"
5+
version = "~> 1.1"
6+
}
7+
}
8+
}
9+
10+
source "docker" "debian" {
11+
image = "debian:12"
12+
commit = true
13+
}
14+
15+
build {
16+
name = "fleur"
17+
sources = [
18+
"source.docker.debian"
19+
]
20+
21+
provisioner "shell" {
22+
scripts = [
23+
"install-fleur.sh",
24+
"clean-apt-files.sh",
25+
]
26+
}
27+
28+
post-processor "docker-tag" {
29+
repository = "fleur"
30+
tags = ["debian-12"]
31+
}
32+
}

‎examples/own-vm-images/flush-disk.sh‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#!/usr/bin/env bash
2+
# must discard the now unused blocks from the disk
3+
4+
dd if=/dev/zero of=/zero bs=4M || true
5+
sync
6+
rm -f /zero

‎examples/own-vm-images/hcloud-debian-12-fleur.pkr.hcl‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
packer {
2+
required_plugins {
3+
hcloud = {
4+
source = "github.com/hetznercloud/hcloud"
5+
version = "~> 1.6"
6+
}
7+
}
8+
}
9+
10+
variable "hcloud_token" {
11+
type = string
12+
sensitive = true
13+
default = "${env("HCLOUD_TOKEN")}"
14+
}
15+
16+
locals {
17+
current_timestamp = "${formatdate("YYYY-MM-DD-hhmm", timestamp())}Z"
18+
}
19+
20+
source "hcloud" "debian" {
21+
token = var.hcloud_token
22+
23+
image = "debian-12"
24+
location = "fsn1"
25+
server_type = "ccx13" # you need at least 8 Gb of RAM
26+
server_name = "fleur-build-${local.current_timestamp}"
27+
28+
# keep disk small
29+
user_data = <<-EOF
30+
#cloud-config
31+
growpart:
32+
mode: "off"
33+
resize_rootfs: false
34+
EOF
35+
36+
ssh_username = "root"
37+
38+
snapshot_name = "fleur-debian-${local.current_timestamp}"
39+
}
40+
41+
build {
42+
sources = ["source.hcloud.debian"]
43+
44+
provisioner "shell" {
45+
inline = [
46+
"cloud-init status --wait --long"
47+
]
48+
valid_exit_codes = [0, 2]
49+
}
50+
51+
provisioner "shell" {
52+
inline = [
53+
"apt-get update",
54+
]
55+
}
56+
57+
provisioner "shell" {
58+
scripts = [
59+
"install-fleur.sh",
60+
"reset-cloud-init.sh",
61+
"reset-ssh-host-keys.sh",
62+
"clean-apt-files.sh",
63+
"clean-logs.sh",
64+
"clean-root.sh",
65+
"flush-disk.sh",
66+
]
67+
}
68+
}

‎examples/own-vm-images/install-fleur.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env bash
2+
# build and install fleur and inpgen
3+
4+
set -euxo pipefail
5+
6+
FLEUR_REPO="https://iffgit.fz-juelich.de/fleur/fleur.git"
7+
FLEUR_VERSION="MaX-R6.2"
8+
DEPS=(libarpack2 libgomp1 libscalapack-openmpi2.2 openmpi-bin openmpi-common)
9+
# this will be removed after compilation
10+
BUILD_DEPS=(build-essential cmake doxygen gfortran git libarpack2-dev libopenblas-dev libopenmpi-dev libscalapack-mpi-dev libxc-dev libxml2-dev)
11+
BUILD_DEPS_DO_NOT_REMOVE=(python3-minimal)
12+
WORKDIR=$(mktemp -d)
13+
14+
export DEBIAN_FRONTEND=noninteractive
15+
export FC=mpifort
16+
export CXX=mpicxx
17+
export CC=mpicc
18+
19+
# install deps
20+
apt-get update
21+
apt-get -y install "${DEPS[@]}" "${BUILD_DEPS[@]}" "${BUILD_DEPS_DO_NOT_REMOVE[@]}"
22+
23+
# build in ram
24+
mount -t tmpfs -o size=6G fleur_build "$WORKDIR"
25+
df -h
26+
27+
# build and install fleur
28+
git clone --depth 1 --branch "$FLEUR_VERSION" "$FLEUR_REPO" "$WORKDIR"
29+
pushd "$WORKDIR"
30+
./configure.sh -libxc TRUE -hdf5 TRUE
31+
pushd build
32+
make -j 4
33+
install -t /usr/local/bin fleur_MPI inpgen
34+
ln -s /usr/local/bin/fleur_MPI /usr/local/bin/fleur
35+
popd
36+
popd
37+
38+
# cleanup
39+
umount "$WORKDIR"
40+
rm -rf "$WORKDIR"
41+
apt-get -y remove "${BUILD_DEPS[@]}"

‎examples/own-vm-images/reset-cloud-init.sh‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/usr/bin/env bash
2+
# reset cloud init state
3+
4+
cloud-init clean --logs --machine-id --seed
5+
6+
rm -rf /run/cloud-init/*
7+
rm -rf /var/lib/cloud/*

0 commit comments

Comments
 (0)