Skip to content

Commit 73d7282

Browse files
andresmmujicasandrobonazzola
andresmmujica
authored and
sandrobonazzola
committed
Create installing_hosted_engine_after_redhat.md
1 parent 3249141 commit 73d7282

File tree

1 file changed

+333
-0
lines changed

1 file changed

+333
-0
lines changed

‎source/develop/installing_hosted_engine_after_redhat.md‎

Lines changed: 333 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,333 @@
1+
## 1. Introduction
2+
3+
This document provides step-by-step guidance for deploying oVirt with a hosted engine in a fully virtualized environment. This setup is **not recommended for production** due to the complexities and known issues related to oVirt's recent challenges following Red Hat's discontinuation of CentOS and its impact on upstream projects like oVirt. While this guide aims to simplify the process, expect potential hurdles due to mismatches between official documentation and recent oVirt releases.
4+
5+
## 2. Tested and validated Versions
6+
7+
The following table lists the specific versions of oVirt components, ISO files, and RPM packages that have been verified to work during the installation process. These versions were tested and confirmed to deploy the hosted engine successfully.
8+
9+
| **Component** | **Version** | **Description** | **Download Link** |
10+
|-----------------------------|----------------------------------------------------|------------------------------------------------------|-------------------|
11+
| **oVirt Engine Appliance RPM** | `ovirt-engine-appliance-4.5-20240817071039.1.el9.x86_64` | The oVirt Engine appliance RPM package used for deployment. | [Download RPM](https://resources.ovirt.org/repos/ovirt/github-ci/ovirt-appliance/el9/ovirt-engine-appliance-4.5-20240817071039.1.el9.x86_64.rpm) |
12+
| **oVirt ISO** | `ovirt-node-ng-installer-4.5.6-2024081806.el9.iso` | The oVirt ISO file used for installing the environment. | [Download ISO](https://resources.ovirt.org/repos/ovirt/github-ci/ovirt-node-ng-image/ovirt-node-ng-installer-4.5.6-2024081806.el9.iso) |
13+
| **oVirt Engine Backend** | `ovirt-engine-backend-4.5.7-0.master.20240719063419.git7d868bf487.el9` | Backend engine of the oVirt Engine used in the deployment. | N/A |
14+
| **oVirt Engine Tools** | `ovirt-engine-tools-4.5.7-0.master.20240719063419.git7d868bf487.el9` | The oVirt Engine tools package. | N/A |
15+
| **oVirt Engine WildFly** | `ovirt-engine-wildfly-24.0.1-1.el9.x86_64` | WildFly application server used in oVirt deployment. | N/A |
16+
| **oVirt Engine AAA JDBC Extension** | `ovirt-engine-extension-aaa-jdbc-1.3.1-0.142.202307281505.el9.noarch` | DBC-based authentication extension for oVirt Engine, used to integrate external databases for user authentication. | N/A |
17+
18+
19+
## 3. Setting Up the Virtualized Environment
20+
21+
In this step, we install oVirt Node using the latest available image at the time of writing and deploy the hosted engine on the same node. This fully virtualized environment is intended for **testing purposes** only, not for production use.
22+
23+
| **Resource** | **Minimum Requirement** | **Notes** |
24+
|---------------------|--------------------------|------------------------------------------------|
25+
| **CPU** | 4 vCPUs | Adequate for managing oVirt Engine and VMs |
26+
| **Disk Space** | 80 GB | Sufficient for oVirt and hosted engine |
27+
| **Network Interfaces** | 2 NICs | 1 NAT (for local access), 1 Bridged (for external access) |h
28+
29+
Ensure static IPs and DNS are correctly configured to avoid networking issues during the hosted engine setup.
30+
31+
### 3. Network Configuration
32+
33+
Having a properly configured network is crucial for a successful oVirt deployment. Both **DNS** and **NTP** must be correctly set up to ensure smooth communication between the oVirt Engine and the hypervisor nodes.
34+
35+
#### DNS Setup
36+
Ensure that both forward (A) and reverse (PTR) DNS records are configured and resolvable from the hypervisors and the oVirt Engine, regardless of their network. Incorrect DNS resolution can lead to communication and authentication failures during the hosted engine setup.
37+
38+
#### Testbed Environment Setup
39+
In our testbed environment, we used **libvirt's dnsmasq** and manually configured `/etc/hosts` for the host (where the ovirt node vm is running), the ovirt node itself so it provides dns to the Hosted Engine, and finally for the engine but please note that the deployment process does this setup by itself. The critical point is to ensure that each could resolve the others by FQDN.
40+
41+
Example `/etc/hosts` configuration:
42+
```bash
43+
127.0.0.1 localhost localhost.localdomain
44+
::1 localhost localhost.localdomain
45+
10.0.3.15 node1.ovirt.local node1
46+
10.0.3.18 engine.ovirt.local engine
47+
```
48+
This approach works in small setups, but for production environments, a dedicated DNS service is recommended.
49+
50+
#### NTP Configuration
51+
Accurate time synchronization is equally important for avoiding issues in distributed systems. Ensure all nodes and the engine are using the same NTP server for time synchronization.
52+
53+
---
54+
55+
## 4. NFS Storage setup
56+
57+
In order to have a successful deployment is critical to ensure the proper setup of your storage, following you will find the instructions to setup an Ubuntu server as NFS Server compatible with oVirt. Independent on what do you use for the NFS Storage or the shared storage for that matters is critical to check the proper access from the nodes as explained later in this document.
58+
59+
#### Setting NFS Server
60+
61+
**Ubuntu 22.04/24.04**
62+
63+
Edit /etc/default/nfs-kernel-server and remove the --manage-gids option from rpc.mountd
64+
65+
```console
66+
#RPCMOUNTDOPTS="--manage-gids"
67+
RPCMOUNTDOPTS=""
68+
```
69+
Edit /etc/export and configure your exported resource as follows:
70+
71+
```console
72+
/storage *(rw,insecure,all_squash,anonuid=36,anongid=36)
73+
```
74+
75+
The insecure flag is required when the connection from your hosts come from a port >1024, which is the case in virtualized environments.
76+
The all_squash is to force all connections to use the defined anonuid and anongid ids.
77+
78+
Restart the NFS server.
79+
80+
```console
81+
systemctl restart nfs-kernel-server
82+
```
83+
84+
You can go to 6. and test the storage at once or wait until you have your OVirt node deployed to test it from there.
85+
86+
## 5. OVirt Node installation
87+
88+
This is the step-by-step installation process of oVirt with a self-hosted engine. It is essential to ensure network and DNS resolution are configured properly before proceeding with the steps outlined below, as well as the proper access to the shared storage as explained.
89+
90+
---
91+
92+
#### Step 1: Install the ISO
93+
94+
Begin by installing the oVirt Node ISO on your system. The following system configuration is recommended:
95+
96+
- **RAM**: Minimum of 16 GB
97+
- **Disk**: 80 GB
98+
- **Network Interface Cards (NICs)**: In the testbed environment we used two NICs, one NAT for libvirt and the other for public access, however this may vary depending your needs. Just have in mind the need of proper DNS resolution and connectivity.
99+
100+
Once you have launched the VM or physical system with the oVirt Node ISO, proceed with the following:
101+
102+
1. Use the entire drive for the installation.
103+
2. Configure both network cards (ensure one is for public access and one for storage/private networking).
104+
3. Assign a **hostname** (e.g., `node1.ovirt.local`), and set a **password** for access.
105+
106+
---
107+
108+
#### Step 2: Post-Installation Configuration
109+
110+
Once the installation is complete, SSH into the newly installed system and perform the following checks:
111+
112+
1. **DNS Resolution**: Ensure proper resolution for both A and PTR records by checking network connectivity and hostname resolution.
113+
114+
Example:
115+
```bash
116+
ping engine.ovirt.local
117+
```
118+
119+
2. **Edit `/etc/hosts`**: Make sure `/etc/hosts` contains the correct IP-to-hostname mappings for the engine and the node itself.
120+
Example:
121+
```bash
122+
127.0.0.1 localhost
123+
10.0.3.15 node1.ovirt.local
124+
10.0.3.18 engine.ovirt.local
125+
```
126+
127+
3. **Edit `/etc/resolv.conf`**: Ensure the correct DNS server is set up:
128+
```bash
129+
nameserver 192.168.122.1
130+
```
131+
As we are using Libvirt's Dnsmasq in this setup, the nameserver is 192.168.122.1 but that may vary in your environment, even if it's also KVM/Libvirt.
132+
133+
4. **Test Connectivity**: Ensure access to the shared storage (NFS/GlusterFS/etc.) and verify that network services are working.
134+
135+
```bash
136+
host engine.ovirt.local
137+
showmount -e 10.0.3.16
138+
```
139+
140+
---
141+
142+
#### Step 3: Install Necessary Packages
143+
144+
Before setting up the oVirt engine, install the necessary packages by adding the appropriate repository and syncing with the correct versions:
145+
146+
```bash
147+
dnf copr enable -y ovirt/ovirt-master-snapshot centos-stream-9
148+
dnf install -y ovirt-release-master
149+
dnf distro-sync --nobest
150+
dnf install ovirt-engine-appliance
151+
```
152+
153+
This ensures you are using the correct nightly build and package versions for a smooth deployment according to the latest recommendation from the developers. [1|https://www.ovirt.org/develop/dev-process/install-nightly-snapshot.html]
154+
155+
The ovirt-engine-appliance tested is the one referred at the beginning of this document, it may not work with a different version, if that's the case download the RPM indicated and install it directly on the OVirt node.
156+
157+
---
158+
159+
#### Step 4: Prepare Ansible Variables & Start the Hosted Engine Setup
160+
161+
Create an `ansible-vars.yml` file to include critical options for the hosted engine setup:
162+
163+
```bash
164+
cat /root/ansible-vars.yml
165+
---
166+
he_pause_host: true
167+
he_offline_deployment: true
168+
he_pause_before_engine_setup: true
169+
he_enable_keycloak: false
170+
he_debug_mode: true
171+
```
172+
173+
The one that you really need is the he_offline_deployment because this avoids the updating process that mess up the installation, however in order to check the advance of the installation and validate that everything is going well you may need the pause and debug options as well. I used the he_enable_keycloak but it didn't work, so pay attention to that question and answer no because in my case failed in every attempt.
174+
175+
To ensure the installation process runs smoothly and avoids package updates during deployment, use the following command to start the hosted engine deployment:
176+
177+
```bash
178+
tmux
179+
hosted-engine --deploy --4 --ansible-extra-vars="@/root/ansible-vars.yml"
180+
```
181+
182+
It is recommended to use `tmux` to avoid losing your session if disconnected. This will allow the process to continue running in the background.
183+
184+
---
185+
186+
#### Step 5: Critical Questions During Installation
187+
188+
The hosted engine deployment will ask several important questions. Below are key questions and answers:
189+
190+
- **Please indicate the gateway IP address**:
191+
Use the IP of the gateway on the public interface (the one providing internet access).
192+
Example: `10.0.3.1`
193+
194+
**NOTE: This must be an existing gateway**
195+
196+
197+
- **Configure Keycloak integration on the engine**:
198+
Choose **No** to avoid complications with Keycloak integration (this has been known to cause issues).
199+
Example: `
200+
201+
```bash
202+
Configure Keycloak integration on the engine (Yes, No) [Yes]: No
203+
```
204+
205+
**NOTE: This is a hard NO, it didn't work at all in my tests no matter the versions used.**
206+
207+
- **Please indicate a NIC to set ovirtmgmt bridge**:
208+
Choose the NIC that will act as the management bridge for your public network.
209+
Example: `enp1s0`
210+
211+
**NOTE: This must be the external network card, not the NATed one.**
212+
213+
- **Engine VM FQDN**:
214+
Provide the fully qualified domain name (FQDN) for the engine VM. This must resolve correctly (both A and PTR records).
215+
216+
Example:
217+
```bash
218+
Engine VM FQDN: engine.ovirt.local
219+
```
220+
**NOTE: This must be the right DNS record assigned to your engine and this MUST resolve properly.**
221+
222+
- **How should the engine VM network be configured? (DHCP, Static)**:
223+
Use **Static** for better control.
224+
Example: `Static`
225+
226+
**NOTE: Unless you have proper configured your DHCP to reserve and assign the right address, use Static.**
227+
228+
- **Please enter the IP address to be used for the engine VM**:
229+
Provide the static IP that resolves to the engine’s FQDN.
230+
Example: `10.0.3.14`
231+
232+
**NOTE: This must be the right IP address assigned to your engine and this MUST resolve properly.**
233+
234+
- **Engine VM DNS**:
235+
Provide a comma-separated list of DNS servers the engine should use.
236+
Example: `192.168.122.1,10.0.3.3`
237+
238+
**NOTE: As we are using Libvirt's Dnsmasq the first DNS is that one, it may vary in your setup, I just repeat it again, it needs to properly resolve the engine's IP address.**
239+
240+
- **Check FQDN resolution**:
241+
The installer will verify if the engine VM's FQDN resolves. Ensure it resolves to the correct IP address before proceeding.
242+
243+
Example:
244+
```bash
245+
[ INFO ] The Engine VM FQDN was resolved into: '10.0.3.14'
246+
```
247+
248+
- **Storage Configuration**:
249+
The next step in the deployment process is configuring shared storage. Make sure the shared storage (NFS, GlusterFS, etc.) is accessible and properly configured.
250+
251+
252+
## 6. NFS Storage testing process before continue
253+
254+
255+
From your hosts mount the nfs storage and test with root, vdsm and sanlock users that you have full access to the shared resource.
256+
257+
```console
258+
259+
mount nfs_ip_address:/storage /mnt
260+
cd /mnt
261+
touch test_root
262+
ls -l test_root # Must look like this: -rw-r--r--. 1 vdsm kvm 0 Oct 5 17:22 test_root
263+
rm test_root
264+
sudo su - vdsm -s /bin/bash
265+
cd /mnt
266+
touch test_vdsm
267+
ls -l test_vdsm # Must look like this: -rw-r--r--. 1 vdsm kvm 0 Oct 5 17:22 test_vdsm
268+
rm test_vdsm
269+
sudo su - ceph -s /bin/bash
270+
cd /mnt
271+
touch test_ceph
272+
ls -l test_ceph # Must look like this: -rw-r--r--. 1 vdsm kvm 0 Oct 5 17:22 test_ceph
273+
rm test_ceph
274+
sudo su - sanlock -s /bin/bash
275+
cd /mnt
276+
touch test_sanlock
277+
ls -l test_sanlock # Must look like this: -rw-r--r--. 1 vdsm kvm 0 Oct 5 17:22 test_sanlock
278+
rm test_sanlock
279+
```
280+
281+
This assumes there is full connectivity between your hosts and the NFS server and all the firewalls involved are properly setup.
282+
283+
The engine doesn't really need access to the storage, just the nodes but it won't harm you to test from there too.
284+
285+
---
286+
287+
## 7: Continue the host-deploy process:
288+
289+
During the storage configuration step, ensure that the shared storage is available and reachable from the host and the engine VM. The deployment will prompt you to provide details for the storage domain, such as the NFS server and path:
290+
291+
- **Please indicate where the storage should be mounted**:
292+
Provide the mount path for the NFS or other shared storage location where the engine VM will reside.
293+
294+
Example:
295+
```bash
296+
Storage mount path: 10.0.3.16:/engine_storage
297+
```
298+
299+
Ensure that the NFS or other storage protocol is properly configured, permissions are set, and network connectivity allows access from both the host and the engine VM.
300+
301+
---
302+
303+
## 8: Completing the Hosted Engine Setup
304+
305+
Once the engine VM starts, the deployment process will continue with configuring the shared storage, moving the engine VM to shared storage, and initializing services. You will see tasks like:
306+
307+
```bash
308+
TASK [ovirt.ovirt.hosted_engine_setup : Copy local VM disk to shared storage]
309+
[ INFO ] changed: [localhost]
310+
```
311+
312+
As the process continues, the engine VM will be initialized, and services like `ovirt-ha-agent` will start:
313+
314+
```bash
315+
TASK [ovirt.ovirt.hosted_engine_setup : Start ovirt-ha-agent service on the host]
316+
[ INFO ] changed: [localhost]
317+
```
318+
319+
After the process completes successfully, you will see the following message:
320+
321+
```bash
322+
[ INFO ] Hosted Engine successfully deployed
323+
```
324+
325+
At this point, the self-hosted engine setup is complete, and you can begin managing your virtual infrastructure using the oVirt engine.
326+
327+
---
328+
329+
## Summary
330+
331+
By following these steps, you will have successfully deployed oVirt with a self-hosted engine in your environment. Pay special attention to DNS resolution, network configuration, and storage access to avoid issues during the installation. Use `tmux` to ensure the process runs uninterrupted, and leverage the `he_offline_deployment` variable to prevent unexpected updates during the setup.
332+
333+
```

0 commit comments

Comments
 (0)