Do not abort import if source cluster is not ready yet during preflight checks

[votdev]

IMPORTANT: Please do not create a Pull Request without creating an issue first.

Problem:
We currently, whether that is Openstack Source or VMWare based, can't just give Harvester a big manifest that contains all the credentials/locations and vms to import.

It will fail to process and reconcile everything all at once.

Solution:
Reconcile VirtualMachineImport until the source cluster is ready and do NOT abort the import.

Related Issue:
harvester/harvester#8463

Test plan:
Prereqs

• Create a big YAML file that contains the Secret, OpenstackSource or VmwareSource and VirtualMachineImport.

Case 1

• Make sure the source cluster is not reachable (e.g. by shutting down the VPN).
• Apply the YAML file, e.g. k apply -f ~/Data/k8s/openstack_all.yaml.
• The logs of the vm-import-controller should look like

time="2025-06-24T10:59:05Z" level=info msg="Starting k8s.cni.cncf.io/v1, Kind=NetworkAttachmentDefinition controller"
time="2025-06-24T10:59:29Z" level=info msg="Reconciling source" kind=OpenstackSource name=devstack namespace=default
time="2025-06-24T10:59:29Z" level=info msg="Running preflight checks ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:29Z" level=warning msg="The source cluster is not ready yet" kind=openstacksource name=devstack status=
time="2025-06-24T10:59:29Z" level=error msg="error syncing 'default/cirros-tiny': handler virtualmachine-import-job-change: source cluster not yet ready, requeuing"
time="2025-06-24T10:59:29Z" level=info msg="Running preflight checks ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:29Z" level=warning msg="The source cluster is not ready yet" kind=openstacksource name=devstack status=
time="2025-06-24T10:59:29Z" level=error msg="error syncing 'default/cirros-tiny': handler virtualmachine-import-job-change: source cluster not yet ready, requeuing"
time="2025-06-24T10:59:29Z" level=info msg="Running preflight checks ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:29Z" level=warning msg="The source cluster is not ready yet" kind=openstacksource name=devstack status=
time="2025-06-24T10:59:29Z" level=error msg="error syncing 'default/cirros-tiny': handler virtualmachine-import-job-change: source cluster not yet ready, requeuing"
time="2025-06-24T10:59:29Z" level=info msg="Running preflight checks ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:29Z" level=warning msg="The source cluster is not ready yet" kind=openstacksource name=devstack status=
time="2025-06-24T10:59:29Z" level=error msg="error syncing 'default/cirros-tiny': handler virtualmachine-import-job-change: source cluster not yet ready, requeuing"
...

• Make sure the source cluster is reachable (e.g. by activating the VPN).
• After some while the VM import should continue.

...
time="2025-06-24T10:59:30Z" level=error msg="error syncing 'default/cirros-tiny': handler virtualmachine-import-job-change: source cluster not yet ready, requeuing"
time="2025-06-24T10:59:31Z" level=info msg="found 4 servers"
time="2025-06-24T10:59:31Z" level=info msg="Reconciling source" kind=OpenstackSource name=devstack namespace=default
time="2025-06-24T10:59:32Z" level=info msg="Running preflight checks ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:33Z" level=info msg="Checking the source network as part of the preflight checks" name=cirros-tiny namespace=default sourceNetwork=shared
time="2025-06-24T10:59:33Z" level=info msg="Sanitizing the import spec ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:34Z" level=info msg="The sanitization of the import spec was successful" kind=VirtualMachineImport name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny status.importedVirtualMachineName=cirros-tiny
time="2025-06-24T10:59:34Z" level=info msg="Importing client disk images ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:36Z" level=info msg="Shutting down guest OS of the source VM" name=cirros-tiny namespace=default spec.gracefulShutdownTimeoutSeconds=60 spec.sourceCluster.kind=OpenstackSource spec.sourceCluster.name=devstack spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:38Z" level=info msg="Importing client disk images ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:40Z" level=info msg="Importing client disk images ..." name=cirros-tiny namespace=default spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:41Z" level=info msg="Exporting source VM" name=cirros-tiny namespace=default spec.sourceCluster.kind=OpenstackSource spec.sourceCluster.name=devstack spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:42Z" level=info msg="Creating a new snapshot" name=cirros-tiny namespace=default opts.name=import-controller-cirros-tiny-0 opts.volumeID=e6565b2e-6f99-45e8-9278-fd4b4b35a1ea spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:43Z" level=info msg="Waiting for snapshot to be available" name=cirros-tiny namespace=default snapshot.id=2b78c386-cbfb-4285-8149-eae4595a30b0 snapshot.name=import-controller-cirros-tiny-0 snapshot.size=1 snapshot.volumeID=e6565b2e-6f99-45e8-9278-fd4b4b35a1ea spec.virtualMachineName=cirros-tiny
time="2025-06-24T10:59:45Z" level=info msg="Waiting for volume to be available" name=cirros-tiny namespace=default retryCount=30 retryDelay=10 spec.virtualMachineName=cirros-tiny volume.createdAt="2025-06-24 10:59:45.247009 +0000 UTC" volume.id=c4a182eb-1d16-40f6-9bd0-480a1f0896a4 volume.size=1 volume.snapshotID=2b78c386-cbfb-4285-8149-eae4595a30b0 volume.status=creating
...

[mergify]

This pull request is now in conflict. Could you fix it @votdev? 🙏