Converting Omnia to Ansible Collections

[lwilson]

Before discussions were enabled we had talked about the possibility of using Ansible collections in #139. We did not go to collections at that time, but did refactor the code in an attempt to reduce the number of independent roles.

@j0hnL and I have discussed how we could potentially break the repository up into different collections under the omnia namespace (see Omnia collections).

It might make sense to make the common plays, which has overlap between Slurm and Kubernetes, in both the slurm and kubernetes collections, in order to allow each of those collections to run independently.

Moving to collections would allow collaborators/users to take advantages of parts of the Omnia framework, without having the entire set of plays as a monolithic piece. I think this would lead to greater adoption of Omnia, as not all sites need all components.

[lwilson]

Created a wiki page for continuing to iterate on the structure of Omnia collections.

[sujit-jadhav]

Under omnia.provision, in Roles we need to add "iDRAC" along with cobbler and ww4.

[lwilson]

Defininitely need to add omnia.provision.idrac. The list was not intended to be complete, but more of an example. I'll add it on the wiki.

[rajeevarakkal]

@lwilson
Apart from this internal re-structuring, you can refer the ansible collection checklist for a quick overview of collection requirements

[lwilson]

Thanks @rajeevarakkal. We'll definitely make sure we follow the guidelines as we migrate from roles to collections.

[lwilson]

Ansible has now enabled use of playbooks from collections (see ansible/ansible#68844 and ansible/ansible#67435), so I think it would be worthwhile to use the capability for tying together the various roles within each collection.

For example, the omnia.wlm collection (see Omnia collections) could have a playbook kubernetes.yml which takes advantages of the roles omnia.wlm.k8s_manager and omnia.wlm.k8s_worker. It could be called from a top-level cluster builder script (e.g., create.yml playbook in an omnia.cluster collection) by using import_playbook:

-name: Deploy accelerators
 import_playbook: omnia.accelerator.deploy.yml

-name: Deploy Slurm
 import_playbook: omnia.wlm.slurm.yml
 tags: slurm

-name: Deploy Kubernetes
 import_playbook: omnia.wlm.kubernetes.yml
 tags: kubernetes

-name: Deploy Kubeflow
 import_playbook: omnia.platforms.kubeflow.yml
 tags: kubeflow

The inventory could include host sets manager and worker, which would be used by omnia.wlm.kubernetes.yml to differentiate between the roles. This makes the top-level scripts (e.g., omnia.cluster.create.yml, omnia.cluster.destroy.yml, omnia.cluster.grow.yml, omnia.cluster.shrink.yml) much cleaner and easier to understand.

Tags could be placed at each import_playbook call so that the cluster deployment is customizable. If you want to deploy a kubernetes and kubeflow cluster using the above example:

ansible_playbook -i inventory_file omnia.cluster.create.yml --skip_tags slurm