Skip to content

HOLD FOR RELEASE: Add CMX VMs beta docs #3407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

HOLD FOR RELEASE: Add CMX VMs beta docs #3407

wants to merge 20 commits into from

Conversation

hyu
Copy link
Contributor

@hyu hyu commented Jul 28, 2025

No description provided.

@hyu hyu requested a review from a team as a code owner July 28, 2025 14:37
Copy link

netlify bot commented Jul 28, 2025

Deploy Preview for replicated-docs ready!

Name Link
🔨 Latest commit bf90ee8
🔍 Latest deploy log https://app.netlify.com/projects/replicated-docs/deploys/689120b84887c20007af4d03
😎 Deploy Preview https://deploy-preview-3407--replicated-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@replicated-ci replicated-ci added type::docs Improvements or additions to documentation type::feature labels Jul 28, 2025
Copy link

netlify bot commented Jul 28, 2025

Deploy Preview for replicated-docs-upgrade ready!

Name Link
🔨 Latest commit bf90ee8
🔍 Latest deploy log https://app.netlify.com/projects/replicated-docs-upgrade/deploys/689120b8bf28d500085efc43
😎 Deploy Preview https://deploy-preview-3407--replicated-docs-upgrade.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hyu hyu marked this pull request as draft July 28, 2025 15:17
replicated vm port rm VMID_OR_VMNAME
```

## Connect a CMX VM with a CMX Cluster
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oo fancy.
Makes me wonder if we might want to consider how this content fits with or overlaps with https://docs.replicated.com/vendor/testing-ingress.
As in, do we want separate VMs and cluster networking pages?, where would this info live that touches on both?, do we want the headlines framed around "networking", "accessing your app in the cluster/VM", or maybe something else? etc

I don't really think this needs to be addressed as part of this PR, but something we can talk through

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's discuss this! Very good questions.

ssh -T [email protected]
```

1. For the prompt `Are you sure you want to continue connecting (yes/no/[fingerprint])?`, type `yes` and press Enter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. For the prompt Are you sure you want to continue connecting (yes/no/[fingerprint])?, type yes and press Enter.

I got this output when I was testing. I assume you are just supposed to say Yes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note from 8/4: we should clarify that you may or may not see this prompt.


```bash
replicated vm create --distribution ubuntu --version 20.04 --ssh-public-key ~/.ssh/id_rsa.pub --ssh-public-key ~/.ssh/id_ed25519.pub
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To automate the creation of VMs in your CI/CD workflows, you can use the flag --ssh-public-key to provide the SSH public key for a GitHub service account.

This section doesn't have any info about how to set up the service account/use it in your CI/CD workflows. I think that's fine for the first iteration of these docs, but one thing that might be important is how would the Vendor Portal be made aware of this service account?
Would we like to recommend that someone on the team needs to add the name of the service account in their Vendor Portal account? Or would be prefer the username override method?

replicated vm create --distribution ubuntu --version 22.04 --disk 50 --instance-type r1.medium
```

## Connect to a VM
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I did some reorg here so that you have three methods listed for how to connect to a VM (previously, the steps on connecting manually were elsewhere).

I also moved the steps on how to transfer files to their own h2 section. My assumption was that it would be worthwhile to include steps just for how to connect, without also telling people how to transfer. If that's not a valid use case though, we can combine the "connect" and "copy files" sections into one. LMK.

@paigecalvert paigecalvert marked this pull request as ready for review August 1, 2025 19:47

* [**Direct SSH**](#direct-ssh): When you connect to a VM using direct SSH, you can use your SSH tool of choice and pass any client supported flags, without any added connection lag of being routed through the Compatibility Matrix Forwarder. Example use cases for direct SSH include transferring large assets such as air gap bundles to the VM using SCP, or passing specific SHH flags during testing workflows.

* [**Connect to a VM Manually**](#connect-to-a-vm-manually): If the above options are not supported with your preferred SSH client, you can connect to a VM manually.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ this "connect manually" doesn't need to be a totally different option. instead, just add it as an alternative to the vm ssh-endpoint command

:::note
You can also get the SSH endpoint and port in JSON format by running `replicated vm ls --output json`.
:::
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ new note that replaces the "connect manually" steps

Copy link
Contributor

@paigecalvert paigecalvert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hyu updated based on our convo today and approved! marking as "hold for release", and I'll leave it to you to merge when you all are ready :)

@paigecalvert paigecalvert changed the title Add CMX VMs beta docs HOLD FOR RELEASE: Add CMX VMs beta docs Aug 4, 2025
Creating VMs with Compatibility Matrix has the following limitations:

- Creating VMs with Compatibility Matrix is a Beta feature.
- Installing Embedded Cluster on a VM created with Compatibility Matrix is supported for Embedded Cluster versions 1.21.0 or later. To reboot a Compatibility Matrix VM, you can run the Embedded Cluster [reset](embedded-using#reset-a-node) command.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We support rebooting, sudo reboot works fine and is supported. I don't think we need to mention the reboot portion at all

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed 👍

To automate the creation of VMs in your CI/CD workflows, you can use the flag `--ssh-public-key` to provide the SSH public key for a GitHub service account. For example:
```bash
replicated vm create --distribution ubuntu --version 20.04 --ssh-public-key ~/.ssh/id_rsa.pub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might make more sense for the version to be one we support in the example:

Suggested change
replicated vm create --distribution ubuntu --version 20.04 --ssh-public-key ~/.ssh/id_rsa.pub
replicated vm create --distribution ubuntu --version 24.04 --ssh-public-key ~/.ssh/id_rsa.pub

Using multiple SSH public keys:
```bash
replicated vm create --distribution ubuntu --version 20.04 --ssh-public-key ~/.ssh/id_rsa.pub --ssh-public-key ~/.ssh/id_ed25519.pub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
replicated vm create --distribution ubuntu --version 20.04 --ssh-public-key ~/.ssh/id_rsa.pub --ssh-public-key ~/.ssh/id_ed25519.pub
replicated vm create --distribution ubuntu --version 24.04 --ssh-public-key ~/.ssh/id_rsa.pub --ssh-public-key ~/.ssh/id_ed25519.pub

```
Where:
* `NAME` is any name for the VM. If `--name` is excluded, a name is automatically generated for the cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `NAME` is any name for the VM. If `--name` is excluded, a name is automatically generated for the cluster.
* `NAME` is any name for the VM. If `--name` is excluded, a name is automatically generated for the VM.


```
ID NAME DISTRIBUTION VERSION STATUS CREATED EXPIRES COST
760a30b1 laughing_tu ubuntu 24.04 queued 2025-01-28 16:07 PST - $0.60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This output is inaccruate and is missing the NETWORK column, for example:

ID          NAME                           DISTRIBUTION    VERSION       STATUS          NETWORK      CREATED                           EXPIRES                           COST
760a30b1    suspicious_poitras             ubuntu          24.04         assigned        accbd6a7     2025-08-04 13:24 PDT              -                                 $0.60


```
ID NAME STATUS CREATED EXPIRES
accbd6a7 affect_mend running 2025-01-28 16:04 PST 2025-01-28 17:05 PST
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This table output is not lined up, and the output is an old version of network ls, here's what it should look like:

ID          NAME                           STATUS          CREATED                           EXPIRES                           POLICY                            REPORTING
accbd6a7    suspicious_poitras             preparing       2025-08-04 13:24 PDT              -                                 open                              off


## Create and Manage Clusters
For information about creating VMs with Compatibility Matrix to test Replicated Embedded Cluster installers, air-gap installations, or when you need full OS control, see [Create VMs](/vendor/testing-vm-create).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Airgap is referenced here but we never talk about it in the Create VMs page.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah I had noticed that too and forgot to remove it. Good catch!

Where:
* `NAME` is any name for the VM. If `--name` is excluded, a name is automatically generated for the cluster.
* `DISTRIBUTION` is the operating system distribution for the VM (e.g., ubuntu, almalinux).
* `VERSION` is the version of the distribution to provision (e.g., 20.04, 22.04 for Ubuntu).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `VERSION` is the version of the distribution to provision (e.g., 20.04, 22.04 for Ubuntu).
* `VERSION` is the version of the distribution to provision (e.g., 22.04, 24.04 for Ubuntu).

### After Connecting to the VM with the Forwarder

:::note
Transferring files using Compatibility Matrix Forwarder is slower than using direct SSH. Compatibility Matrix servers run on EKS, so depending on your location, using the Forwarder adds latency. If you want to transfer large files such as air gap bundles onto the VM, use direct SSH in combination with SCP. See [Using the SCP Endpoint](#using-the-scp-endpoint) above.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should expose here that this runs on EKS.


You can make a Compatibility Matrix cluster available on the same network as a Compatibility Matrix VM.

**Compatible clusters:** Openshift, K3s, RKE2, EC, kURL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Compatible clusters:** Openshift, K3s, RKE2, EC, kURL
**Compatible clusters:** Openshift, K3s, RKE2, EC, kURL, kind

Comment on lines +53 to +58
Example output:

```
ID NAME DISTRIBUTION VERSION STATUS CREATED
EXPIRES COST
b09cf035 affect_mend k3s 1.32.0 queued 2025-01-28 16:04 PST - $0.60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should already show the network id?

760a30b1 laughing_tu ubuntu 24.04 queued 2025-01-28 16:07 PST - $0.60
```

In this example, both the cluster `b09cf035` and the VM `760a30b1` are now on the same tailnet.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this example, both the cluster `b09cf035` and the VM `760a30b1` are now on the same tailnet.
In this example, both the cluster `b09cf035` and the VM `760a30b1` are now on the same network.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type::docs Improvements or additions to documentation type::feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants