Support AMD SEV-SNP on AWS #2424

fangge1212 · 2025-07-28T08:49:35Z

AMD SEV-SNP is one of the confidential computing technologies. This commit adds support for AMD SEV-SNP on AWS, so users can utilize the confidential computing on the cluster nodes.

Upstream CAPA PR: kubernetes-sigs/cluster-api-provider-aws#5598

openshift-ci · 2025-07-28T08:50:00Z

Hello @fangge1212! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

openshift-ci · 2025-07-28T08:53:30Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: fangge1212
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

bgartzi · 2025-07-29T08:38:17Z

machine/v1beta1/types_awsprovider.go

@@ -17,6 +17,9 @@ type AWSMachineProviderConfig struct {
 	AMI AWSResourceReference `json:"ami"`
 	// instanceType is the type of instance to create. Example: m4.xlarge
 	InstanceType string `json:"instanceType"`
+	// cpuOptions is the set of cpu options for the instance, where you can enable/disable AMD SEV-SNP on the instance.
+	// +optional
+	CpuOptions *CpuOptions `json:"cpuOptions,omitempty"`
 	// tags is the set of tags to add to apply to an instance, in addition to the ones


Hi @fangge1212, would it make sense to use a field named ConfidentialComputing (or something similar) that could be Disabled or AmdSevSnp instead?

In case AWS supports other confidential computing technologies in the future, that would be easier to extend than this approach IMO. As it would require adding another new field to the API instead of a new value to the enum.

Same as kubernetes-sigs/cluster-api-provider-aws#5598, waiting for more opinions before chaning it

everettraven · 2025-07-31T19:45:51Z

machine/v1beta1/types_awsprovider.go

+)
+
+// CpuOptions defines the cpu options for the instance.
+type CpuOptions struct {


A couple questions to help me better understand the direction this should take:

Because all fields are optional here, what happens if this is set to the zero value of the struct {}?

Because there is no constraints here as to how many properties can be specified at once, what would it mean if in the future there are multiple options to configure and I configure more than one of them?

If it is an empty struct, cpuOptions=nil will be passed to the AWS API

I don't get your question. If there are multiple options to configure in the future, then we can configure multiple options.

I don't get your question. If there are multiple options to configure in the future, then we can configure multiple options

What I was trying to get at is similar to this question upstream: kubernetes-sigs/cluster-api-provider-aws#5598 (comment)

If you are only ever going to have mutually exclusive CPU options then using an enum for those options likely makes sense.

If you are going to have a subset of mutually exclusive options but the rest can be specified together, then this structure is generally OK, but I'd think an enum is still reasonable for a single mutual exclusivity field.

Looking at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-specify-cpu-options.html I'd expect something like:

cpuOptions: cores: 4 threadsPerCore: 1 confidentialComputing: AmdSevSnp

to be reasonable (assuming that something like confidentialComputing would be mutually exclusive so you can't set something like AmdSevSnp and TDX if it were to be added in the future)

Updated to:

cpuOptions: confidentialComputing: AMDSevSnp

everettraven · 2025-08-07T17:13:02Z

machine/v1beta1/types_awsprovider.go

+type CPUOptions struct {
+	// confidentialCompute specifies whether confidential computing should be enabled for the instance,
+	// and, if so, which confidential computing technology to use.
+	// If set to Disabled, the instance will not use confidential computing.


"Disabled" is often times an overloaded term and we generally try to avoid using it where we can.

For scenarios like this, I generally go for "None" and would encourage that usage here as well.

IMO, confidentialCompute: None reads more intuitively than confidentialCompute: Disabled.

everettraven · 2025-08-07T17:15:54Z

machine/v1beta1/types_awsprovider.go

+	// If set to Disabled, the instance will not use confidential computing.
+	// If set to AMDSevSnp, the instance will be configured with AMD SEV-SNP.
+	// In this case, ensure the following conditions are met:
+	// 1) The selected instance type supports AMD SEV-SNP.
+	// 2) The selected AWS region supports AMD SEV-SNP.
+	// 3) The selected AMI supports AMD SEV-SNP.
+	// More details can be checked at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sev-snp.html
+	// If omitted, the platform will apply a default value — currently Disabled, but this may change over time.


Suggested change

// If set to Disabled, the instance will not use confidential computing.

// If set to AMDSevSnp, the instance will be configured with AMD SEV-SNP.

// In this case, ensure the following conditions are met:

// 1) The selected instance type supports AMD SEV-SNP.

// 2) The selected AWS region supports AMD SEV-SNP.

// 3) The selected AMI supports AMD SEV-SNP.

// More details can be checked at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sev-snp.html

// If omitted, the platform will apply a default value — currently Disabled, but this may change over time.

// When set to Disabled, the instance will not use confidential computing.

// When set to AMDSevSnp, the instance will be configured with AMD SEV-SNP.

// In this case, ensure the following conditions are met:

// 1) The selected instance type supports AMD SEV-SNP.

// 2) The selected AWS region supports AMD SEV-SNP.

// 3) The selected AMI supports AMD SEV-SNP.

// More details can be checked at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sev-snp.html

// When omitted, this means no opinion and the platform is left to choose a reasonable default, which is subject to change without notice. The current default is Disabled.

everettraven · 2025-08-07T17:43:49Z

machine/v1beta1/types_awsprovider.go

+)
+
+// CPUOptions defines the cpu options for the instance.
+type CPUOptions struct {


We try to avoid things like cpuOptions: {} being valid inputs because it is usually semantically the same as just not specifying the field altogether.

Add the +kubebuilder:validation:MinProperties=1 marker so that it is required for at least one property to be specified, making {} an invalid input.

When I add +kubebuilder:validation:MinProperties=1, kubeapilinter asked me to add tag omitzero and don't use make CPUOptions a pointer. I followed its prompt, but this makes many tests in cluster-api-provider-aws failed:

--- FAIL: TestSSHKeyName/SSH_key_name_is_nil_is_valid (0.01s) sshkeyname_test.go:89: ValidateCreate() error = AWSMachine.infrastructure.cluster.x-k8 "machine-9zsqb" is invalid: spec.cpuOptions: Invalid value: 0: spec.cpuOptions in body should have at least 1 properties, wantErr false

everettraven · 2025-08-07T17:45:29Z

machine/v1beta1/types_awsprovider.go

@@ -17,6 +17,9 @@ type AWSMachineProviderConfig struct {
 	AMI AWSResourceReference `json:"ami"`
 	// instanceType is the type of instance to create. Example: m4.xlarge
 	InstanceType string `json:"instanceType"`
+	// cpuOptions is the set of cpu options for the instance.
+	// +optional


What happens if this field is not specified by a user?

If unset, no CPU options are passed to the AWS platform and AWS default values are used.

everettraven · 2025-08-07T17:48:48Z

machine/v1beta1/types_awsprovider.go

@@ -17,6 +17,9 @@ type AWSMachineProviderConfig struct {
 	AMI AWSResourceReference `json:"ami"`
 	// instanceType is the type of instance to create. Example: m4.xlarge
 	InstanceType string `json:"instanceType"`
+	// cpuOptions is the set of cpu options for the instance.


I think we could improve the GoDoc here to be more descriptive.

https://github.com/openshift/enhancements/blob/master/dev-guide/api-conventions.md#write-user-readable-documentation-in-godoc is a good starting point for the things that you should take into consideration and include in the documentation for a field.

Thanks. I updated the doc, please have a look again.

fangge1212 · 2025-08-08T09:11:42Z

/retest-required

fangge1212 · 2025-08-11T02:31:21Z

/retest

damdo · 2025-08-11T16:01:10Z

machine/v1beta1/types_awsprovider.go

+const (
+	// AWSConfidentialComputePolicyNone disables confidential computing for the instance.
+	AWSConfidentialComputePolicyNone AWSConfidentialComputePolicy = "None"
+	// AWSConfidentialComputePolicySEVSNP enables AMD SEV-SNP as the confidential computing technology for the instance.
+	AWSConfidentialComputePolicySEVSNP AWSConfidentialComputePolicy = "AmdSevSnp"
+)


Let's follow the same approach we took for the GCP types: https://github.com/openshift/api/pull/2165/files#diff-9bd66f01f63b7ce0455fe993aa03dfe93964ac62b31278a6dbf3db2fcb7f792eR66-R71

In detail:

Let's use Enabled/Disabled (rather than None)

Let's fully spell out the AmdSevSnp to AMDEncryptedVirtualizationNestedPaging

I updated to Disabled;AMDEncryptedVirtualizationNestedPaging
I don't think Enabled is a meaningfull value, because all the other values than Disabled implies Enabled

Enabled is normally for the default value (so in this case OpenShift engineers choose the default for the user) as you can see here on the GCP one:

// When set to Enabled, the machine will be configured as a confidential computing instance with no preference on the confidential compute policy used. In this mode, the platform chooses a default that is subject to change over time. Currently, the default is to use AMD Secure Encrypted Virtualization.

Ideally we should try and keep consistency across providers for the same feature, so I'd be happy to see it added

Hi @damdo, @fangge1212!

The problem I see in the AWS case is that AFAICT, it doesn't support the default GCP case: AMD SEV. It only supports AMD SEV-SNP. In that case, we would end up defining Enabled as a valid case for AWS, but would have a different behavior than the one from GCP, configuring AMD SEV-SNP machines instead. In my opinion this would be misleading.

This is another discussion, but at the moment the Enabled option for GCP is redundant, as AMDEncryptedVirtualization achieves the same result. In my opinion, Deprecating Enabled in the GCP provider in the future would help getting rid of that redundancy and also having consistency across providers for the same feature, as you mention.

If we are thinking about deprecating Enabled then it is probably ok to leave it out from this one.
cc. @JoelSpeed

AMD SEV-SNP is one of the confidential computing technologies. This commit adds support for AMD SEV-SNP on AWS, so users can utilize the confidential computing on the cluster nodes. Signed-off-by: Fangge Jin <[email protected]>

openshift-ci · 2025-08-12T11:44:20Z

@fangge1212: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 28, 2025

openshift-ci bot requested review from everettraven and mandre July 28, 2025 08:53

fangge1212 force-pushed the aws_amd_sev_snp branch 2 times, most recently from 1271931 to a6478c1 Compare July 29, 2025 08:18

bgartzi reviewed Jul 29, 2025

View reviewed changes

everettraven reviewed Jul 31, 2025

View reviewed changes

fangge1212 force-pushed the aws_amd_sev_snp branch 3 times, most recently from 82e877d to 1df992a Compare August 6, 2025 22:44

everettraven reviewed Aug 7, 2025

View reviewed changes

fangge1212 force-pushed the aws_amd_sev_snp branch 5 times, most recently from bbef962 to e48669f Compare August 8, 2025 06:42

fangge1212 mentioned this pull request Aug 11, 2025

✨ Add support for AMD SEV-SNP instances kubernetes-sigs/cluster-api-provider-aws#5605

Open

5 tasks

damdo reviewed Aug 11, 2025

View reviewed changes

fangge1212 force-pushed the aws_amd_sev_snp branch 2 times, most recently from 607be56 to f28b17a Compare August 12, 2025 10:46

Support AMD SEV-SNP on AWS

520141d

AMD SEV-SNP is one of the confidential computing technologies. This commit adds support for AMD SEV-SNP on AWS, so users can utilize the confidential computing on the cluster nodes. Signed-off-by: Fangge Jin <[email protected]>

fangge1212 force-pushed the aws_amd_sev_snp branch from f28b17a to 520141d Compare August 12, 2025 11:09

Support AMD SEV-SNP on AWS #2424

Are you sure you want to change the base?

Support AMD SEV-SNP on AWS #2424

Uh oh!

Conversation

fangge1212 commented Jul 28, 2025

Uh oh!

openshift-ci bot commented Jul 28, 2025

Uh oh!

openshift-ci bot commented Jul 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fangge1212 commented Aug 8, 2025

Uh oh!

fangge1212 commented Aug 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

damdo Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Aug 12, 2025

Uh oh!

Uh oh!

damdo Aug 12, 2025 •

edited

Loading