feature: Add Public API Server Address Support #986

rossigee · 2025-10-09T00:45:06Z

Add Public API Server Address Support

Overview

This PR introduces support for specifying a custom public API server address for TenantControlPlane instances. This allows using DNS hostnames in cluster-info ConfigMaps and kubeconfigs instead of LoadBalancer IPs, enabling better certificate SAN matching and avoiding x509 certificate errors.

Key Changes

API Changes

Added PublicAPIServerAddress field to ServiceSpec in the TenantControlPlane API
Field is optional and allows specifying a custom hostname for the API server
Updated CRDs and generated code (deepcopy, etc.)

Implementation

Added PublicControlPlaneAddress() method to TenantControlPlane struct
Method returns the public address if specified, otherwise falls back to assigned address
Optimized port handling using cmp.Or for default port (6443)
Integrated public address usage in kubeconfig generation for controller-manager and scheduler components
Post-processes generated kubeconfigs to replace server URLs with the public hostname

Tests

Added comprehensive unit tests for PublicControlPlaneAddress() method
Added E2E test validating the feature in cluster scenarios
Tests cover various scenarios: default/custom ports, address set/unset, error cases

Documentation

Updated API reference docs to include the new field
Added field description with usage examples

Usage Example

apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
spec:
  controlPlane:
    service:
      serviceType: LoadBalancer
      publicAPIServerAddress: "k8s-api.example.com"
  # ... other spec

Benefits

Enables use of DNS names that match certificate SANs
Prevents x509 certificate errors in control plane components
Allows more flexible networking setups with custom hostnames

Breaking Changes

None. The field is optional and defaults to existing behavior.

Related Issues

Add PublicAPIServerAddress field to ServiceSpec for specifying custom API server hostnames. Implement PublicControlPlaneAddress method to return public address with fallback to assigned address. Integrate public address usage in kubeconfig generation for controller-manager and scheduler. Add comprehensive tests for the functionality.

- Regenerate CRDs to include the new PublicAPIServerAddress field in ServiceSpec. - Update API reference docs to document the new field.

netlify · 2025-10-09T00:45:11Z

✅ Deploy Preview for kamaji-documentation canceled.

Name	Link
🔨 Latest commit	`2bea3f5`
🔍 Latest deploy log	https://app.netlify.com/projects/kamaji-documentation/deploys/68f33d9743b68e00089341f1

charts/kamaji/crds/kamaji.clastix.io_datastores.yaml

prometherion

I'm a bit lost here: it seems the proposed public API Server address is not pushed into the kubeadm functions, such as:

kamaji/internal/resources/kubeadm_config.go

Line 106 in 99fb71e

    
           TenantControlPlaneEndpoint:      r.getControlPlaneEndpoint(tenantControlPlane.Spec.ControlPlane.Ingress, address, port),

and the following one for kubeconfig

kamaji/internal/resources/kubeconfig.go

Line 234 in 99fb71e

    
           config.InitConfiguration.ControlPlaneEndpoint = fmt.Sprintf("%s.%s.svc:%d", tenantControlPlane.Name, tenantControlPlane.Namespace, tenantControlPlane.Spec.NetworkProfile.Port)

prometherion · 2025-10-18T14:02:04Z

Makefile

 controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary.
 $(CONTROLLER_GEN): $(LOCALBIN)
-	test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.1
+	test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0


Suggested change

test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0

test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.1

Unless it's required, we need to stick to 0.16.1

prometherion · 2025-10-18T14:05:06Z

api/v1alpha1/tenantcontrolplane_public_address.go

+	// Fall back to the assigned control plane address, but use configured port
+	assignedAddress, _, err := in.AssignedControlPlaneAddress()
+	if err != nil {
+		return "", 0, err
+	}
+
+	return assignedAddress, port, nil


Suggested change

// Fall back to the assigned control plane address, but use configured port

assignedAddress, _, err := in.AssignedControlPlaneAddress()

if err != nil {

return "", 0, err

}

return assignedAddress, port, nil

// Fall back to the assigned control plane address

return in.AssignedControlPlaneAddress()

Unless I'm missing something, there's no need to replicate the same behavior of AssignedControlPlaneAddress.

prometherion · 2025-10-18T14:07:21Z

internal/resources/kubeconfig.go

+			// Post-process the kubeconfig to use the public API server address for
+			// controller manager and scheduler components instead of localhost/IP address
+			if strings.Contains(r.KubeConfigFileName, "controller-manager") || strings.Contains(r.KubeConfigFileName, "scheduler") {
+				kubeconfig, kcErr = r.replaceServerURLWithPublicAddress(kubeconfig, tenantControlPlane)
+				if kcErr != nil {
+					logger.Error(kcErr, "cannot replace server URL in kubeconfig")
+					return kcErr
+				}
+			}


Unless I'm missing something, this is not required: Kamaji leverages the concept of using loopback to communicate with the API Server; otherwise, we would end up with extra network hops with potential egress traffic.

prometherion · 2025-10-18T14:08:53Z

internal/resources/kubeconfig.go

+func (r *KubeconfigResource) usePublicAPIServerAddress(config *kubeadm.Configuration, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error {
+	// Use the public API server endpoint configured in the TenantControlPlane
+	// instead of localhost for controller manager and scheduler kubeconfigs.
+	// This ensures that internal control plane components connect using the proper hostname
+	// which matches the certificate SANs, rather than failing with x509 certificate errors.
+
+	// Note: We don't modify the kubeadm configuration here because LocalAPIEndpoint.AdvertiseAddress
+	// must be an IP address according to kubeadm validation. Instead, we'll post-process
+	// the generated kubeconfig to replace the server URL with the hostname.


Same discussion as we said before: Scheduler and Controller Manager must rely on the loopback address.

rossigee added 2 commits October 9, 2025 07:31

Update CRDs and API docs for PublicAPIServerAddress field.

d909318

- Regenerate CRDs to include the new PublicAPIServerAddress field in ServiceSpec. - Update API reference docs to document the new field.

rossigee mentioned this pull request Oct 9, 2025

feat: add pregenerated certificates support with comprehensive testing #928

Draft

prometherion requested changes Oct 10, 2025

View reviewed changes

charts/kamaji/crds/kamaji.clastix.io_datastores.yaml Show resolved Hide resolved

rossigee added 2 commits October 18, 2025 14:01

Merge branch 'master' into feature/public-address

61a6dc2

Update CRDs.

2bea3f5

rossigee requested a review from prometherion October 18, 2025 07:11

prometherion requested changes Oct 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feature: Add Public API Server Address Support #986

feature: Add Public API Server Address Support #986

Uh oh!

rossigee commented Oct 9, 2025

Uh oh!

netlify bot commented Oct 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

prometherion left a comment

Uh oh!

prometherion Oct 18, 2025

Uh oh!

prometherion Oct 18, 2025

Uh oh!

prometherion Oct 18, 2025

Uh oh!

prometherion Oct 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	test -s $(LOCALBIN)/controller-gen \|\| GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.19.0
	test -s $(LOCALBIN)/controller-gen \|\| GOBIN=$(LOCALBIN) CGO_ENABLED=0 go install -ldflags="-s -w" sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.1

feature: Add Public API Server Address Support #986

Are you sure you want to change the base?

feature: Add Public API Server Address Support #986

Uh oh!

Conversation

rossigee commented Oct 9, 2025

Add Public API Server Address Support

Overview

Key Changes

API Changes

Implementation

Tests

Documentation

Usage Example

Benefits

Breaking Changes

Related Issues

Uh oh!

netlify bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kamaji-documentation canceled.

Uh oh!

Uh oh!

prometherion left a comment

Choose a reason for hiding this comment

Uh oh!

prometherion Oct 18, 2025

Choose a reason for hiding this comment

Uh oh!

prometherion Oct 18, 2025

Choose a reason for hiding this comment

Uh oh!

prometherion Oct 18, 2025

Choose a reason for hiding this comment

Uh oh!

prometherion Oct 18, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

netlify bot commented Oct 9, 2025 •

edited

Loading