StorageSpec accessMode is not required but errors if not provided

[SDBrett]

If an ectdCluster is configured with a storageSpec and the accessMode field is not configured the statefulSet cannot create pods and provide the below error.

create Claim etcd-data-example-0 for Pod example-0 in StatefulSet example failed error: PersistentVolumeClaim "etcd-data-example-0" is invalid: spec.accessModes: Unsupported value: "": supported values: "ReadOnlyMany", "ReadWriteMany", "ReadWriteOnce", "ReadWriteOncePod"

Example etcdCluster config to get the above error

apiVersion: operator.etcd.io/v1alpha1
kind: EtcdCluster
metadata:
  name: example
  namespace: etcd-test
spec:
  size: 1
  version: v3.5.17
  etcdOptions:
    - "--experimental-peer-skip-client-san-verification"
  storageSpec:
    volumeSizeRequest: 100M

Snippet of statuefulSet created with above config

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: example
  namespace: etcd-test
spec:
  serviceName: example
      spec:
        accessModes:
          - ''
        resources:
          limits:
            storage: 100M
          requests:
            storage: 100M
        volumeMode: Filesystem
      status:
        phase: Pending

[shubhindia]

@SDBrett I just tried with the same spec as yours and it seems to be working fine.

❯ k get etcdcluster -n etcd-test example -o yaml
apiVersion: operator.etcd.io/v1alpha1
kind: EtcdCluster
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"operator.etcd.io/v1alpha1","kind":"EtcdCluster","metadata":{"annotations":{},"name":"example","namespace":"etcd-test"},"spec":{"etcdOptions":["--experimental-peer-skip-client-san-verification"],"size":1,"storageSpec":{"volumeSizeRequest":"100M"},"version":"v3.5.17"}}
  creationTimestamp: "2025-08-12T09:24:17Z"
  generation: 1
  name: example
  namespace: etcd-test
  resourceVersion: "522"
  uid: 9d07b710-0710-43b9-9e2a-27811c62d059
spec:
  etcdOptions:
  - --experimental-peer-skip-client-san-verification
  size: 1
  storageSpec:
    volumeSizeRequest: 100M
  version: v3.5.17

❯ k describe sts -n etcd-test example
Name:               example
Namespace:          etcd-test
CreationTimestamp:  Tue, 12 Aug 2025 14:54:17 +0530
Selector:           app=example,controller=example
Labels:             <none>
Annotations:        <none>
Replicas:           1 desired | 1 total
Update Strategy:    RollingUpdate
  Partition:        0
Pods Status:        1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=example
           controller=example
  Containers:
   etcd:
    Image:       gcr.io/etcd-development/etcd:v3.5.17
    Ports:       2379/TCP, 2380/TCP
    Host Ports:  0/TCP, 0/TCP
    Command:
      /usr/local/bin/etcd
    Args:
      --name=$(POD_NAME)
      --listen-peer-urls=http://0.0.0.0:2380
      --listen-client-urls=http://0.0.0.0:2379
      --initial-advertise-peer-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2380
      --advertise-client-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2379
      --experimental-peer-skip-client-san-verification
    Environment Variables from:
      example-state  ConfigMap  Optional: false
    Environment:
      POD_NAME:        (v1:metadata.name)
      POD_NAMESPACE:   (v1:metadata.namespace)
    Mounts:
      /var/lib/etcd from etcd-data (rw)
  Volumes:         <none>
  Node-Selectors:  <none>
  Tolerations:     <none>
Volume Claims:
  Name:          etcd-data
  StorageClass:
  Labels:        <none>
  Annotations:   <none>
  Capacity:      100M
  Access Modes:  [ReadWriteOnce]
Events:
  Type    Reason            Age   From                    Message
  ----    ------            ----  ----                    -------
  Normal  SuccessfulCreate  7m    statefulset-controller  create Claim etcd-data-example-0 Pod example-0 in StatefulSet example success
  Normal  SuccessfulCreate  7m    statefulset-controller  create Pod example-0 in StatefulSet example successful

❯ k describe pvc -n etcd-test etcd-data-example-0
Name:          etcd-data-example-0
Namespace:     etcd-test
StorageClass:  local-path
Status:        Bound
Volume:        pvc-46dede60-962d-4eb3-a3c0-bf83e6a1ac16
Labels:        app=example
               controller=example
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: rancher.io/local-path
               volume.kubernetes.io/selected-node: colima
               volume.kubernetes.io/storage-provisioner: rancher.io/local-path
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      100M
Access Modes:  RWO
VolumeMode:    Filesystem
Used By:       example-0
Events:
  Type    Reason                 Age                    From                                                                                                Message
  ----    ------                 ----                   ----                                                                                                -------
  Normal  WaitForFirstConsumer   7m24s                  persistentvolume-controller                                                                         waiting for first consumer to be created before binding
  Normal  Provisioning           7m24s                  rancher.io/local-path_local-path-provisioner-5cf85fd84d-7lmmb_302b08f7-6662-4012-ab2b-9b109a4d37c7  External provisioner is provisioning volume for claim "etcd-test/etcd-data-example-0"
  Normal  ExternalProvisioning   7m23s (x2 over 7m24s)  persistentvolume-controller                                                                         Waiting for a volume to be created either by the external provisioner 'rancher.io/local-path' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.
  Normal  ProvisioningSucceeded  7m21s                  rancher.io/local-path_local-path-provisioner-5cf85fd84d-7lmmb_302b08f7-6662-4012-ab2b-9b109a4d37c7  Successfully provisioned volume pvc-46dede60-962d-4eb3-a3c0-bf83e6a1ac16

[ahrtr]

We need to double confirm whether it can be reproduced, and close this one if not reproducible.

[Prakhar824]

@SDBrett, I used the config you provided and was able to create the etcd cluster without any issues.

$ kubectl describe etcdclusters -n etcd-test example 
Name:         example
Namespace:    etcd-test
Labels:       <none>
Annotations:  <none>
API Version:  operator.etcd.io/v1alpha1
Kind:         EtcdCluster
Metadata:
  Creation Timestamp:  2025-11-01T12:57:27Z
  Generation:          1
  Resource Version:    6242
  UID:                 0768aa91-c33b-4cb0-a57b-dbb9a7216f3b
Spec:
  Etcd Options:
    --experimental-peer-skip-client-san-verification
  Size:  1
  Storage Spec:
    Volume Size Request:  100M
  Version:                v3.5.17
Events:                   <none>

$ kubectl describe sts -n etcd-test example  
Name:               example
Namespace:          etcd-test
CreationTimestamp:  Sat, 01 Nov 2025 18:27:27 +0530
Selector:           app=example,controller=example
Labels:             <none>
Annotations:        <none>
Replicas:           1 desired | 1 total
Update Strategy:    RollingUpdate
  Partition:        0
Pods Status:        1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=example
           controller=example
  Containers:
   etcd:
    Image:       gcr.io/etcd-development/etcd:v3.5.17
    Ports:       2379/TCP, 2380/TCP
    Host Ports:  0/TCP, 0/TCP
    Command:
      /usr/local/bin/etcd
    Args:
      --name=$(POD_NAME)
      --listen-peer-urls=http://0.0.0.0:2380
      --listen-client-urls=http://0.0.0.0:2379
      --initial-advertise-peer-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2380
      --advertise-client-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2379
      --experimental-peer-skip-client-san-verification
    Environment Variables from:
      example-state  ConfigMap  Optional: false
    Environment:
      POD_NAME:        (v1:metadata.name)
      POD_NAMESPACE:   (v1:metadata.namespace)
    Mounts:
      /var/lib/etcd from etcd-data (rw)
  Volumes:  <none>
Volume Claims:
  Name:          etcd-data
  StorageClass:  
  Labels:        <none>
  Annotations:   <none>
  Capacity:      100M
  Access Modes:  [ReadWriteOnce]
Events:
  Type    Reason            Age   From                    Message
  ----    ------            ----  ----                    -------
  Normal  SuccessfulCreate  13m   statefulset-controller  create Claim etcd-data-example-0 Pod example-0 in StatefulSet example success
  Normal  SuccessfulCreate  13m   statefulset-controller  create Pod example-0 in StatefulSet example successful

$ kubectl describe pod -n etcd-test example-0 
Name:             example-0
Namespace:        etcd-test
Priority:         0
Service Account:  default
Node:             kind-control-plane/172.18.0.2
Start Time:       Sat, 01 Nov 2025 18:27:31 +0530
Labels:           app=example
                  apps.kubernetes.io/pod-index=0
                  controller=example
                  controller-revision-hash=example-674bdccfd8
                  statefulset.kubernetes.io/pod-name=example-0
Annotations:      <none>
Status:           Running
IP:               10.244.0.13
IPs:
  IP:           10.244.0.13
Controlled By:  StatefulSet/example
Containers:
  etcd:
    Container ID:  containerd://fce8ba345b89b8753a548f7cae7b90069f380679039b8bc41e35445478f2a0c7
    Image:         gcr.io/etcd-development/etcd:v3.5.17
    Image ID:      gcr.io/etcd-development/etcd@sha256:15b1882f07aeaf357a48fbbcf35a566ae3ea110395704711488ea263fa86838d
    Ports:         2379/TCP, 2380/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /usr/local/bin/etcd
    Args:
      --name=$(POD_NAME)
      --listen-peer-urls=http://0.0.0.0:2380
      --listen-client-urls=http://0.0.0.0:2379
      --initial-advertise-peer-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2380
      --advertise-client-urls=http://$(POD_NAME).example.$(POD_NAMESPACE).svc.cluster.local:2379
      --experimental-peer-skip-client-san-verification
    State:          Running
      Started:      Sat, 01 Nov 2025 18:27:43 +0530
    Ready:          True
    Restart Count:  0
    Environment Variables from:
      example-state  ConfigMap  Optional: false
    Environment:
      POD_NAME:       example-0 (v1:metadata.name)
      POD_NAMESPACE:  etcd-test (v1:metadata.namespace)
    Mounts:
      /var/lib/etcd from etcd-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wrxx7 (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  etcd-data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  etcd-data-example-0
    ReadOnly:   false
  kube-api-access-wrxx7:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  14m   default-scheduler  Successfully assigned etcd-test/example-0 to kind-control-plane
  Normal  Pulling    14m   kubelet            Pulling image "gcr.io/etcd-development/etcd:v3.5.17"
  Normal  Pulled     14m   kubelet            Successfully pulled image "gcr.io/etcd-development/etcd:v3.5.17" in 11.382s (11.382s including waiting). Image size: 20249236 bytes.
  Normal  Created    14m   kubelet            Created container: etcd
  Normal  Started    14m   kubelet            Started container etcd

[mehrdadbn9]

@ahrtr to answer the "can it still be reproduced" question: it can't, and there's a specific reason for that.

The report is genuine, but it was fixed on the same day it was filed, by 3905389 ("Defaults to ReadWriteOnce when it's empty or not provided"). Before that commit the claim template copied the field straight out of the spec:

AccessModes: []corev1.PersistentVolumeAccessMode{ec.Spec.StorageSpec.AccessModes},

so an unset accessModes reached the API server as accessModes: [""], which is exactly the accessModes: - '' in the StatefulSet @SDBrett pasted, and why the PVC was rejected and the pods stayed pending. The commit replaced that with a hardcoded corev1.ReadWriteOnce for the ReadWriteOnce, "" branch, so anyone testing after 2025-03-22 gets a working cluster - which matches what @shubhindia and @Prakhar824 saw.

I confirmed this rather than assuming it: reverting that one line locally and creating a cluster with the config from the report reproduces accessModes: [""], and putting it back makes it ReadWriteOnce again.

The one thing left over is that nothing guards the defaulting. The e2e cases always set accessModes: ReadWriteOnce explicitly, so the empty path isn't covered anywhere and the fix can be dropped again silently. I opened #452 with a table-driven unit test over createOrPatchStatefulSet covering unset, explicit ReadWriteOnce, ReadWriteMany with and without pvcName, and an unsupported mode. Reverting 3905389 turns exactly the unset case red.

So this issue can be closed as already fixed - #452 just keeps it from coming back.

One optional follow-up, happy to do it separately if you think it's worth it: StorageSpec.AccessModes currently has no kubebuilder markers, so the CRD schema is a bare type: string. A bad value is accepted at admission and only fails later in reconcile with AccessMode %s is not supported, which is only visible in the operator log. Adding +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany (plus possibly +kubebuilder:default=ReadWriteOnce) would reject it at apply time instead. I left it out of #452 since it changes the published API and that's your call, not a test change.