Skip to content

Commit 2efb83b

Browse files
sam-mcbrsam-mcbr
committed
Add ability to configure StatefulSet PVC retention policy
Signed-off-by: Sam McBroom <[email protected]>
1 parent 7caefcc commit 2efb83b

File tree

7 files changed

+57
-0
lines changed

7 files changed

+57
-0
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
* [CHANGE] Updated default `align_queries_with_step` to **false** to match documentation #513
66
* [ENHANCEMENT] Add `nginx.config.upstream_protocol` field to configure the upstream protocol in the nginx configuration #506
7+
* [ENHANCEMENT] Add ability to set `persistentVolumeClaimRetentionPolicy` on alertmanager, ingester, compactor, and store-gateway StatefulSets
78
* [BUGFIX] fix: upstream_protocol reference in auth_orgs #509
89
* [DEPENDENCY] Update quay.io/cortexproject/cortex Docker tag to v1.18.1 #510
910

‎README.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,10 +112,13 @@ Kubernetes: `^1.19.0-0`
112112
| alertmanager.&ZeroWidthSpace;nodeSelector | object | `{}` | |
113113
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;accessModes | list | `["ReadWriteOnce"]` | Alertmanager data Persistent Volume access modes Must match those of existing PV or dynamic provisioner Ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ |
114114
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;annotations | object | `{}` | Alertmanager data Persistent Volume Claim annotations |
115+
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enableRetentionPolicy | bool | `false` | Enable StatefulSetAutoDeletePVC feature https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention |
115116
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enabled | bool | `true` | If true and alertmanager.statefulSet.enabled is true, Alertmanager will create/use a Persistent Volume Claim If false, use emptyDir |
116117
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;size | string | `"2Gi"` | Alertmanager data Persistent Volume size |
117118
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;storageClass | string | `nil` | Alertmanager data Persistent Volume Storage Class If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. |
118119
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;subPath | string | `""` | Subdirectory of Alertmanager data Persistent Volume to mount Useful if the volume's root directory is not empty |
120+
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenDeleted | string | `"Retain"` | |
121+
| alertmanager.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenScaled | string | `"Retain"` | |
119122
| alertmanager.&ZeroWidthSpace;podAnnotations | object | `{"prometheus.io/port":"8080","prometheus.io/scrape":"true"}` | Pod Annotations |
120123
| alertmanager.&ZeroWidthSpace;podDisruptionBudget | object | `{"maxUnavailable":1}` | If not set then a PodDisruptionBudget will not be created |
121124
| alertmanager.&ZeroWidthSpace;podLabels | object | `{}` | Pod Labels |
@@ -183,10 +186,13 @@ Kubernetes: `^1.19.0-0`
183186
| compactor.&ZeroWidthSpace;nodeSelector | object | `{}` | |
184187
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;accessModes | list | `["ReadWriteOnce"]` | compactor data Persistent Volume access modes Must match those of existing PV or dynamic provisioner Ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ |
185188
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;annotations | object | `{}` | compactor data Persistent Volume Claim annotations |
189+
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enableRetentionPolicy | bool | `false` | Enable StatefulSetAutoDeletePVC feature https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention |
186190
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enabled | bool | `true` | If true compactor will create/use a Persistent Volume Claim If false, use emptyDir |
187191
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;size | string | `"2Gi"` | |
188192
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;storageClass | string | `nil` | compactor data Persistent Volume Storage Class If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. |
189193
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;subPath | string | `""` | Subdirectory of compactor data Persistent Volume to mount Useful if the volume's root directory is not empty |
194+
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenDeleted | string | `"Retain"` | |
195+
| compactor.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenScaled | string | `"Retain"` | |
190196
| compactor.&ZeroWidthSpace;podAnnotations | object | `{"prometheus.io/port":"8080","prometheus.io/scrape":"true"}` | Pod Annotations |
191197
| compactor.&ZeroWidthSpace;podDisruptionBudget.&ZeroWidthSpace;maxUnavailable | int | `1` | |
192198
| compactor.&ZeroWidthSpace;podLabels | object | `{}` | Pod Labels |
@@ -345,10 +351,13 @@ Kubernetes: `^1.19.0-0`
345351
| ingester.&ZeroWidthSpace;nodeSelector | object | `{}` | |
346352
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;accessModes | list | `["ReadWriteOnce"]` | Ingester data Persistent Volume access modes Must match those of existing PV or dynamic provisioner Ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ |
347353
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;annotations | object | `{}` | Ingester data Persistent Volume Claim annotations |
354+
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enableRetentionPolicy | bool | `false` | Enable StatefulSetAutoDeletePVC feature https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention |
348355
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enabled | bool | `true` | If true and ingester.statefulSet.enabled is true, Ingester will create/use a Persistent Volume Claim If false, use emptyDir |
349356
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;size | string | `"2Gi"` | Ingester data Persistent Volume size |
350357
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;storageClass | string | `nil` | Ingester data Persistent Volume Storage Class If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. |
351358
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;subPath | string | `""` | Subdirectory of Ingester data Persistent Volume to mount Useful if the volume's root directory is not empty |
359+
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenDeleted | string | `"Retain"` | |
360+
| ingester.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenScaled | string | `"Retain"` | |
352361
| ingester.&ZeroWidthSpace;podAnnotations | object | `{"prometheus.io/port":"8080","prometheus.io/scrape":"true"}` | Pod Annotations |
353362
| ingester.&ZeroWidthSpace;podDisruptionBudget.&ZeroWidthSpace;maxUnavailable | int | `1` | |
354363
| ingester.&ZeroWidthSpace;podLabels | object | `{}` | Pod Labels |
@@ -814,10 +823,13 @@ Kubernetes: `^1.19.0-0`
814823
| store_gateway.&ZeroWidthSpace;nodeSelector | object | `{}` | |
815824
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;accessModes | list | `["ReadWriteOnce"]` | Store-gateway data Persistent Volume access modes Must match those of existing PV or dynamic provisioner Ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ |
816825
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;annotations | object | `{}` | Store-gateway data Persistent Volume Claim annotations |
826+
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enableRetentionPolicy | bool | `false` | Enable StatefulSetAutoDeletePVC feature https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention |
817827
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;enabled | bool | `true` | If true Store-gateway will create/use a Persistent Volume Claim If false, use emptyDir |
818828
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;size | string | `"2Gi"` | Store-gateway data Persistent Volume size |
819829
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;storageClass | string | `nil` | Store-gateway data Persistent Volume Storage Class If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. |
820830
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;subPath | string | `""` | Subdirectory of Store-gateway data Persistent Volume to mount Useful if the volume's root directory is not empty |
831+
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenDeleted | string | `"Retain"` | |
832+
| store_gateway.&ZeroWidthSpace;persistentVolume.&ZeroWidthSpace;whenScaled | string | `"Retain"` | |
821833
| store_gateway.&ZeroWidthSpace;podAnnotations | object | `{"prometheus.io/port":"8080","prometheus.io/scrape":"true"}` | Pod Annotations |
822834
| store_gateway.&ZeroWidthSpace;podDisruptionBudget.&ZeroWidthSpace;maxUnavailable | int | `1` | |
823835
| store_gateway.&ZeroWidthSpace;podLabels | object | `{}` | Pod Labels |

‎templates/alertmanager/alertmanager-statefulset.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,11 @@ spec:
2121
{{- toYaml .Values.alertmanager.statefulStrategy | nindent 4 }}
2222
serviceName: {{ template "cortex.fullname" . }}-alertmanager-headless
2323
{{- if .Values.alertmanager.persistentVolume.enabled }}
24+
{{- if and (semverCompare ">=1.23-0" .Capabilities.KubeVersion.Version) (.Values.alertmanager.persistentVolume.enableRetentionPolicy) }}
25+
persistentVolumeClaimRetentionPolicy:
26+
whenDeleted: {{ .Values.alertmanager.persistentVolume.whenDeleted }}
27+
whenScaled: {{ .Values.alertmanager.persistentVolume.whenScaled }}
28+
{{- end }}
2429
volumeClaimTemplates:
2530
- metadata:
2631
name: storage

‎templates/compactor/compactor-statefulset.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ spec:
1818
{{- toYaml .Values.compactor.strategy | nindent 4 }}
1919
serviceName: {{ template "cortex.fullname" . }}-compactor
2020
{{- if .Values.compactor.persistentVolume.enabled }}
21+
{{- if and (semverCompare ">=1.23-0" .Capabilities.KubeVersion.Version) (.Values.compactor.persistentVolume.enableRetentionPolicy) }}
22+
persistentVolumeClaimRetentionPolicy:
23+
whenDeleted: {{ .Values.compactor.persistentVolume.whenDeleted }}
24+
whenScaled: {{ .Values.compactor.persistentVolume.whenScaled }}
25+
{{- end }}
2126
volumeClaimTemplates:
2227
- metadata:
2328
name: storage

‎templates/ingester/ingester-statefulset.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,11 @@ spec:
2121
podManagementPolicy: "{{ .Values.ingester.statefulSet.podManagementPolicy }}"
2222
serviceName: {{ template "cortex.fullname" . }}-ingester-headless
2323
{{- if .Values.ingester.persistentVolume.enabled }}
24+
{{- if and (semverCompare ">=1.23-0" .Capabilities.KubeVersion.Version) (.Values.ingester.persistentVolume.enableRetentionPolicy) }}
25+
persistentVolumeClaimRetentionPolicy:
26+
whenDeleted: {{ .Values.ingester.persistentVolume.whenDeleted }}
27+
whenScaled: {{ .Values.ingester.persistentVolume.whenScaled }}
28+
{{- end }}
2429
volumeClaimTemplates:
2530
- metadata:
2631
name: storage

‎templates/store-gateway/store-gateway-statefulset.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,11 @@ spec:
2121
podManagementPolicy: {{ .Values.store_gateway.podManagementPolicy | quote }}
2222
serviceName: {{ template "cortex.fullname" . }}-store-gateway-headless
2323
{{- if .Values.store_gateway.persistentVolume.enabled }}
24+
{{- if and (semverCompare ">=1.23-0" .Capabilities.KubeVersion.Version) (.Values.store_gateway.persistentVolume.enableRetentionPolicy) }}
25+
persistentVolumeClaimRetentionPolicy:
26+
whenDeleted: {{ .Values.store_gateway.persistentVolume.whenDeleted }}
27+
whenScaled: {{ .Values.store_gateway.persistentVolume.whenScaled }}
28+
{{- end }}
2429
volumeClaimTemplates:
2530
- metadata:
2631
name: storage

‎values.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,12 @@ alertmanager:
217217
# set, choosing the default provisioner.
218218
storageClass: null
219219

220+
# -- Enable StatefulSetAutoDeletePVC feature
221+
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
222+
enableRetentionPolicy: false
223+
whenDeleted: Retain
224+
whenScaled: Retain
225+
220226
startupProbe:
221227
httpGet:
222228
path: /ready
@@ -543,6 +549,12 @@ ingester:
543549
# set, choosing the default provisioner.
544550
storageClass: null
545551

552+
# -- Enable StatefulSetAutoDeletePVC feature
553+
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
554+
enableRetentionPolicy: false
555+
whenDeleted: Retain
556+
whenScaled: Retain
557+
546558
# -- Startup/liveness probes for ingesters are not recommended.
547559
# Ref: https://cortexmetrics.io/docs/guides/running-cortex-on-kubernetes/#take-extra-care-with-ingesters
548560
startupProbe: {}
@@ -1404,6 +1416,12 @@ store_gateway:
14041416
# set, choosing the default provisioner.
14051417
storageClass: null
14061418

1419+
# -- Enable StatefulSetAutoDeletePVC feature
1420+
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
1421+
enableRetentionPolicy: false
1422+
whenDeleted: Retain
1423+
whenScaled: Retain
1424+
14071425
startupProbe:
14081426
failureThreshold: 60
14091427
initialDelaySeconds: 120
@@ -1520,6 +1538,12 @@ compactor:
15201538
# set, choosing the default provisioner.
15211539
storageClass: null
15221540

1541+
# -- Enable StatefulSetAutoDeletePVC feature
1542+
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
1543+
enableRetentionPolicy: false
1544+
whenDeleted: Retain
1545+
whenScaled: Retain
1546+
15231547
startupProbe:
15241548
failureThreshold: 60
15251549
initialDelaySeconds: 120

0 commit comments

Comments
 (0)