Does HelmRelease support force recreate resource?

[RammusXu]

When my helm chart deployment labels are changed, HelmRelease is broken.
And this does not work.

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: rammus-vector
  namespace: logging-nonprod
spec:
  ...
  upgrade:
    force: true

I also find this comment for Kustomize, so, I'm wondering if HelmRelease support recreate resource which is caused by field is immutable.

Full error message:

message: "Helm upgrade failed: cannot patch \"rammus-vector\" with kind Deployment: Deployment.apps \"rammus-vector\" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"app.kubernetes.io/instance\":\"rammus-vector\", \"app.kubernetes.io/managed-by\":\"Helm\", \"app.kubernetes.io/name\":\"vector\", \"app.kubernetes.io/version\":\"0.21.2\", \"helm.sh/chart\":\"vector-3.0.0\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable\n\nLast Helm logs:\n\nPatch Deployment \"rammus-vector\" in namespace logging-nonprod\nerror updating the resource \"rammus-vector\":\n\t cannot patch \"rammus-vector\" with kind Deployment: Deployment.apps \"rammus-vector\" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"app.kubernetes.io/instance\":\"rammus-vector\", \"app.kubernetes.io/managed-by\":\"Helm\", \"app.kubernetes.io/name\":\"vector\", \"app.kubernetes.io/version\":\"0.21.2\", \"helm.sh/chart\":\"vector-3.0.0\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable\nPatch HorizontalPodAutoscaler \"rammus-vector\" in namespace logging-nonprod\nLooks like there are no changes for CronJob \"rammus-vector-ead-cronjob\"\nwarning: Upgrade \"rammus-vector\" failed: cannot patch \"rammus-vector\" with kind Deployment: Deployment.apps \"rammus-vector\" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"app.kubernetes.io/instance\":\"rammus-vector\", \"app.kubernetes.io/managed-by\":\"Helm\", \"app.kubernetes.io/name\":\"vector\", \"app.kubernetes.io/version\":\"0.21.2\", \"helm.sh/chart\":\"vector-3.0.0\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable"

[masterkain]

depending on the release and what you might lose I'd just remove the deployment, push, waiting for the resources to clean up and activate the release again

this is common also for example when helm charts upgrade themselves or their dependencies to an incompatible version that requires redeployments

[kingdonb]

spec.upgrade.force should use the Replace strategy according to this doc:

	// Force forces resource updates through a replacement strategy.
	// +optional
	Force bool `json:"force,omitempty"`

Were you able to get this working?

It does, which means a different thing than delete and recreate – replace is patch, which still cannot update immutable fields. (differs from kubectl replace --force which actually does a delete and recreate when needed)

[kingdonb]

I'm still deleting resources manually, but we have flux reconcile helmrelease --force now - in case you haven't enabled drift detection, don't want to wait for the next interval to arrive, nor are able to push a trivial/no-op change to your helm values, this one is new since Flux 2.3 (I think? Which one made Helm Controller GA)

I still don't think Helm Controller should be deleting and recreating resources in any case, but if you have a specific use case that isn't served well, I'd be happy to talk about it with you in a more interactive setting (join us on the #flux channel in the CNCF slack!)

[Nicola-Sergio]

@kingdonb In case I was managing a statefulset via FluxCD to comply with the GitOps approach, and I modified non-modifiable parameters, what would be the reasoning behind not allowing to destroy that resource and recreate it to avoid the update error? For visibility of what happens or something else? I ask to better understand the issue

[stefanprodan]

Flux helm-controller is limited to Helm SDK features, if the Helm CLI does not recreate statefulsets when run with helm upgrade --force neither will Flux.

[mnaser]

That's fair enough, I get the idea of sticking to what the SDK provides in terms of features. It's unfortunate that upstream seems to see it as "okay" to just "delete" resources manually, so I think that's something we'll have to push to fix in Helm if that's the case.

[Nicola-Sergio]

So what I would do when I have to modify unmodifiable parameter of whatever kind of deploy?
I usually do this:

• Suspend ks flux-system
• Delete manually the helm release
• Push changes to the repo
• Ensure that the resource is completely deleted
• Resume ks and reconcile it with -- with-source flag

Is it the correct approach?