Skip to content

Add support for reading Helm API versions from files #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 7, 2024
Merged

Add support for reading Helm API versions from files #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1f28115
Add support for reading API versions from foot comments
abstrask Oct 3, 2024
d3c32fc
Add Argo Workflows test manifests
abstrask Oct 3, 2024
ee2206f
Not the purpose of the `argo-workflows` test files
abstrask Oct 4, 2024
4dd4f33
Readme: Add TOC, document how to specify API versions
abstrask Oct 4, 2024
67f75e6
Readme: Add remainder of sentence
abstrask Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 51 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
<!-- omit in toc -->
# Flux Helm chart diff action

A composite GitHub Action for use with PR workflows in repos with [Flux Helm manifests](https://fluxcd.io/flux/use-cases/helm/).
Expand All @@ -14,6 +15,17 @@ Combine with these awesome projects for maximum workflow smoothness:
- [mshick/add-pr-comment](https://github.com/mshick/add-pr-comment): Add diff report as comment to PR
- [Renovate](https://github.com/renovatebot/renovate): Automatically create PRs when new charts versions are available)

<!-- omit in toc -->
## Table of Contents

- [Dependencies](#dependencies)
- [Inputs](#inputs)
- [Outputs](#outputs)
- [Usage](#usage)
- [Dry-running/Emulating API Capabilities](#dry-runningemulating-api-capabilities)
- [Example Output/PR comment](#example-outputpr-comment)
- [Testing](#testing)

## Dependencies

Requires [Helm](https://helm.sh/) and [yq](https://mikefarah.gitbook.io/yq). Both can be installed using [Arkade](https://github.com/alexellis/arkade-get) if needed.
Expand All @@ -35,6 +47,7 @@ In `diff_markdown` the output for each file will either be:

## Usage

<!-- omit in toc -->
### TL;DR

```yaml
Expand All @@ -45,6 +58,7 @@ In `diff_markdown` the output for each file will either be:
helm_files: ${{ steps.changed_files_helm.outputs.all_changed_files }}
```

<!-- omit in toc -->
### Full Example

Run on pull requests:
Expand Down Expand Up @@ -174,13 +188,40 @@ Optionally, cause check to fail, if any Helm file failed to render:

See [example-workflow.yaml](example-workflow.yaml) for coherent example.


## Dry-running/Emulating API Capabilities

When installing, Helm can access the available Kubernetes APIs and versions, through "[Built-in Objects](https://helm.sh/docs/chart_template_guide/builtin_objects/)".

This enable charts to deploy custom resources, or tweak properties as needed, based on the APIs offered in the cluster. For example, starting with `argo-workflows` chart 0.41.0, the `ServiceMonitor` resource doesn't even get deployed, if [`.Capabilities.APIVersions.Has`](https://github.com/argoproj/argo-helm/blob/argo-workflows-0.41.0/charts/argo-workflows/templates/controller/workflow-controller-servicemonitor.yaml#L2) doesn't contain [`monitoring.coreos.com/v1`](https://github.com/argoproj/argo-helm/blob/argo-workflows-0.41.0/charts/argo-workflows/templates/_helpers.tpl#L200).

This does however also make it difficult to dry-run (using the `helm template` command), with no cluster access. As a workaround, it's possible to specify API version to be used when running the `template` command as commented YAML. The comments has to be the last in the file and must have the document start `---` above. Example:

```yaml
---
# helm-api-versions:
# - myapi/v0
# - monitoring.coreos.com/v1
```

You can verify that the APIs are read correctly from the log output of the "Helm diff" step of the action:

```
Processing file "infrastructure/base/argo-workflows/helm.yaml"
(...)
head API versions: myapi/v0,monitoring.coreos.com/v1
(...)
```

## Example Output/PR comment

<!-- omit in toc -->
### infrastructure/base/dcgm-exporter/helm.yaml
```diff
No changes
```

<!-- omit in toc -->
### infrastructure/base/nvidia-device-plugin/helm.yaml
```diff
(abbreviated)
Expand Down Expand Up @@ -258,13 +299,15 @@ helm_files=($(find ./test/head -type f -name 'helm.yaml' | sed "s|^./test/head/|
GITHUB_OUTPUT=debug.out HELM_FILES="${helm_files[@]}" TEST=1 ./flux-helm-diff.sh; cat debug.out
```

<!-- omit in toc -->
### Testing files

| Name | Scenario tested | Expected output |
| ----------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------- |
| `dcgm-exporter` | Chart added in `head` that doesn't exist in `base` | Diff shows entire rendered template as added |
| `metaflow` | Very non-standard way of publishing charts (not sure if should be supported) | TBD |
| `nvidia-device-plugin` | HelmRepository (using `https`), minor chart version bump | Diff (with potentially breaking `nodeAffinity`) |
| `weave-gitops-helm2oci` | Repository type changed from HelmRepository (type `oci`) to OCIRepository | No changes |
| `weave-gitops-helmrepo` | HelmRepository with type `oci` | Diff |
| `weave-gitops-ocirepo` | OCIRepository | Diff |
| Name | Scenario tested | Expected output |
| ----------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| `argo-workflows` | Read API from comment in helm file (otherwise `ServiceMonitor` resource is not rendered) | Diff shows change to `ServiceMonitor`, instead of being removed |
| `dcgm-exporter` | Chart added in `head` that doesn't exist in `base` | Diff shows entire rendered template as added |
| `metaflow` | Very non-standard way of publishing charts (not sure if should be supported) | TBD |
| `nvidia-device-plugin` | HelmRepository (using `https`), minor chart version bump | Diff (with potentially breaking `nodeAffinity`) |
| `weave-gitops-helm2oci` | Repository type changed from HelmRepository (type `oci`) to OCIRepository | No changes |
| `weave-gitops-helmrepo` | HelmRepository with type `oci` | Diff |
| `weave-gitops-ocirepo` | OCIRepository | Diff |
8 changes: 6 additions & 2 deletions flux-helm-diff.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -eu -o pipefail

helm_files=(${HELM_FILES[@]})
Expand Down Expand Up @@ -61,6 +61,9 @@ helm_template() {
namespace=$(yq '. | select(.kind == "HelmRelease").metadata.namespace' "${helm_file}")
values=$(yq '. | select(.kind == "HelmRelease").spec.values' "${helm_file}")

# Use Capabilities.APIVersions
mapfile -t api_versions < <(yq '. | foot_comment' "${helm_file}" | yq '.helm-api-versions[]')

# Let's see what information we got out about the chart...
echo "${ref} repo type: ${repo_type}" >&2
echo "${ref} repo name: ${repo_name}" >&2
Expand All @@ -69,6 +72,7 @@ helm_template() {
echo "${ref} chart version: ${version}" >&2
echo "${ref} release name: ${name}" >&2
echo "${ref} release namespace: ${namespace}" >&2
echo "${ref} API versions: $(IFS=,; echo "${api_versions[*]}")" >&2

# Syntax for chart repos is different from OCI repos (as HelmRepo kind)
if [[ "${url}" = "oci://"* ]]; then
Expand All @@ -78,7 +82,7 @@ helm_template() {
fi

# Render template
template_out=$(helm template "${name}" ${chart_args[@]} --version "${version}" -n "${namespace}" -f <(echo "${values}") 2>&1) || {
template_out=$(helm template "${name}" ${chart_args[@]} --version "${version}" -n "${namespace}" -f <(echo "${values}") --api-versions "$(IFS=,; echo "${api_versions[*]}")" 2>&1) || {
echo "$template_out"
echo "$template_out" >&2
return 2
Expand Down
90 changes: 90 additions & 0 deletions test/base/infrastructure/base/argo-workflows/helm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: argo
namespace: argo
spec:
interval: 15m
url: https://argoproj.github.io/argo-helm

---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: argo-workflows
namespace: argo
spec:
interval: 5m
targetNamespace: argo
chart:
spec:
chart: argo-workflows
version: "0.40.14"
sourceRef:
kind: HelmRepository
name: argo
interval: 15m
install:
skipCRDs: true
values:
crds:
install: false
useStaticCredentials: false
controller:
parallelism: 1500 # this affects apps/processing/base/synchronization.yaml
metricsConfig:
enabled: true
workflowDefaults:
spec:
podGC: # https://argoproj.github.io/argo-workflows/fields/#podgc
strategy: OnPodCompletion
ttlStrategy: # https://argoproj.github.io/argo-workflows/fields/#ttlstrategy
secondsAfterSuccess: 300 # 5 minutes
secondsAfterFailure: 86400 # 24 hours
secondsAfterCompletion: 86400 # 24 hours
priorityClassName: high-priority
resources:
requests:
cpu: 100m
memory: 10Gi
limits:
memory: 10Gi
extraArgs:
- --qps=200
- --burst=300
workflowWorkers: 64
workflowTTLWorkers: 16
podCleanupWorkers: 16
serviceMonitor:
enabled: true
additionalLabels:
instance: primary
server:
extraArgs:
- --auth-mode=server
- --kube-api-qps=120.0
- --kube-api-burst=180
priorityClassName: high-priority
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
memory: 1Gi
ingress:
enabled: true
annotations:
cert-manager.io/cluster-issuer: letsencrypt
ingressClassName: nginx
hosts:
- ${flux_argo_instance}wf.${flux_base_domain_name}
pathType: Prefix
paths:
- /
tls:
- secretName: argo-tls
hosts:
- ${flux_argo_instance}wf.${flux_base_domain_name}
serviceAccount:
name: argo-workflows
96 changes: 96 additions & 0 deletions test/head/infrastructure/base/argo-workflows/helm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: argo
namespace: argo
spec:
interval: 15m
url: https://argoproj.github.io/argo-helm

---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: argo-workflows
namespace: argo
spec:
interval: 5m
targetNamespace: argo
chart:
spec:
chart: argo-workflows
version: "0.42.5"
sourceRef:
kind: HelmRepository
name: argo
interval: 15m
install:
skipCRDs: true
values:
crds:
install: false
useStaticCredentials: false
controller:
parallelism: 1500 # this affects apps/processing/base/synchronization.yaml
metricsConfig:
enabled: true
honorLabels: true
workflowDefaults:
spec:
podGC: # https://argoproj.github.io/argo-workflows/fields/#podgc
strategy: OnPodCompletion
ttlStrategy: # https://argoproj.github.io/argo-workflows/fields/#ttlstrategy
secondsAfterSuccess: 300 # 5 minutes
secondsAfterFailure: 86400 # 24 hours
secondsAfterCompletion: 86400 # 24 hours
priorityClassName: high-priority
resources:
requests:
cpu: 100m
memory: 10Gi
limits:
memory: 10Gi
extraArgs:
- --qps=200
- --burst=300
workflowWorkers: 64
workflowTTLWorkers: 16
podCleanupWorkers: 16
serviceMonitor:
enabled: true
additionalLabels:
instance: primary
server:
extraArgs:
- --auth-mode=server
- --kube-api-qps=120.0
- --kube-api-burst=180
priorityClassName: high-priority
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
memory: 1Gi
ingress:
enabled: true
annotations:
cert-manager.io/cluster-issuer: letsencrypt
ingressClassName: nginx
hosts:
- ${flux_argo_instance}wf.${flux_base_domain_name}
pathType: Prefix
paths:
- /
tls:
- secretName: argo-tls
hosts:
- ${flux_argo_instance}wf.${flux_base_domain_name}
serviceAccount:
name: argo-workflows

---
# helm-api-versions:
# - myapi/v0
# - monitoring.coreos.com/v1