Skip to content

Commit a27b78c

Browse files
authored
Add support for "OCIRepository" source kind, various improvements (#2)
* Update test helm files * More changes to test manifests * helm_files from env not arg, also handle OCIRepository * Update docs * Fix unbound 'TEST' variable error * First attempt at adding test * Minor example correction * Fix test action path snafu * Test action works on all Helm files, not just those changed * Test action should run script directly, not action from branch * Fix filter in test action * Remove filter altogether for test action, because it doesn't make sense * Fix glob version snafu in test action * Fix checkout version, less specific glob version * Set path in test action * Fix test action * Remove todo from readme
1 parent 75051ef commit a27b78c

File tree

14 files changed

+464
-97
lines changed

14 files changed

+464
-97
lines changed

.github/workflows/test.yaml

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# https://github.com/recoord/flux-infrastructure/blob/dev/.github/workflows/update-crds.yml
2+
# https://github.com/recoord/core-devops-engine-k8s-manifests/blob/main/.github/workflows/diff.yml
3+
4+
name: Test Helm diff
5+
on:
6+
- pull_request_target
7+
jobs:
8+
helm_diff:
9+
name: Diff changed Helm templates
10+
runs-on: ubuntu-latest
11+
permissions:
12+
contents: read
13+
pull-requests: write
14+
15+
steps:
16+
- id: dependencies
17+
name: Install dependencies
18+
uses: alexellis/arkade-get@master
19+
with:
20+
helm: latest
21+
yq: latest
22+
23+
- id: checkout_base
24+
uses: actions/checkout@v4
25+
with:
26+
ref: ${{ github.base_ref }}
27+
path: base
28+
29+
- id: checkout_head
30+
uses: actions/checkout@v4
31+
with:
32+
ref: ${{ github.head_ref }}
33+
path: head
34+
35+
- name: Get all Helm files
36+
id: all_files_helm
37+
run: |
38+
helm_files=($(find ./head/test/head -type f -name 'helm.yaml' | sed "s|^./head/test/head/||" | sort))
39+
echo HELM_FILES="${helm_files[@]}" >> $GITHUB_OUTPUT
40+
ls -lR
41+
42+
- id: helm_diff
43+
name: Helm diff
44+
run: ../head/flux-helm-diff.sh
45+
working-directory: ./base
46+
env:
47+
TEST: "1"
48+
HELM_FILES: ${{ steps.all_files_helm.outputs.HELM_FILES }}
49+
shell: bash
50+
51+
- id: pr_comment
52+
name: Add PR comment
53+
uses: mshick/add-pr-comment@v2
54+
if: contains(fromJSON('["pull_request_target"]'), github.event_name)
55+
with:
56+
message-id: diff
57+
refresh-message-position: true
58+
message: |
59+
${{ steps.helm_diff.outputs.markdown }}

README.md

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# Flux Helm chart diff action
22

3-
![](https://github.com/abstrask/actions-playground/actions/workflows/helm-diff.yaml/badge.svg)
4-
53
A composite GitHub Action for use with PR workflows in repos with [Flux Helm manifests](https://fluxcd.io/flux/use-cases/helm/).
64

75
It extracts the repo URL, chart name and version and values, and renders the supplied list of templates before and after PR, and produce a diff report in markdown format.
@@ -44,7 +42,7 @@ In `diff_markdown` the output for each file will either be:
4442
name: Flux Helm diff
4543
uses: abstrask/flux-helm-diff@main
4644
with:
47-
helm_files: ${{ steps.changed_files_helm.outputs.modified_files }}
45+
helm_files: ${{ steps.changed_files_helm.outputs.all_changed_files }}
4846
```
4947
5048
### Full Example
@@ -71,7 +69,7 @@ jobs:
7169
outputs:
7270
helm: ${{ steps.filter.outputs.helm }}
7371
steps:
74-
- id: checkout_base
72+
- id: checkout_head
7573
uses: actions/checkout@v4
7674
with:
7775
ref: ${{ github.head_ref }}
@@ -143,7 +141,7 @@ Render templates and generate diff report:
143141
name: Helm diff
144142
uses: abstrask/actions-playground@main
145143
with:
146-
helm_files: ${{ steps.changed_files_helm.outputs.modified_files }}
144+
helm_files: ${{ steps.changed_files_helm.outputs.all_changed_files }}
147145
148146
```
149147

@@ -253,13 +251,20 @@ No changes
253251
(abbreviated)
254252
```
255253
256-
### infrastructure/base/weave-gitops/helm.yaml
257-
```
258-
Error: looks like "oci://ghcr.io/weaveworks/charts" is not a valid chart repository or cannot be reached: object required
259-
```
260-
261-
## Testing Locally
254+
## Testing
262255
263256
```bash
264-
GITHUB_OUTPUT=debug.out test=1 ./flux-helm-diff.sh helm1.yaml infrastructure/base/weave-gitops/helm.yaml infrastructure/base/nvidia-device-plugin/helm.yaml; cat debug.out
257+
helm_files=($(find ./test/head -type f -name 'helm.yaml' | sed "s|^./test/head/||" | sort))
258+
GITHUB_OUTPUT=debug.out HELM_FILES="${helm_files[@]}" TEST=1 ./flux-helm-diff.sh; cat debug.out
265259
```
260+
261+
### Testing files
262+
263+
| Name | Scenario tested | Expected output |
264+
| ----------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------- |
265+
| `dcgm-exporter` | Chart added in `head` that doesn't exist in `base` | Diff shows entire rendered template as added |
266+
| `metaflow` | Very non-standard way of publishing charts (not sure if should be supported) | TBD |
267+
| `nvidia-device-plugin` | HelmRepository (using `https`), minor chart version bump | Diff (with potentially breaking `nodeAffinity`) |
268+
| `weave-gitops-helm2oci` | Repository type changed from HelmRepository (type `oci`) to OCIRepository | No changes |
269+
| `weave-gitops-helmrepo` | HelmRepository with type `oci` | Diff |
270+
| `weave-gitops-ocirepo` | OCIRepository | Diff |

action.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,5 +29,7 @@ runs:
2929

3030
- id: diff
3131
name: Helm diff
32-
run: flux-helm-diff.sh ${{ inputs.helm_files }}
32+
run: flux-helm-diff.sh
33+
env:
34+
HELM_FILES: ${{ inputs.helm_files }}
3335
shell: bash

example-workflow.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ jobs:
1212
outputs:
1313
helm: ${{ steps.filter.outputs.helm }}
1414
steps:
15-
- id: checkout_base
15+
- id: checkout_head
1616
uses: actions/checkout@v4
1717
with:
1818
ref: ${{ github.head_ref }}
@@ -64,7 +64,7 @@ jobs:
6464
name: Helm diff
6565
uses: abstrask/flux-helm-diff@main
6666
with:
67-
helm_files: ${{ steps.changed_files_helm.outputs.modified_files }}
67+
helm_files: ${{ steps.changed_files_helm.outputs.all_changed_files }}
6868

6969
- id: pr_comment
7070
name: Add PR comment

flux-helm-diff.sh

Lines changed: 73 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,78 @@
11
#!/bin/bash
2+
set -eu -o pipefail
23

3-
if [ "${#}" == "0" ]; then
4+
helm_files=(${HELM_FILES[@]})
5+
if [ "${#helm_files[@]}" == "0" ]; then
46
echo "No Helm files specified, nothing to do"
57
exit
68
fi
7-
helm_files=( "${@}" )
8-
echo "Helm files to render: ${helm_files[*]}"
9+
echo "${#helm_files[@]} Helm file(s) to render: ${helm_files[*]}"
910

1011
helm_template() {
11-
if [ -z "${2}" ]; then
12+
set -eu -o pipefail
13+
if [ -z "${1}" ]; then
1214
echo "Error: Need file name to template" >&2
13-
return 1
15+
return 2
1416
fi
1517

18+
# 'head' or 'base' ref - used for logging output
19+
ref="${1%%/*}"
20+
1621
# Set test = <something> to run against Helm teplates under test/
17-
if [ -z "${test}" ]; then
18-
helm_file="${2}"
22+
if [ -z "${TEST:-}" ]; then
23+
helm_file="${1}"
1924
else
20-
helm_file="test/${2}"
25+
helm_file="test/${1}"
2126
fi
2227

2328
if [ ! -f "${helm_file}" ]; then
24-
echo "Error: File \"${helm_file}\" not found, skipping diff"
25-
echo "Error: File \"${helm_file}\" not found, skipping diff" >&2
29+
# echo "Warn: File \"${helm_file}\" not found, skipping"
30+
echo "File \"${helm_file}\" not found, skipping" >&2
2631
return 1
2732
fi
2833

34+
# Determine repo type - HelmRepository or OCIRepository
35+
# https://fluxcd.io/flux/components/source/helmrepositories/
36+
# https://fluxcd.io/flux/components/source/ocirepositories/
37+
# https://fluxcd.io/flux/components/source/gitrepositories/
38+
if [[ "HelmRepository" == "$(yq '. | select(.kind == "HelmRelease").spec.chart.spec.sourceRef.kind' "${helm_file}")" ]]; then
39+
repo_type=helm
40+
repo_name=$(yq '. | select(.kind == "HelmRelease").spec.chart.spec.sourceRef.name' "${helm_file}")
41+
chart=$(yq '. | select(.kind == "HelmRelease").spec.chart.spec.chart' "${helm_file}")
42+
url=$(yq '. | select(.kind == "HelmRepository").spec.url' "${helm_file}")
43+
version=$(yq '. | select(.kind == "HelmRelease").spec.chart.spec.version' "${helm_file}")
44+
if [[ "${url}" = "oci://"* ]]; then
45+
url="${url}/${chart}" # Syntax for chart repos is different from OCI repos (as HelmRepo kind)
46+
fi
47+
elif [[ "OCIRepository" == "$(yq '. | select(.kind == "HelmRelease").spec.chartRef.kind' "${helm_file}")" ]]; then
48+
repo_type=oci
49+
repo_name=$(yq '. | select(.kind == "HelmRelease").spec.chartRef.name' "${helm_file}")
50+
chart="${repo_name}"
51+
url=$(yq '. | select(.kind == "OCIRepository").spec.url' "${helm_file}")
52+
version=$(yq '. | select(.kind == "OCIRepository").spec.ref.tag' "${helm_file}")
53+
else
54+
echo "Unable to determine repo type, skipping"
55+
echo "Unable to determine repo type, skipping" >&2
56+
return 2
57+
fi
58+
2959
# Extracting chart properties
3060
name=$(yq '. | select(.kind == "HelmRelease").metadata.name' "${helm_file}")
3161
namespace=$(yq '. | select(.kind == "HelmRelease").metadata.namespace' "${helm_file}")
32-
version=$(yq '. | select(.kind == "HelmRelease") | .spec.chart.spec.version' "${helm_file}")
33-
url=$(yq '. | select(.kind == "HelmRepository") | .spec.url' "${helm_file}")
34-
chart=$(yq '. | select(.kind == "HelmRelease") | .spec.chart.spec.chart' "${helm_file}")
3562
values=$(yq '. | select(.kind == "HelmRelease").spec.values' "${helm_file}")
36-
echo "Chart version ${1}: $version ($chart from $url)" >&2
3763

38-
# Syntax for chart repos is different from OCI repos
64+
# Let's see what information we got out about the chart...
65+
echo "${ref} repo type: ${repo_type}" >&2
66+
echo "${ref} repo name: ${repo_name}" >&2
67+
echo "${ref} repo/chart URL: ${url}" >&2
68+
echo "${ref} chart name: ${chart}" >&2
69+
echo "${ref} chart version: ${version}" >&2
70+
echo "${ref} release name: ${name}" >&2
71+
echo "${ref} release namespace: ${namespace}" >&2
72+
73+
# Syntax for chart repos is different from OCI repos (as HelmRepo kind)
3974
if [[ "${url}" = "oci://"* ]]; then
40-
chart_args=("${url}/${chart}") # treat as array, to avoid adding single-quotes
75+
chart_args=("${url}") # treat as array, to avoid adding single-quotes
4176
else
4277
chart_args=("${chart}" --repo "${url}")
4378
fi
@@ -46,19 +81,16 @@ helm_template() {
4681
template_out=$(helm template "${name}" ${chart_args[@]} --version "${version}" -n "${namespace}" -f <(echo "${values}") 2>&1) || {
4782
echo "$template_out"
4883
echo "$template_out" >&2
49-
return 1
84+
return 2
5085
}
5186

5287
# Cleanup template, removing comments, output
5388
template_clean=$(yq -P 'sort_keys(..) comments=""' <(echo "${template_out}"))
5489
echo "$template_clean"
55-
56-
# Debug info
57-
echo "Line count ${1}: values ($(echo "${values}" | wc -l)), template ($(echo "${template_out}" | wc -l)), template_clean ($(echo "${template_clean}" | wc -l))" >&2
5890
}
5991

6092
EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
61-
echo "markdown<<$EOF" >> "$GITHUB_OUTPUT"
93+
echo "markdown<<$EOF" > "$GITHUB_OUTPUT"
6294
echo "## Flux Helm diffs" >> "$GITHUB_OUTPUT"
6395

6496
any_failed=0
@@ -70,26 +102,38 @@ for helm_file in "${helm_files[@]}"; do
70102

71103
# Template before
72104
return_code=0
73-
before_out=$(helm_template before "base/${helm_file}") || return_code=1
74-
if [ $return_code -ne 0 ]; then
105+
base_out=$(helm_template "base/${helm_file}") || return_code=$?
106+
if [ $return_code -eq 2 ]; then # Ignore files skipped
75107
{
76108
echo '```'
77-
echo "${before_out}"
109+
echo "Error rendering base ref:"
110+
echo "${base_out}"
78111
echo '```'
79112
} >> "$GITHUB_OUTPUT"
80113
any_failed=1
81114
continue
82115
fi
83116

84117
# Template after
85-
after_out=$(helm_template after "head/${helm_file}") || true
118+
return_code=0
119+
head_out=$(helm_template "head/${helm_file}") || return_code=$?
120+
if [ $return_code -ne 0 ]; then
121+
{
122+
echo '```'
123+
echo "Error rendering head ref:"
124+
echo "${head_out}"
125+
echo '```'
126+
} >> "$GITHUB_OUTPUT"
127+
any_failed=1
128+
continue
129+
fi
86130

87131
# Template diff
88-
diff_out=$(diff --unified=5 <(echo "${before_out}") <(echo "${after_out}")) || true
132+
diff_out=$(diff --unified=5 <(echo "${base_out}") <(echo "${head_out}")) || true
89133
echo "Diff has $(echo "$diff_out" | wc -l) line(s)"
90134
[ -z "${diff_out}" ] && diff_out="No changes"
91135
{
92-
echo '```'
136+
echo '```diff'
93137
echo "${diff_out}"
94138
echo '```'
95139
} >> "$GITHUB_OUTPUT"
@@ -99,3 +143,5 @@ done
99143
echo "$EOF"
100144
echo "any_failed=$any_failed"
101145
} >> "$GITHUB_OUTPUT"
146+
147+
echo -e "\nAll done"

test/base/infrastructure/base/dcgm-exporter/helm.yaml

Lines changed: 0 additions & 51 deletions
This file was deleted.

0 commit comments

Comments
 (0)