Skip to content

Commit 9a23fa6

Browse files
authored
Merge pull request #156 from control-toolbox/docs/release-setup
feat(release): setup.sh to set a package up for release.yml; document the registration token
2 parents 82ddb00 + b0c5a2d commit 9a23fa6

5 files changed

Lines changed: 166 additions & 43 deletions

File tree

‎.github/workflows/release-tests.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,6 @@ jobs:
4141
run: julia --startup-file=no -e 'using Pkg; Pkg.Registry.add("General")'
4242
- name: Tests
4343
run: julia --startup-file=no scripts/release/test/runtests.jl
44+
- name: Tests of setup.sh
45+
if: matrix.version == '1.12'
46+
run: bash scripts/release/setup_test.sh

‎docs/conventions/secrets.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
| `PROJECT_TOKEN` → `project_token` | organization (all repositories) | `add-to-project.yml` | Write to the organization's project board |
1313
| `CT_PRUNE_COMPAT_CLIENT_ID`, `CT_PRUNE_COMPAT_PRIVATE_KEY` | organization, **CTActions only** | `compat-prune-monthly.yml` (`publish` job) | Token of the organization's [GitHub App](github-app.md) |
1414
| `CT_RELEASE_CLIENT_ID`, `CT_RELEASE_PRIVATE_KEY` | **the environments `release` and `release-autonomous` of each package**, not the repository nor the organization | `release.yml` (jobs `start`, `merge`, `register`) | Token of the App [`ct-release`](github-app.md#ct-release-the-app-of-the-releases): only the jobs of the environment, after its approval, can mint it |
15-
| `CT_RELEASE_REGISTRATION_TOKEN` (recommended) | the same environments | `release.yml` (job `register`) | The token of a release manager who is a **public** member of the organization, for the registration comment of a stable: the registration bot takes no comment of an App. Without it, the comment is handed to a person ([details](../workflows/release.md#the-registration-comment)) |
15+
| `CT_RELEASE_REGISTRATION_TOKEN` (recommended) | the same environments, set by [`setup.sh`](../workflows/release.md#setting-it-up) | `release.yml` (job `register`) | The token of a release manager who is a **public** member of the organization, for the registration comment of a stable: the registration bot takes no comment of an App. Without it, the comment is handed to a person ([details](../workflows/release.md#the-registration-comment)) |
1616
| `CT_RUNNERS_CLIENT_ID`, `CT_RUNNERS_PRIVATE_KEY` | organization (all repositories) | `select-gpu-runner.yml` | Token of the read-only App `ct-runners` ([details](../workflows/select-gpu-runner.md#the-github-app-ct-runners)) |
1717
| `OCCIDATA_SSH_KEY`, `OCCIDATA_KNOWN_HOSTS` | CTActions | `occidata-runner-watchdog.yml` | SSH to `occidata-cluster.irit.fr` |
1818
| `GITHUB_TOKEN` | built in | most workflows | Token of the repository the workflow runs in |

‎docs/workflows/release.md‎

Lines changed: 25 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -217,56 +217,39 @@ Once, by an organization owner:
217217
1. **The App** `ct-release`: see [GitHub App](../conventions/github-app.md#ct-release-the-app-of-the-releases).
218218
2. **The release managers**: `gh variable set CT_RELEASE_MANAGERS --org control-toolbox --visibility all --body 'ocots'`.
219219

220-
In each package, by an administrator of the repository:
220+
In each package, by an administrator of the repository, with
221+
[`scripts/release/setup.sh`](https://github.com/control-toolbox/CTActions/blob/main/scripts/release/setup.sh)
222+
(idempotent: run it again for a new token, or for a new package):
221223

222224
```bash
223-
REPO=control-toolbox/CTParser.jl
224-
# the environments: `release` with a required reviewer, `release-autonomous` without
225-
me=$(gh api users/ocots --jq .id)
226-
gh api --method PUT "repos/$REPO/environments/release" --input - <<EOF
227-
{"reviewers": [{"type": "User", "id": $me}]}
228-
EOF
229-
gh api --method PUT "repos/$REPO/environments/release-autonomous" --input - <<< '{}'
230-
# the App's secrets, in each environment (not in the repository: only these jobs see them)
231-
for env in release release-autonomous; do
232-
gh secret set CT_RELEASE_CLIENT_ID -R "$REPO" --env "$env" --body '<client id>'
233-
gh secret set CT_RELEASE_PRIVATE_KEY -R "$REPO" --env "$env" < ct-release.private-key.pem
234-
done
235-
# the labels
236-
gh label create "release: supervised" -R "$REPO" --color 0E8A16 --description "Hand phases 4 to 8 of this release PR to Release, with approvals"
237-
gh label create "release: autonomous" -R "$REPO" --color B60205 --description "Hand phases 4 to 8 of this release PR to Release, without approvals"
225+
CT_RELEASE_CLIENT_ID='<client id>' \
226+
CT_RELEASE_KEY_FILE=ct-release.private-key.pem \
227+
CT_RELEASE_REGISTRATION_TOKEN='<token, see below>' \
228+
bash scripts/release/setup.sh CTParser.jl CTModels.jl # --dry-run to see the commands
238229
```
239230

240-
Then add the caller. To start with supervised mode only, do not create
241-
`release-autonomous`: GitHub then creates it empty when a job names it, without the App's
242-
secrets, and a run in autonomous mode fails at the token, before anything is done.
231+
For each repository it creates:
243232

244-
**`CT_RELEASE_REGISTRATION_TOKEN`, for an automatic registration.** Set this environment
245-
secret, in each environment, to a fine-grained token of a release manager who is a
246-
**public** member of the organization (the package, *Issues: read and write*). See
247-
[The registration comment](#the-registration-comment).
233+
- the environment `release`, with `ocots` as required reviewer (`CT_RELEASE_REVIEWER` to change it);
234+
- the App's secrets `CT_RELEASE_CLIENT_ID` and `CT_RELEASE_PRIVATE_KEY`, and the token below, in the **environment**, not in the repository: only the jobs of the environment, after its approval, see them;
235+
- the label `release: supervised`.
248236

249-
## The registration comment
237+
`--autonomous` also creates `release-autonomous` (no reviewer, same secrets) and the label `release: autonomous`. Without it, GitHub creates `release-autonomous` empty when a job names it, without the App's secrets, and a run in autonomous mode fails at the token, before anything is done.
250238

251-
The registration bot takes a comment only from a **public member** of the organization: an
252-
App never is one, so it refuses the App's comment ("not a publicly listed member", seen on
253-
CTParser 0.9.6). The comment is therefore a person's:
239+
By hand, once per package: the App `ct-release` must be **installed** on it (organization settings, *Installed GitHub Apps*, *Selected repositories*). Then add the caller above.
254240

255-
| `CT_RELEASE_REGISTRATION_TOKEN` | What the job `register` does | Then |
256-
| --- | --- | --- |
257-
| set (the default to aim at) | posts the comment as that person, once, after the approval in supervised mode | the bot opens General's PR; the workflow follows it |
258-
| not set | posts, as the App, the comment for a person to post: in a code block, its mention broken by an apostrophe (`@'JuliaRegistrator`), with the instruction to remove it | the workflow waits, without asking for an approval again; a person posts it once, and the workflow follows General's PR |
259-
260-
Both modes use the token: in supervised mode the approval of the job is the person's
261-
control, and a copy to paste on top of it would check nothing more. Without the token, both
262-
modes wait for the person.
263-
264-
The mention is broken because the bot reacts to it wherever it is written, even in a code
265-
block or a quote. If the bot refuses a comment anyway (the token's person is not a public
266-
member, for instance), the workflow never posts again: it stops, and its stop shows the
267-
same comment for a person to post. A comment the bot refused registered nothing, so `REL6`
268-
(one registration mention) does not count it: `status` and `release.jl audit` count only
269-
the mentions the bot did not refuse.
241+
### The registration token
242+
243+
`CT_RELEASE_REGISTRATION_TOKEN` is what the registration comment of a stable is posted with ([why](#the-registration-comment)). It is the token of a release manager who is a **public** member of the organization: the registration bot takes no comment of an App.
244+
245+
1. As that person: *Settings, Developer settings, Personal access tokens, Fine-grained tokens, Generate new token*.
246+
2. *Resource owner*: `control-toolbox` (an owner approves the token under the organization's *Personal access tokens, Pending requests* if the organization requires it). *Repository access*: *Only select repositories*, the packages released with `release.yml`. An expiration date: at most a year.
247+
3. *Repository permissions*: **Issues: read and write**, and **Contents: read and write** to cover the registration comment on a commit (when the release PR's merge commit moved); *Metadata: read* comes with them. Nothing else.
248+
4. Copy the value (shown once) and give it to `setup.sh`. One token serves all the packages; add each new package to its repositories.
249+
250+
An environment secret is read after the approval of the environment, which is why it is set per package and not once for the organization (an organization secret would be readable by any workflow of the repositories it is shared with). When the token expires, generate it again and run `setup.sh` again.
251+
252+
Without the token, the workflow hands the comment to a person ([details](#the-registration-comment)).
270253

271254
## Not verified yet
272255

‎scripts/release/setup.sh‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#!/usr/bin/env bash
2+
# Sets a package up for release.yml: the environments, their secrets and the labels
3+
# (docs/workflows/release.md#setting-it-up). Idempotent: run it again after a change of
4+
# the token, or for a new package. Needs `gh` logged in as an administrator of the packages.
5+
#
6+
# CT_RELEASE_CLIENT_ID=... CT_RELEASE_KEY_FILE=ct-release.private-key.pem \
7+
# CT_RELEASE_REGISTRATION_TOKEN=... \
8+
# bash scripts/release/setup.sh [--dry-run] [--autonomous] REPO...
9+
#
10+
# REPO is `owner/name`, or `name` for control-toolbox/name. The environment `release`
11+
# (a required reviewer) is always set up; `release-autonomous` (no reviewer) only with
12+
# `--autonomous`. The token of the registration comment is optional.
13+
# CT_RELEASE_REVIEWER: the login of the reviewer (default: ocots).
14+
set -euo pipefail
15+
16+
dry=false
17+
autonomous=false
18+
repos=()
19+
for a in "$@"; do
20+
case "$a" in
21+
--dry-run) dry=true ;;
22+
--autonomous) autonomous=true ;;
23+
-h|--help) sed -n '2,13p' "$0"; exit 0 ;;
24+
-*) echo "unknown option: $a" >&2; exit 2 ;;
25+
*/*) repos+=("$a") ;;
26+
*) repos+=("control-toolbox/$a") ;;
27+
esac
28+
done
29+
[ ${#repos[@]} -gt 0 ] || { echo "no repository given" >&2; exit 2; }
30+
31+
: "${CT_RELEASE_CLIENT_ID:?the client id of the App ct-release}"
32+
: "${CT_RELEASE_KEY_FILE:?the private key of the App ct-release (a .pem file)}"
33+
[ -r "$CT_RELEASE_KEY_FILE" ] || { echo "cannot read $CT_RELEASE_KEY_FILE" >&2; exit 2; }
34+
reviewer="${CT_RELEASE_REVIEWER:-ocots}"
35+
36+
# do <stdin file or -> <command...>: run it, or only print it with --dry-run
37+
do_() {
38+
local in="$1"; shift
39+
if $dry; then echo " [dry-run] $*"; return 0; fi
40+
if [ "$in" = - ]; then "$@" </dev/null; else "$@" <"$in"; fi
41+
}
42+
43+
if $dry; then me=0; else me=$(gh api "users/$reviewer" --jq .id); fi
44+
envs=(release)
45+
$autonomous && envs+=(release-autonomous)
46+
client=$(mktemp); trap 'rm -f "$client"' EXIT
47+
printf %s "$CT_RELEASE_CLIENT_ID" >"$client"
48+
token=""
49+
if [ -n "${CT_RELEASE_REGISTRATION_TOKEN:-}" ]; then
50+
token=$(mktemp); trap 'rm -f "$client" "$token"' EXIT
51+
printf %s "$CT_RELEASE_REGISTRATION_TOKEN" >"$token"
52+
fi
53+
54+
for repo in "${repos[@]}"; do
55+
echo "== $repo"
56+
if ! $dry; then
57+
[ "$(gh api "repos/$repo" --jq .permissions.admin)" = true ] \
58+
|| { echo " skipped: you are not an administrator of $repo" >&2; continue; }
59+
fi
60+
for env in "${envs[@]}"; do
61+
if [ "$env" = release ]; then
62+
body="{\"reviewers\": [{\"type\": \"User\", \"id\": $me}]}"
63+
else
64+
body='{}'
65+
fi
66+
if $dry; then echo " [dry-run] gh api --method PUT repos/$repo/environments/$env # $body"
67+
else gh api --method PUT "repos/$repo/environments/$env" --input - <<<"$body" >/dev/null; fi
68+
do_ "$client" gh secret set CT_RELEASE_CLIENT_ID -R "$repo" --env "$env"
69+
do_ "$CT_RELEASE_KEY_FILE" gh secret set CT_RELEASE_PRIVATE_KEY -R "$repo" --env "$env"
70+
[ -z "$token" ] || do_ "$token" gh secret set CT_RELEASE_REGISTRATION_TOKEN -R "$repo" --env "$env"
71+
done
72+
do_ - gh label create "release: supervised" -R "$repo" --force --color 0E8A16 \
73+
--description "Hand phases 4 to 8 of this release PR to Release, with approvals"
74+
$autonomous && do_ - gh label create "release: autonomous" -R "$repo" --force --color B60205 \
75+
--description "Hand phases 4 to 8 of this release PR to Release, without approvals"
76+
[ -n "$token" ] || echo " no CT_RELEASE_REGISTRATION_TOKEN: the registration comment will be handed to a person"
77+
echo " to check by hand: the App ct-release is installed on $repo (organization settings, Installed GitHub Apps)"
78+
done

‎scripts/release/setup_test.sh‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
#!/usr/bin/env bash
2+
# Tests of setup.sh, with a fake `gh`: bash scripts/release/setup_test.sh
3+
set -uo pipefail
4+
setup="$(cd "$(dirname "$0")" && pwd)/setup.sh"
5+
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT
6+
failed=0
7+
log="$tmp/gh.log"
8+
cat >"$tmp/gh" <<'FAKE'
9+
#!/usr/bin/env bash
10+
case "$1 $2" in
11+
"api users/ocots") echo 42 ;;
12+
"api repos/control-toolbox/Admin") echo true ;;
13+
"api repos/control-toolbox/Reader") echo false ;;
14+
"api --method")
15+
# the body of the environment comes on stdin
16+
echo "gh $* <<< $(cat)" >>"$GH_LOG"; exit 0 ;;
17+
"secret set") echo "gh $* <<< $(cat)" >>"$GH_LOG"; exit 0 ;;
18+
esac
19+
echo "gh $*" >>"$GH_LOG"
20+
FAKE
21+
chmod +x "$tmp/gh"
22+
printf 'KEY' >"$tmp/key.pem"
23+
export PATH="$tmp:$PATH" GH_LOG="$log" CT_RELEASE_CLIENT_ID=Iv1.abc CT_RELEASE_KEY_FILE="$tmp/key.pem"
24+
25+
check() { # <name> <pattern> <file>
26+
if grep -qF -- "$2" "$3"; then echo "ok $1"; else echo "FAIL $1: no '$2' in:"; sed 's/^/ /' "$3"; failed=1; fi
27+
}
28+
lacks() {
29+
if grep -qF -- "$2" "$3"; then echo "FAIL $1: found '$2'"; failed=1; else echo "ok $1"; fi
30+
}
31+
32+
: >"$log"
33+
bash "$setup" Admin >"$tmp/out" 2>&1 || { echo "FAIL a run"; cat "$tmp/out"; failed=1; }
34+
check "the environment release has the reviewer" '"id": 42' "$log"
35+
check "the key is read from the file" "secret set CT_RELEASE_PRIVATE_KEY -R control-toolbox/Admin --env release <<< KEY" "$log"
36+
check "the client id is a secret of the environment" "secret set CT_RELEASE_CLIENT_ID -R control-toolbox/Admin --env release <<< Iv1.abc" "$log"
37+
check "the supervised label" "label create release: supervised -R control-toolbox/Admin --force" "$log"
38+
lacks "no autonomous environment by default" "release-autonomous" "$log"
39+
lacks "no token, no secret for it" "REGISTRATION_TOKEN" "$log"
40+
check "a missing token is said" "no CT_RELEASE_REGISTRATION_TOKEN" "$tmp/out"
41+
42+
: >"$log"
43+
CT_RELEASE_REGISTRATION_TOKEN=github_pat_x bash "$setup" --autonomous Admin >/dev/null 2>&1
44+
check "the token is a secret of the environment" "secret set CT_RELEASE_REGISTRATION_TOKEN -R control-toolbox/Admin --env release <<< github_pat_x" "$log"
45+
check "the autonomous environment" "environments/release-autonomous" "$log"
46+
check "the autonomous label" "label create release: autonomous" "$log"
47+
48+
: >"$log"
49+
bash "$setup" Reader >"$tmp/out" 2>&1
50+
check "a repository without admin rights is skipped" "skipped" "$tmp/out"
51+
lacks "and left alone" "secret set" "$log"
52+
53+
: >"$log"
54+
bash "$setup" --dry-run owner/Name >"$tmp/out" 2>&1
55+
check "dry run prints" "[dry-run] gh secret set CT_RELEASE_PRIVATE_KEY -R owner/Name" "$tmp/out"
56+
if [ -s "$log" ]; then echo "FAIL dry run called gh"; failed=1; else echo "ok dry run runs nothing"; fi
57+
58+
if CT_RELEASE_CLIENT_ID="" bash "$setup" Admin >/dev/null 2>&1; then echo "FAIL needs the client id"; failed=1; else echo "ok the client id is required"; fi
59+
exit $failed

0 commit comments

Comments
 (0)