Skip to content

Commit 063bbd7

Browse files
committed
feat(docker): Support CA certificates during image builds, and add docker build helpers.
1 parent 9a79bc7 commit 063bbd7

8 files changed

Lines changed: 586 additions & 34 deletions

File tree

docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,5 @@ To use the repo's artifacts in your project:
1919
For language/tool-specific guides, see:
2020

2121
* [CA trust for containerized builds](../exports/docker/ca-trust/README.md)
22+
* [Docker build helpers](../exports/docker/build/README.md)
2223
* [Linting for C++ projects](lint-tools-cpp.md)

exports/docker/build/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Docker build
2+
3+
Small helpers for assembling a `docker build` command line.
4+
5+
Building an image from inside a company network usually means repeating the same handful of options every time: send your proxy settings into the build, point the package manager at an internal mirror, refresh the base image, and record which commit the image came from. Every project ends up writing that slightly differently. These functions write it once.
6+
7+
They have nothing to do with certificates — for that, see [ca-trust](../ca-trust/README.md).
8+
9+
## Requirements
10+
11+
* `bash` 3.2 or newer (macOS's `/bin/bash` qualifies).
12+
* `docker` with `buildx` (Docker 23 or newer). `docker_build_run` checks for it and fails with a clear message if it's missing.
13+
14+
## Usage
15+
16+
```bash
17+
source tools/yscope-dev-utils/exports/docker/build/host.sh
18+
19+
build_cmd=(docker buildx build --tag <tag> --file <dockerfile> <context>)
20+
docker_build_finalize build_cmd "${repo_root}" APT_MIRROR_URL
21+
```
22+
23+
`docker_build_finalize` runs everything below in order. Call the individual functions instead if you need to leave one out.
24+
25+
## What each function does
26+
27+
`docker_build_add_proxy_args <cmd-array-name>` copies your proxy settings into the build, so downloads that happen while the image is being built go through the same proxy your shell uses. It reads `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY`, in either upper or lower case.
28+
29+
It also handles a trap that's easy to hit: if your proxy runs on your own machine, the build can't reach it by default, because Docker gives the build its own private network where "this machine" means something else. When an address like `localhost`, `127.0.0.1`, or `[::1]` is detected — including when it's written as `http://user:pass@127.0.0.1:8080` — the build is switched to your machine's network so the proxy is reachable. Set `DOCKER_NETWORK` to choose the network yourself.
30+
31+
`docker_build_add_env_build_args <cmd-array-name> [var-name...]` passes named environment variables into the build, skipping any that aren't set. Use it for your own settings — an internal package mirror, say — without this library needing to know what they're called.
32+
33+
`docker_build_add_pull_arg <cmd-array-name>` re-downloads the base image before building, so you don't silently build on a stale local copy. Set `DOCKER_PULL=false` to skip it, e.g. when building offline.
34+
35+
`docker_build_add_oci_labels <cmd-array-name> <repo-dir>` stamps the image with the commit and repository URL it was built from, so a built image can be traced back to its source. Does nothing if `<repo-dir>` isn't a git checkout.
36+
37+
`docker_build_run <cmd-array-name>` checks that `buildx` is available, prints the assembled command, and runs it.
38+
39+
Both drop credentials from URLs first: a remote like `https://user:token@github.com/org/repo` would otherwise be written into an image label that follows the image to every registry, and a proxy password would end up in the build log. The command itself still runs with the real values.
40+
41+
## Why the functions take an array's *name*
42+
43+
Each function is given the name of a bash array — `build_cmd`, not `"${build_cmd[@]}"` — and appends to it in place, so the command stays a list of separate words. Building it as one long string instead would mean the shell re-splitting that text later, and a proxy or mirror URL containing a space, a quote, or a `$` — common in real ones — would come apart.
44+
45+
Appending by name normally calls for a bash "name reference", but those need bash 4.3 and macOS ships 3.2, so `utils.sh` does it with `printf %q` instead: each value is written in a form the shell reads back as exactly the original bytes.

exports/docker/build/host.sh

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
#!/usr/bin/env bash
2+
3+
# Host-side helpers for assembling a `docker build` command.
4+
#
5+
# Each function appends flags to a caller-owned bash array, passed by name.
6+
# docker_utils_append_args does the appending; see utils.sh for why it's written
7+
# the way it is.
8+
9+
if [[ "${_DOCKER_BUILD_HOST_SH_LOADED:-}" == "1" ]]; then
10+
return 0
11+
fi
12+
readonly _DOCKER_BUILD_HOST_SH_LOADED=1
13+
14+
# shellcheck source=exports/docker/utils.sh
15+
source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd)/utils.sh"
16+
17+
# Proxy variables forwarded into the build, in both spellings because tools
18+
# disagree about which they read.
19+
readonly _DOCKER_BUILD_PROXY_VARS=(
20+
HTTP_PROXY http_proxy
21+
HTTPS_PROXY https_proxy
22+
ALL_PROXY all_proxy
23+
NO_PROXY no_proxy
24+
)
25+
26+
# Echoes <value> with any URL credentials replaced by <replacement> (removed when
27+
# that is omitted).
28+
#
29+
# Proxy URLs and git remotes both carry credentials in practice --
30+
# `http://user:token@proxy.corp:8080`, `https://user:token@github.com/org/repo` --
31+
# and this library would otherwise copy them into an image label or a build log,
32+
# where they outlive the build.
33+
#
34+
# Args: <value> [replacement]
35+
_docker_build_replace_userinfo() {
36+
local value="$1" replacement="${2:-}"
37+
# Greedy `.*://` so the last scheme wins, and `[^/@]*` keeps the match inside
38+
# the authority: an `@` later in a path is not credentials.
39+
if [[ "${value}" =~ ^(.*://)[^/@]*@(.*)$ ]]; then
40+
printf '%s%s%s' "${BASH_REMATCH[1]}" "${replacement}" "${BASH_REMATCH[2]}"
41+
return 0
42+
fi
43+
printf '%s' "${value}"
44+
}
45+
46+
# Appends `--build-arg` flags for any set proxy variables, and picks a network
47+
# mode.
48+
#
49+
# A proxy on the host's loopback address is unreachable from the build
50+
# container's default bridge network, so `--network host` is selected
51+
# automatically in that case. DOCKER_NETWORK overrides the choice entirely.
52+
#
53+
# Args: <cmd-array-name>
54+
docker_build_add_proxy_args() {
55+
if (( $# != 1 )) || [[ -z "$1" ]]; then
56+
echo >&2 "ERROR: docker_build_add_proxy_args requires a command array"
57+
return 2
58+
fi
59+
local cmd_name="$1"
60+
61+
local has_loopback_proxy=false
62+
local var value
63+
for var in "${_DOCKER_BUILD_PROXY_VARS[@]}"; do
64+
value="${!var:-}"
65+
[[ -z "${value}" ]] && continue
66+
docker_utils_append_args "${cmd_name}" "--build-arg" "${var}=${value}"
67+
# Match the host after either the scheme or userinfo credentials, so
68+
# http://user:pass@127.0.0.1:8080 is recognized too. Also accept a bare
69+
# trailing host with no port or path.
70+
if [[ "${var}" != *NO_PROXY* && "${var}" != *no_proxy* ]] \
71+
&& [[ "${value}" =~ (://|@)(localhost|127\.0\.0\.1|\[::1\])([:/]|$) ]]; then
72+
has_loopback_proxy=true
73+
fi
74+
done
75+
76+
if [[ -n "${DOCKER_NETWORK:-}" ]]; then
77+
docker_utils_append_args "${cmd_name}" "--network" "${DOCKER_NETWORK}"
78+
elif [[ "${has_loopback_proxy}" == "true" ]]; then
79+
docker_utils_append_args "${cmd_name}" "--network" "host"
80+
fi
81+
}
82+
83+
# Appends a `--build-arg` for each named environment variable that is set and
84+
# non-empty. Lets consumers forward their own build knobs (package mirrors, for
85+
# instance) without this library knowing their names.
86+
#
87+
# Args: <cmd-array-name> [var-name...]
88+
docker_build_add_env_build_args() {
89+
if (( $# < 1 )) || [[ -z "$1" ]]; then
90+
echo >&2 "ERROR: docker_build_add_env_build_args requires a command array"
91+
return 2
92+
fi
93+
local cmd_name="$1"
94+
shift
95+
96+
local var value
97+
for var in "$@"; do
98+
value="${!var:-}"
99+
[[ -n "${value}" ]] && docker_utils_append_args "${cmd_name}" \
100+
"--build-arg" "${var}=${value}"
101+
done
102+
# Explicit: the `&&` above leaves a nonzero status when the last variable is
103+
# unset, which would abort callers running under `errexit`.
104+
return 0
105+
}
106+
107+
# Appends `--pull` unless DOCKER_PULL is "false", so builds refresh their base
108+
# image by default.
109+
#
110+
# Args: <cmd-array-name>
111+
docker_build_add_pull_arg() {
112+
if (( $# != 1 )) || [[ -z "$1" ]]; then
113+
echo >&2 "ERROR: docker_build_add_pull_arg requires a command array"
114+
return 2
115+
fi
116+
[[ "${DOCKER_PULL:-true}" != "false" ]] && docker_utils_append_args "$1" "--pull"
117+
return 0
118+
}
119+
120+
# Appends OCI source-provenance labels derived from the git repo at <repo-dir>.
121+
# A no-op outside a git work tree.
122+
#
123+
# Args: <cmd-array-name> <repo-dir>
124+
docker_build_add_oci_labels() {
125+
if (( $# != 2 )) || [[ -z "$1" || -z "$2" ]]; then
126+
echo >&2 "ERROR: docker_build_add_oci_labels requires a command array and a repo directory"
127+
return 2
128+
fi
129+
local cmd_name="$1" repo_dir="$2"
130+
131+
command -v git &>/dev/null || return 0
132+
git -C "${repo_dir}" rev-parse --is-inside-work-tree &>/dev/null || return 0
133+
134+
local revision
135+
if revision="$(git -C "${repo_dir}" rev-parse HEAD 2>/dev/null)"; then
136+
docker_utils_append_args "${cmd_name}" \
137+
"--label" "org.opencontainers.image.revision=${revision}"
138+
fi
139+
140+
local remote_url
141+
if remote_url="$(git -C "${repo_dir}" remote get-url origin 2>/dev/null)"; then
142+
# Credentials stripped: a label travels with the image to every registry
143+
# and `docker inspect` that ever sees it.
144+
remote_url="$(_docker_build_replace_userinfo "${remote_url}")"
145+
docker_utils_append_args "${cmd_name}" \
146+
"--label" "org.opencontainers.image.source=${remote_url}"
147+
fi
148+
return 0
149+
}
150+
151+
# Echoes and runs the assembled command.
152+
#
153+
# Args: <cmd-array-name>
154+
docker_build_run() {
155+
if (( $# != 1 )) || [[ -z "$1" ]]; then
156+
echo >&2 "ERROR: docker_build_run requires a command array"
157+
return 2
158+
fi
159+
local length
160+
length="$(docker_utils_array_length "$1")" || return 2
161+
if (( length == 0 )); then
162+
echo >&2 "ERROR: docker_build_run got an empty command array: $1"
163+
return 2
164+
fi
165+
166+
if ! docker buildx version &>/dev/null; then
167+
echo >&2 "ERROR: docker buildx is required (Docker 23 or newer)."
168+
return 1
169+
fi
170+
171+
# Copied out by name, since the array belongs to the caller.
172+
local cmd
173+
eval "cmd=(\"\${$1[@]}\")"
174+
175+
# The echoed line is for humans and CI logs, so proxy credentials are masked
176+
# there. The command itself runs with the values untouched.
177+
local arg printable=()
178+
for arg in "${cmd[@]}"; do
179+
printable+=("$(_docker_build_replace_userinfo "${arg}" "***@")")
180+
done
181+
echo "Running: ${printable[*]}"
182+
183+
"${cmd[@]}"
184+
}
185+
186+
# The common composition: proxy args, forwarded build args, pull, labels, run.
187+
# CA trust is deliberately not included -- it's opt-in and the consumer calls
188+
# ca_trust_add_build_args itself.
189+
#
190+
# Args: <cmd-array-name> <repo-dir> [build-arg-var-name...]
191+
docker_build_finalize() {
192+
if (( $# < 2 )) || [[ -z "$1" || -z "$2" ]]; then
193+
echo >&2 "ERROR: docker_build_finalize requires a command array and a repo directory"
194+
return 2
195+
fi
196+
local cmd_name="$1" repo_dir="$2"
197+
shift 2
198+
199+
docker_build_add_proxy_args "${cmd_name}" || return
200+
docker_build_add_env_build_args "${cmd_name}" "$@" || return
201+
docker_build_add_pull_arg "${cmd_name}" || return
202+
docker_build_add_oci_labels "${cmd_name}" "${repo_dir}" || return
203+
docker_build_run "${cmd_name}"
204+
}

0 commit comments

Comments
 (0)