feat(docker): Move the ca-trust library here from clp-plugin-presto-connector.

docs/index.md

@@ -18,4 +18,5 @@ To use the repo's artifacts in your project:
 
 For language/tool-specific guides, see:
 
+* [CA trust for containerized builds](../exports/docker/ca-trust/README.md)
 * [Linting for C++ projects](lint-tools-cpp.md)

exports/docker/ca-trust/README.md

@@ -0,0 +1,75 @@
+# CA trust
+
+A library for propagating the host's trusted CA certificates into containerized builds that run behind a TLS-inspecting (e.g., corporate) gateway. Trust is wired up through environment variables and a bind-mounted staging directory, so certificates are never installed into an image or persisted in layers, caches, or artifacts.
+
+The examples below assume the consuming project has this repo as a submodule at `tools/yscope-dev-utils` (see the [usage docs](../../../docs/index.md#usage)) and mounts itself at `/repo` inside the build container; adjust the paths to your layout.
+
+## Requirements
+
+* Host: `bash`, plus a container runtime that supports bind mounts (the examples use Docker).
+  * `openssl` (optional): used to drop expired certificates during staging; without it, the bundle is copied as-is.
+* Container: `bash`.
+  * `findmnt` (optional): used to verify `CA_TRUST_DIR` is not on the container's writable overlay; without it, a warning is printed and the build proceeds.
+  * A JDK providing `keytool` (JVM builds only): used to generate the PKCS#12 trust store; without it, JVM trust setup is skipped.
+
+## Quick start
+
+On the host, stage the CA bundle into a temporary directory. Bind-mount that directory (writable) into the container, point `CA_TRUST_DIR` at the mount, and source `container.sh` before running the build:
+
+```bash
+# Host side
+source tools/yscope-dev-utils/exports/docker/ca-trust/host.sh
+
+CA_TRUST_HOST_DIR="$(mktemp -d)"
+trap 'rm -rf "${CA_TRUST_HOST_DIR}"' EXIT
+
+# Creates ${CA_TRUST_HOST_DIR}/ca-bundle.pem (read-only). Check the status: running
+# the build without host CA trust is the failure this library exists to avoid.
+stage_host_ca_bundle "${CA_TRUST_HOST_DIR}" || exit 1
+
+docker run --rm \
+    --mount "type=bind,src=${PWD},dst=/repo" \
+    --mount "type=bind,src=${CA_TRUST_HOST_DIR},dst=${CA_TRUST_CONTAINER_DIR}" \
+    --env "CA_TRUST_DIR=${CA_TRUST_CONTAINER_DIR}" \
+    --env "CA_TRUST_JVM=1" \
+    --env MAVEN_OPTS \
+    <image> \
+    bash -c '
+        source /repo/tools/yscope-dev-utils/exports/docker/ca-trust/container.sh
+        # Run the build; curl, git, pip, and Maven now trust the host CAs.
+    '
+```
+
+`CA_TRUST_JVM=1` and `--env MAVEN_OPTS` are only needed for JVM builds; see [JVM builds](#jvm-builds).
+
+## Host API (`host.sh`)
+
+`stage_host_ca_bundle <trust-dir>` writes the host's CA bundle to `<trust-dir>/${CA_TRUST_BUNDLE_FILENAME}` (read-only, `0444`):
+
+* The bundle is taken from `SSL_CERT_FILE` when set; otherwise, common Linux CA-bundle locations are searched. If none is found (e.g., on macOS without `SSL_CERT_FILE`), an empty file is created and the build proceeds without host CA context.
+* Expired certificates are dropped during staging (when `openssl` is available on the host), since a single expired certificate in a bundle can break TLS verification for otherwise-valid chains.
+
+Constants:
+
+* `CA_TRUST_BUNDLE_FILENAME` (`ca-bundle.pem`): the staged bundle's filename; `container.sh` reads it from `CA_TRUST_DIR` by this name.
+* `CA_TRUST_CONTAINER_DIR` (`/run/ca-trust`): the conventional in-container mount point for the staged trust directory, passed to the container as `CA_TRUST_DIR`.
+
+The caller owns the staging directory and cleans it up (e.g., with `trap`, as above). The scripts never modify the host's or the container's installed trust stores.
+
+## Container API (`container.sh`)
+
+Source it after setting `CA_TRUST_DIR` to the (writable) mount of the staged trust directory. It's a no-op when `CA_TRUST_DIR` is unset, so builds that don't mount a trust directory are unaffected.
+
+When the staged bundle is non-empty, it exports `CURL_CA_BUNDLE`, `GIT_SSL_CAINFO`, `PIP_CERT`, `REQUESTS_CA_BUNDLE`, and `SSL_CERT_FILE`, covering most TLS clients used in builds.
+
+### JVM builds
+
+JVM tools (Maven, Gradle, ...) don't read the environment variables above, so JVM support is opt-in via `CA_TRUST_JVM=1`. When it's set, the bundle is non-empty, and `keytool` is available, `container.sh` uses the container's own JDK to generate a PKCS#12 trust store from the bundle at `${CA_TRUST_DIR}/truststore.p12`, then appends the corresponding `-Djavax.net.ssl.trustStore*` options to `MAVEN_OPTS`, preserving any caller-supplied value (forward `MAVEN_OPTS` into the container, as in the quick start). A generation failure is an error. See [generators/java-pkcs12](generators/java-pkcs12/README.md) for details.
+
+## Persistence contract
+
+`CA_TRUST_DIR` must be a writable host bind-mount or tmpfs, not the container's writable overlay: a file on the overlay would be retained by `docker commit`, while a bind mount is not part of any committed image. `container.sh` verifies this with `findmnt` and refuses to write to the overlay; if `findmnt` is unavailable, it warns and proceeds. All staged and generated files live in the caller's staging directory and disappear when the caller cleans it up.
+
+## Extensibility
+
+Add a backend under `generators/` when a trust format can't consume the PEM bundle directly. Keep host discovery and lifecycle in `host.sh`; keep format-specific conversion in the backend, run in-container. See [generators/java-pkcs12](generators/java-pkcs12/README.md) as a template.

exports/docker/ca-trust/container.sh

@@ -0,0 +1,94 @@
+#!/usr/bin/env bash
+
+# Container-side configuration for CA trust. Source it after setting
+# CA_TRUST_DIR to a writable mount of the staged trust directory, which must
+# contain ca-bundle.pem. Set CA_TRUST_JVM=1 as well if the build runs on a JVM
+# (Maven, Gradle, ...) that needs its trust store configured: a Java PKCS#12
+# trust store is then generated here, inside the container, from the PEM
+# bundle using the container's own JDK (keytool) -- no separate generator
+# container or host JDK is required -- and written back to CA_TRUST_DIR
+# alongside the bundle.
+#
+# Persistence contract: CA_TRUST_DIR must be a writable host bind-mount (or
+# tmpfs), not the container's writable overlay. A file on the overlay is retained
+# by `docker commit`; a bind mount is not part of any committed image. This
+# script refuses to write to the overlay.
+
+if [[ -z "${CA_TRUST_DIR:-}" ]]; then
+    return 0 2>/dev/null || exit 0
+fi
+
+_ca_trust_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+# Spelled out rather than read from host.sh's CA_TRUST_BUNDLE_FILENAME: host.sh
+# is the host's half of the library and isn't guaranteed to be beside this file
+# in the container. Renaming the bundle means changing both.
+#
+# Internal to this file, like the other bare names here; the caller gets the
+# environment variables exported below, and all of these are unset at the end.
+HOST_CA_BUNDLE="${CA_TRUST_DIR}/ca-bundle.pem"
+
+if [[ -s "${HOST_CA_BUNDLE:-}" ]]; then
+    export CURL_CA_BUNDLE="${HOST_CA_BUNDLE}"
+    export GIT_SSL_CAINFO="${HOST_CA_BUNDLE}"
+    export PIP_CERT="${HOST_CA_BUNDLE}"
+    export REQUESTS_CA_BUNDLE="${HOST_CA_BUNDLE}"
+    export SSL_CERT_FILE="${HOST_CA_BUNDLE}"
+fi
+
+# Say why when an explicit opt-in does nothing. Silence here surfaces much later
+# as a PKIX path-building error inside the JVM build, far from the cause.
+if [[ -n "${CA_TRUST_JVM:-}" ]]; then
+    if [[ ! -s "${HOST_CA_BUNDLE:-}" ]]; then
+        echo >&2 "WARNING: CA_TRUST_JVM is set but ${HOST_CA_BUNDLE} is empty;" \
+            "skipping the JVM trust store."
+    elif ! command -v keytool &>/dev/null; then
+        echo >&2 "WARNING: CA_TRUST_JVM is set but keytool isn't on PATH;" \
+            "skipping the JVM trust store."
+    fi
+fi
+
+# Generate a Java PKCS#12 trust store in-container from the staged PEM bundle and
+# point Maven at it. Opt-in via CA_TRUST_JVM=1, since not every caller of this
+# library runs on a JVM. Also skipped when the bundle is empty or keytool is
+# unavailable, so CI builds without a trust directory and PEM-only staging
+# (empty bundle) are unaffected.
+if [[ -n "${CA_TRUST_JVM:-}" ]] && [[ -s "${HOST_CA_BUNDLE:-}" ]] && command -v keytool &>/dev/null; then
+    if ! mkdir -p "${CA_TRUST_DIR}"; then
+        echo >&2 "ERROR: cannot create Java trust store dir: ${CA_TRUST_DIR}"
+        return 1 2>/dev/null || exit 1
+    fi
+
+    # Refuse to write to the container's writable overlay: a file there is
+    # retained by `docker commit`, violating the no-persistence invariant. A
+    # bind mount or tmpfs has its own mount target; the root overlay resolves
+    # to "/". Warn (but proceed) if findmnt is unavailable to check.
+    if command -v findmnt &>/dev/null; then
+        _ca_trust_mount_target="$(findmnt -T "${CA_TRUST_DIR}" -o TARGET -n 2>/dev/null || true)"
+        if [[ -z "${_ca_trust_mount_target}" || "${_ca_trust_mount_target}" == "/" ]]; then
+            echo >&2 "ERROR: CA_TRUST_DIR (${CA_TRUST_DIR}) is on the container's writable overlay,"
+            echo >&2 "       which docker commit would retain. Mount a writable host directory or tmpfs there."
+            return 1 2>/dev/null || exit 1
+        fi
+    else
+        echo >&2 "WARNING: findmnt unavailable; cannot verify CA_TRUST_DIR is off the overlay."
+    fi
+
+    HOST_CA_JAVA_TRUST_STORE="${CA_TRUST_DIR}/truststore.p12"
+    if ! bash "${_ca_trust_dir}/generators/java-pkcs12/generate.sh" \
+            "${HOST_CA_BUNDLE}" "${HOST_CA_JAVA_TRUST_STORE}"; then
+        echo >&2 "ERROR: failed to generate Java PKCS#12 trust store from ${HOST_CA_BUNDLE}"
+        return 1 2>/dev/null || exit 1
+    fi
+
+    # Preserve any Maven options supplied by the caller.
+    _host_ca_maven_opts="${MAVEN_OPTS:-}"
+    [[ -n "${_host_ca_maven_opts}" ]] && _host_ca_maven_opts="${_host_ca_maven_opts} "
+    _host_ca_maven_opts="${_host_ca_maven_opts}-Djavax.net.ssl.trustStore=${HOST_CA_JAVA_TRUST_STORE}"
+    _host_ca_maven_opts="${_host_ca_maven_opts} -Djavax.net.ssl.trustStoreType=PKCS12"
+    # The store contains only public certificates; this is an integrity password, not a secret.
+    _host_ca_maven_opts="${_host_ca_maven_opts} -Djavax.net.ssl.trustStorePassword=changeit"
+    export MAVEN_OPTS="${_host_ca_maven_opts}"
+    unset _host_ca_maven_opts _ca_trust_mount_target
+fi
+
+unset _ca_trust_dir HOST_CA_BUNDLE HOST_CA_JAVA_TRUST_STORE

exports/docker/ca-trust/generators/java-pkcs12/README.md

@@ -0,0 +1,16 @@
+# Java PKCS#12 generator
+
+A `generators/` backend that produces a Java PKCS#12 trust store from a PEM CA bundle. Invoked by `container.sh` inside the build container; also runnable directly:
+
+```bash
+tools/yscope-dev-utils/exports/docker/ca-trust/generators/java-pkcs12/generate.sh \
+    <ca-bundle.pem> <truststore.p12>
+```
+
+It requires a JDK (`keytool` is located via `JAVA_HOME`, falling back to `PATH`); the build container already has one for the build, so no separate generator container or host JDK is needed. Given the inputs, it:
+
+1. Copies the JDK's base trust store (`jssecacerts` if present, else `cacerts`) into a new PKCS#12 store, keeping the standard public CA set alongside the bundle's CAs so downloads from hosts not behind the gateway still verify.
+2. Imports each certificate from the PEM bundle under a unique `host-ca-<n>` alias, splitting the bundle first since `keytool -importcert` reads only the first certificate of a multi-cert file. Certificates already present in the store are silently skipped.
+3. Writes the store to the output path with password `changeit` (an integrity password for public certificates, not a secret).
+
+`container.sh` points the JVM at the result via `-Djavax.net.ssl.trustStore*` options appended to `MAVEN_OPTS`, avoiding edits to the JDK's installed `cacerts`. The store is written to the caller-supplied output path -- for `container.sh`, inside `CA_TRUST_DIR`, a writable bind mount rather than the container's overlay -- so it never enters an image, cache, or artifact, and is removed when the caller cleans up the staging directory.

exports/docker/ca-trust/generators/java-pkcs12/generate.sh

@@ -0,0 +1,118 @@
+#!/usr/bin/env bash
+
+# Generates a Java PKCS#12 trust store from a PEM CA bundle, merging the
+# selected JDK's default certificates with the bundle's certificates.
+#
+# Runs inside the build container, which already provides a JDK (keytool +
+# cacerts); no separate generator container or host JDK is required.
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+if (( $# != 2 )) || [[ -z "$1" || -z "$2" ]]; then
+    echo >&2 "ERROR: generate.sh requires an input CA bundle and output path"
+    exit 2
+fi
+
+input_bundle="$1"
+output_trust_store="$2"
+if [[ ! -f "${input_bundle}" || ! -r "${input_bundle}" ]]; then
+    echo >&2 "ERROR: input CA bundle is not a readable regular file: ${input_bundle}"
+    exit 1
+fi
+output_dir="$(dirname "${output_trust_store}")"
+if [[ ! -d "${output_dir}" || ! -w "${output_dir}" ]]; then
+    echo >&2 "ERROR: output directory is not writable: ${output_dir}"
+    exit 1
+fi
+if [[ -e "${output_trust_store}" && ! -f "${output_trust_store}" ]]; then
+    echo >&2 "ERROR: output path is not a regular file: ${output_trust_store}"
+    exit 1
+fi
+
+# Integrity password for a store of public CA certificates; not a secret.
+readonly STOREPASS=changeit
+
+# Locate keytool and the JDK's default trust store. Match Java's trust-store
+# lookup order: jssecacerts overrides cacerts.
+java_home="${JAVA_HOME:-}"
+if [[ -n "${java_home}" ]]; then
+    keytool="${java_home}/bin/keytool"
+else
+    keytool="$(command -v keytool)" || {
+        echo >&2 "ERROR: keytool was not found in PATH and JAVA_HOME is unset"
+        exit 1
+    }
+    keytool="$(readlink -f "${keytool}")"
+    java_home="${keytool%/bin/keytool}"
+fi
+if [[ ! -x "${keytool}" ]]; then
+    echo >&2 "ERROR: keytool is not executable: ${keytool}"
+    exit 1
+fi
+
+java_security_dir="${java_home}/lib/security"
+base_java_trust_store="${java_security_dir}/cacerts"
+if [[ -f "${java_security_dir}/jssecacerts" && -s "${java_security_dir}/jssecacerts" ]]; then
+    base_java_trust_store="${java_security_dir}/jssecacerts"
+fi
+if [[ ! -f "${base_java_trust_store}" || ! -r "${base_java_trust_store}" \
+        || ! -s "${base_java_trust_store}" ]]; then
+    echo >&2 "ERROR: JDK default trust store is not readable: ${base_java_trust_store}"
+    exit 1
+fi
+
+# Append each certificate from the PEM bundle. keytool -importcert reads only
+# the first certificate from a multi-cert PEM file, so split the bundle into
+# per-cert buffers and import each under a unique alias.
+work_dir="$(mktemp -d)"
+trap 'rm -rf "${work_dir}"' EXIT
+
+# Start from a copy of the JDK's default trust store as PKCS#12. This keeps the
+# standard Mozilla CA set alongside the host's corporate CAs, so downloads to
+# public mirrors (not behind the corporate gateway) still verify. keytool prints
+# one progress line per entry to stderr; capture it so success is quiet but a
+# failure still surfaces the cause.
+if ! "${keytool}" -importkeystore -noprompt \
+        -srckeystore "${base_java_trust_store}" -srcstoretype JKS -srcstorepass "${STOREPASS}" \
+        -destkeystore "${output_trust_store}" -deststoretype PKCS12 -deststorepass "${STOREPASS}" \
+        >/dev/null 2>"${work_dir}/import.err"; then
+    echo >&2 "ERROR: keytool -importkeystore failed:"
+    cat >&2 "${work_dir}/import.err"
+    exit 1
+fi
+
+count=0
+cert_buf=""
+cert_file="${work_dir}/cert.pem"
+while IFS= read -r line || [[ -n "${line}" ]]; do
+    cert_buf+="${line}"$'\n'
+    if [[ "${line}" == "-----END CERTIFICATE-----" ]]; then
+        printf '%s' "${cert_buf}" > "${cert_file}"
+        # -noprompt skips the "trust this certificate?" prompt. A certificate
+        # already present under any alias is silently skipped by keytool, so
+        # duplicates in the bundle (or shared with cacerts) are harmless.
+        if ! "${keytool}" -importcert -noprompt \
+                -alias "host-ca-${count}" -file "${cert_file}" \
+                -keystore "${output_trust_store}" -storetype PKCS12 -storepass "${STOREPASS}" \
+                >/dev/null 2>"${work_dir}/import-cert.err"; then
+            echo >&2 "ERROR: failed to import certificate #${count} from bundle:"
+            cat >&2 "${work_dir}/import-cert.err"
+            exit 1
+        fi
+        count=$((count + 1))
+        cert_buf=""
+    fi
+done < "${input_bundle}"
+
+if (( count == 0 )); then
+    echo >&2 "ERROR: input bundle contains no complete PEM certificates: ${input_bundle}"
+    exit 1
+fi
+
+if [[ ! -s "${output_trust_store}" ]]; then
+    echo >&2 "ERROR: generated trust store is empty: ${output_trust_store}"
+    exit 1
+fi
+echo "==> Generated Java PKCS#12 trust store: ${output_trust_store} (${count} bundle certificate(s) processed)"