feat: register managed hooks via git config on git 2.54+

[ichoosetoaccept]

🧢 Changes

On git ≥ 2.54, GitButler registers its workspace-guard hooks through config-based hooks instead of writing into .git/hooks/:

• Guard scripts live in <git-common-dir>/gitbutler/hooks/, registered via hook.<name>.event / hook.<name>.command in local config.
• .git/hooks/ is never touched, so prek / pre-commit / husky / lefthook keep working unchanged — git runs config hooks first, then the hooks-directory hook.
• Installing on 2.54+ also cleans up hooks that older GitButler versions wrote into .git/hooks/ (signature-checked) and restores any displaced user hook from its backup. Installation re-runs on every workspace update, so after a git upgrade this migration happens on the user's next GitButler operation — no manual step. Until then the old file hooks simply keep working.
• git config gitbutler.installHooks false opts out on any git version, and but setup says so when it skips.
• but teardown now reports hooks it couldn't remove (hookWarnings in JSON) and exits 3 instead of claiming full success.

For git < 2.54 nothing changes by default: the file-based installation stays exactly as it is today (hooks run under whatever git the user invokes, so support is detected via git --version, not the library GitButler links). The only version-independent additions are the opt-out and teardown's honest reporting. Checking out away from gitbutler/workspace still cleans everything up (git switch and git checkout both fire the post-checkout hook — verified with both).

About two-thirds of the added lines are tests. They run against real repositories, including end-to-end with a real git ≥ 2.54: an actual git commit on the workspace branch is blocked, an actual git checkout away removes the config entries and scripts (self-skipping on older CI git).

What I validated personally, with a release build of this branch on macOS:

• Disposable repo with a local bare remote and a pre-commit hook installed by an actual prek install: but setup, inspected the config entries and both hook directories, git commit blocked with GITBUTLER_ERROR (prek's own hook still ran and passed alongside it), git checkout main removed the config entries and scripts — then the same staged change committed successfully through prek. The prek hook's checksum was identical before, during, and after.
• Second repo, exiting via but branch new + but teardown instead of the checkout: same clean end state, same untouched checksum. (One caveat: but teardown straight after but setup, with no branch created, stops at "Finding active branch to check out" before reaching the hooks — pre-existing behavior, --checkout-to gets past it and hook cleanup is then complete.)
• First validated on git 2.54.0; after Homebrew moved this machine to 2.55.0 I re-ran both scenarios and the automated suite against it — everything still passes
• The pre-2.54 fallback and the upgrade migration, in a Debian container (reproduction script in a comment below): with git 2.53.0 but setup installs today's file-based hooks — user hook displaced to pre-commit-user but still chained and running, commit blocked; after switching to git 2.55.0 the next install registers the config hooks, sweeps the signed file hooks, and restores the user hook byte-identical; checkout away then removes everything. 18 scripted checks, all green.

The core of the manual run:

$ but setup
$ git config --local --get-regexp '^hook\.'
hook.gitbutler-pre-commit.event pre-commit
hook.gitbutler-pre-commit.command '.../.git/gitbutler/hooks/pre-commit'
...
$ git commit -m test
GITBUTLER_ERROR: Cannot commit directly to gitbutler/workspace branch.
$ git checkout main
NOTE: You have left GitButler's managed workspace branch.
$ git config --local --get-regexp '^hook\.'   # exits 1 — nothing left

☕️ Reasoning

This is the surgical alternative discussed in #12748 after @Alxandr pointed out git 2.54's config-based hooks. The ownership-model PR (#12980) made sharing .git/hooks/ slots safe; config-based hooks make the slots uncontested by construction, so nearly all of that machinery becomes unnecessary — "nearly", because a small signature-checked remnant survives as the migration sweep that removes hooks older GitButler versions wrote and restores displaced user hooks. None of the rest has to be merged only to be removed as git ≥ 2.54 spreads.

The biggest structural win: the old approach needed adapter code per hook manager — a detection signature, config matching, and integration instructions for prek first, then again for pre-commit, husky, lefthook, hk, and whatever comes next. Here that whole axis disappears. GitButler never inspects or even notices the other tool, so every hook manager — including ones that don't exist yet — is supported with zero code. This honestly is my favorite part of this new PR compared to the old one. A close second favorite is that this PR contains drastically less code and attempts to contribute something that should hold for a long time, very much unlike the first attempt which would have added maintenance burden to the project, no matter how well-meaning.

Git 2.54 shipped in April 2026, so package-manager gits (Homebrew, winget, Arch, Fedora…) are already past it — my machine is on 2.55 — while slower channels (Apple's Command Line Tools, Debian/Ubuntu LTS) will lag for a while. That's fine: users below 2.54 keep today's file-based behavior and lose nothing, and the share on the config-based path only grows from here.

Two review notes:

• This landed bigger than the couple-hundred lines you mused about — a good part of the implementation is the hook scripts themselves, and another chunk is but teardown learning to report hooks it couldn't remove instead of claiming success. Happy to split that teardown part into its own PR if you'd rather keep this one minimal.
• The new integration tests keep the test_ prefix you asked to avoid on feat: hook ownership model with external manager detection #12980: they extend the existing managed_hooks_tests.rs, where all 21 existing tests use it, and matching the file seemed better than mixing conventions or renaming untouched tests. Glad to rename if you prefer.

🎫 Affected issues

Fixes: #12748

[ichoosetoaccept]

To validate the pre-2.54 fallback and the upgrade migration end-to-end, I ran the
script below in a Debian container: it builds git 2.53.0 and 2.55.0 from source
(no package archive carries both), builds but from this branch, and walks one
repo through the full journey — file-based hooks under 2.53 (user hook displaced
but chained, commit blocked), the migration to config-based hooks after switching
to 2.55 (signed file hooks swept, user hook restored byte-identical), and the
checkout-away cleanup. 18 checks, all green on my machine.

To reproduce, from a checkout of this branch:

docker run --rm \
  -v "$PWD":/src:ro \
  -v "$PWD"/validate-migration.sh:/test.sh:ro \
  -v but-build-cache:/build -v but-registry-cache:/usr/local/cargo/registry \
  rust:slim-bookworm bash /test.sh

(The two named volumes are optional but make re-runs incremental).

validate-migration.sh (80 lines)

#!/usr/bin/env bash
# Runs INSIDE the container. Validates pre-2.54 file-based behavior and the 2.53→2.55 migration.
set -uo pipefail
say() { printf '\n===== %s =====\n' "$*"; }
ok()  { echo "OK   $*"; }
bad() { echo "FAIL $*"; FAILURES=$((FAILURES+1)); }
FAILURES=0

say "install build deps"
apt-get update -qq >/dev/null
apt-get install -y -qq build-essential curl ca-certificates pkg-config libssl-dev zlib1g-dev cmake perl libdbus-1-dev >/dev/null 2>&1

say "build git 2.53.0 and 2.55.0 from source"
for v in 2.53.0 2.55.0; do
  curl -sL "https://mirrors.edge.kernel.org/pub/software/scm/git/git-$v.tar.gz" | tar xz -C /tmp
  make -C /tmp/git-$v -j"$(nproc)" prefix=/opt/git-${v%.0} NO_TCLTK=1 NO_GETTEXT=1 NO_CURL=1 NO_EXPAT=1 NO_PERL=1 install >/dev/null 2>&1 \
    && ok "built git $v" || bad "building git $v"
done
/opt/git-2.53/bin/git version && /opt/git-2.55/bin/git version

say "build but from the branch (debug)"
export CARGO_TARGET_DIR=/build
export PATH=/opt/git-2.55/bin:$PATH   # build scripts may invoke git
(cd /src && cargo build -p but >/tmp/build.log 2>&1) || { bad "cargo build"; tail -40 /tmp/build.log; }
BUT=/build/debug/but
[ -x "$BUT" ] && ok "but binary built" || { bad "no but binary"; exit 1; }

say "create test repo with a user pre-commit hook"
export E2E_TEST_APP_DATA_DIR=/tmp/appdata NOPAGER=1
mkdir -p /tmp/appdata
export PATH=/opt/git-2.53/bin:$PATH
git init -q --bare /tmp/origin.git
git init -q -b main /tmp/repo
cd /tmp/repo
git config user.email t@e.c && git config user.name t && git config commit.gpgsign false
printf '#!/bin/sh\necho USER-HOOK-RAN\nexit 0\n' > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
SHA_BEFORE=$(sha1sum .git/hooks/pre-commit | cut -d' ' -f1)
echo hi > f && git add f && git commit -qm init
git remote add origin /tmp/origin.git && git push -qu origin main 2>/dev/null

say "PHASE 1: git 2.53 — expect FILE-BASED hooks, user hook displaced+chained"
git version
"$BUT" setup >/tmp/setup1.log 2>&1 || { bad "but setup (2.53)"; tail -5 /tmp/setup1.log; }
if git config --local --get-regexp '^hook\.' >/dev/null 2>&1; then bad "config entries present on 2.53"; else ok "no config entries on 2.53"; fi
grep -q GITBUTLER_MANAGED_HOOK .git/hooks/pre-commit && ok "managed file hook installed" || bad "no managed file hook"
[ -f .git/hooks/pre-commit-user ] && ok "user hook displaced to pre-commit-user" || bad "no -user backup"
[ -f .git/hooks/post-checkout ] && grep -q GITBUTLER_MANAGED_HOOK .git/hooks/post-checkout && ok "managed post-checkout installed" || bad "no managed post-checkout"
echo x >> f && git add f
OUT=$(git commit -m nope 2>&1); RC=$?
echo "$OUT" | grep -q USER-HOOK-RAN && ok "displaced user hook still chained+ran" || bad "user hook did not run"
echo "$OUT" | grep -q GITBUTLER_ERROR && [ $RC -ne 0 ] && ok "commit blocked on 2.53" || bad "commit NOT blocked on 2.53 (rc=$RC)"

say "PHASE 2: upgrade to git 2.55 — expect config hooks + migration sweep"
export PATH=/opt/git-2.55/bin:$PATH
git version
"$BUT" setup >/tmp/setup2.log 2>&1 || { bad "but setup (2.55)"; tail -5 /tmp/setup2.log; }
N=$(git config --local --get-regexp '^hook\.' | wc -l)
[ "$N" -eq 4 ] && ok "4 config entries registered" || { bad "expected 4 config entries, got $N"; git config --local --get-regexp '^hook\.'; }
[ -d .git/gitbutler/hooks ] && ok "scripts dir created" || bad "no scripts dir"
SHA_AFTER=$(sha1sum .git/hooks/pre-commit | cut -d' ' -f1)
[ "$SHA_BEFORE" = "$SHA_AFTER" ] && ok "user hook restored byte-identical" || bad "user hook differs after migration"
[ ! -e .git/hooks/pre-commit-user ] && ok "-user backup consumed" || bad "-user backup still present"
[ ! -e .git/hooks/post-checkout ] && ok "legacy post-checkout swept" || bad "legacy post-checkout still present"
OUT=$(git commit -m nope2 2>&1); RC=$?
echo "$OUT" | grep -q GITBUTLER_ERROR && [ $RC -ne 0 ] && ok "commit blocked on 2.55 via config hook" || bad "commit NOT blocked on 2.55 (rc=$RC)"
echo "$OUT" | grep -q USER-HOOK-RAN && ok "user hook ran from hooks dir" || bad "user hook did not run on 2.55"

say "PHASE 3: checkout away — expect full cleanup, user hook intact"
git checkout main 2>&1 | tail -2
if git config --local --get-regexp '^hook\.' >/dev/null 2>&1; then bad "config entries survived checkout"; else ok "config entries removed"; fi
[ ! -e .git/gitbutler/hooks ] && ok "scripts dir removed" || bad "scripts dir survived"
SHA_FINAL=$(sha1sum .git/hooks/pre-commit | cut -d' ' -f1)
[ "$SHA_BEFORE" = "$SHA_FINAL" ] && ok "user hook still byte-identical" || bad "user hook mutated"
OUT=$(git commit -m works 2>&1); RC=$?
[ $RC -eq 0 ] && echo "$OUT" | grep -q USER-HOOK-RAN && ok "normal commit works again through user hook" || bad "post-cleanup commit failed (rc=$RC)"

say "RESULT"
[ "$FAILURES" -eq 0 ] && echo "ALL CHECKS PASSED" || echo "$FAILURES CHECK(S) FAILED"
exit "$FAILURES"

[Byron]

Thanks so much for making this happen! As opposed to the previous PR, this one seems 'small' enough to review with options to split the teardown portion into its own PR.

As this affects the CLI which I am not very familiar with, I think it should be reviewed (or at least taken-over) by someone who is. So let me reel in @slarse to see what he thinks this PR can go.

Thanks everyone!