feat(cloudflare-warp): declarative Zero Trust enrollment with per-host service modes - #448
feat(cloudflare-warp): declarative Zero Trust enrollment with per-host service modes#448Bad3r wants to merge 60 commits into
Conversation
The shared baseline switched every shareCommon host to warp-svc at mkOverride 1100. Zero Trust enrollment is a per-host decision: system76 runs Full "Gateway with WARP" while tpnix runs tunnel-only to keep its NetworkManager dnsmasq private-host mappings, and a host without secrets/cloudflare-warp.yaml only reaches a degraded un-enrolled daemon. Default the app off so modules/system76/cloudflare-warp.nix and modules/tpnix/cloudflare-warp.nix own enablement explicitly.
…tials The module only placed warp-cli on PATH, so enrolling a device meant running `warp-cli register` by hand and the Zero Trust team name landed in shell history. Drive upstream services.cloudflare-warp instead and render /var/lib/cloudflare-warp/mdm.xml from a sops template, so organization, auth_client_id, and auth_client_secret reach the daemon without entering the Nix store. mdm.xml is authoritative for service_mode, so serviceMode/autoConnect/ switchLocked are exposed as options and `warp-cli mode` is never called. Lifecycle details that are not obvious from the diff: - templates."cloudflare-warp-mdm".restartUnits restarts cloudflare-warp.service on credential rotation, because restartTriggers hashes only the non-secret fields and would otherwise leave warp-svc on the stale service token until reboot (same pattern as usbguard.nix and duplicati-r2.nix). - The un-enrolled branch adds an ExecStartPre `rm -f` for mdm.xml: the file caches the service token and pins service_mode, so a leftover copy would keep the daemon in managed mode instead of degrading. - after/requires use flake.lib.security.sopsInstallSecretsDeps, which resolves to [] on activation-script hosts because sops-install-secrets.service only exists under sops.useSystemdActivation (issue #37). Warnings cover the three ways the setup silently misbehaves: a local resolver on 127.0.0.1:53 fighting Gateway DNS, a missing secrets/cloudflare-warp.yaml running warp-svc un-enrolled, and strict checkReversePath dropping the asymmetric return traffic on the CloudflareWARP interface. Validation: nix-instantiate --parse; nix fmt; nix eval path:.#nixosConfigurations.system76.config.programs.cloudflare-warp.extended.serviceMode
…guard warp-svc starts idle: mdm.xml auto_connect only governs reconnect after a manual disconnect, so a freshly booted enrolled device stays outside the tunnel until someone runs `warp-cli connect`. Add a best-effort oneshot that performs that connect, gated so it can never select consumer WARP. The guard is the point of the unit, not the connect. `warp-cli connect` on an un-enrolled daemon registers the device against Cloudflare's consumer service, which is a different security posture than Zero Trust. So the script: - exits early when no managed mdm.xml is possible (secrets/cloudflare-warp.yaml absent); - compares `warp-cli registration organization` against the sops-provisioned organization before every connect attempt, and refuses to connect when they do not match; - disconnects a tunnel that reached Connected without a confirmed managed registration. Bounds: each IPC call gets `timeout 5s`, the retry loop stops at 30 attempts or a 120s deadline, and TimeoutStartSec=180 covers the polling plus shell overhead, because warp-svc's IPC socket and mdm.xml registration settle at different times. The unit exits 0 after logging the outcome so a user who deliberately keeps WARP disconnected does not boot into a failed unit. Validation: nix-instantiate --parse; nix fmt (0 changed)
services.system76 set services.cloudflare-warp directly, which runs warp-svc but never writes mdm.xml, so the host sat un-enrolled and any Zero Trust posture had to be established by hand. Move enablement to programs.cloudflare-warp.extended so the host picks up managed enrollment, the connect-on-boot guard, and the mdm.xml lifecycle. serviceMode "warp" is Full / Gateway with WARP: system76 has no local resolver bound to 127.0.0.1:53, so Gateway DNS filtering does not collide the way it would on tpnix. autoConnect stays 0 so a manual disconnect sticks, and switchLocked stays false so the tunnel remains user-controllable. The raw services.cloudflare-warp block is removed rather than left alongside: the wrapper sets the same options plus package/udpPort/openFirewall, and a second definition of services.cloudflare-warp.package would conflict. Validation: nix eval path:.#nixosConfigurations.system76.config.programs.cloudflare-warp.extended.serviceMode
tpnix runs NetworkManager with dns = "dnsmasq" for its private-host mappings, so serviceMode "warp" would hand DNS to Gateway and break those mappings (the module warns about exactly this collision). Use "tunnelonly" instead: WARP still provides the tunnel, HTTP filtering, network policies, and posture checks, and Gateway DNS is left off deliberately. enable is gated on flake.lib.nixos.hosts.tpnix.sopsRuntimeReady, matching the other tpnix sops consumers (duplicati.nix, printing.nix, fonts.nix). The flag is true since PR #305, so the gate is not what keeps the host un-enrolled; it is a kill switch. If tpnix loses its runtime decryption key, flipping the flag drops the WARP stack with it, removes the rendered mdm.xml on the next activation, and keeps the sops.secrets."cloudflare-warp/*" declarations and the cloudflare-warp-mdm template from failing activation on an un-decryptable payload. Validation: nix eval path:.#nixosConfigurations.tpnix.config.programs.cloudflare-warp.extended.serviceMode
The module picks its branch on `builtins.pathExists (secretsRoot + "/cloudflare-warp.yaml")`. CI has no secrets submodule, so host evaluation only ever exercises the un-enrolled path, and a regression in the mdm template, the secret-backed ExecStartPre, or the connect script's registration guard would reach main unnoticed. Add checks."apps/cloudflare-warp-module-eval", which evaluates the module twice against an in-repo non-secret fixture (cloudflare-warp-check-fixtures/cloudflare-warp.yaml) to force the managed branch, and against a missing path to keep the un-enrolled assertions in the same check. The managed side is deep-forced so the sops template content, the install ExecStartPre, and the connect script text are all evaluated rather than left as thunks. The fixture holds placeholder values only and is not sops-encrypted: the check needs the path to exist and the three keys to resolve, not real credentials. Validation: nix-instantiate --parse
Managed enrollment has prerequisites that live outside this repository (a Zero Trust team name, a service token with the right scope, and a device-enrollment policy that accepts it), and getting them wrong fails at activation or leaves the device silently un-enrolled. Document the dashboard steps, the sops payload shape, and the failure modes. New pages under docs/cloudflare/warp/: deployment (dashboard prerequisites and the secrets/cloudflare-warp.yaml payload), modes (why system76 runs "warp" and tpnix runs "tunnelonly", and which modes collide with a local resolver), operations (verifying registration, reading cloudflare-warp-connect logs, recovering a stale mdm.xml), reference (option-to-mdm.xml key mapping), and cheatsheet (warp-cli commands). Linked from docs/cloudflare/README.md and docs/index.md so they are reachable from the existing docs tree rather than a new surface. Validation: nix develop -c pre-commit run --all-files --hook-stage manual (markdown hooks)
…match warp-cli 2026.3.846.0 answers `registration organization` with exit 0 and an empty string when the device carries no Teams registration, so a real mismatch is distinguishable from a check that never answered. The connect oneshot collapsed both into an empty managed_registration, so one `timeout 5s` firing on a busy warp-svc made refresh_status run `warp-cli disconnect` against a healthy managed tunnel, and the connect gate then refused to bring it back inside the same run. registration_state now carries confirmed/mismatch/unknown and only mismatch disconnects; unknown leaves the tunnel up and retries. Two cost fixes in the same loop: the second per-attempt refresh_registration is gone because `warp-cli connect` does not change the registration organization, and an empty managed_org, read once at startup, now exits after the first status report instead of spinning ~20 no-op attempts until the 120-second deadline. Validation: nix fmt; nix build path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"; nix flake check path:. --accept-flake-config --no-build --offline; bash -n and shellcheck on the rendered enrolled and un-enrolled connect scripts.
|
Claude finished @Bad3r's task in 5m 7s —— View job Review
Two findings, both in the connect script's registration state machine. Posted inline.
Checked and not reported: the |
There was a problem hiding this comment.
The PR introduces evaluation errors due to non-existent options and contains a logic bug in the connection script that causes unintended disconnections.
-
File:
modules/apps/cloudflare-warp.nix
services.cloudflare-warp.rootDirandservices.cloudflare-warp.openFirewallare accessed/set in theconfigblock but are not standard options in the upstream NixOScloudflare-warpmodule. This will cause evaluation to fail when the module is enabled.
Fix: HardcoderootDirto/var/lib/cloudflare-warp(the standardStateDirectoryfor this service) and removeopenFirewallfrom theservices.cloudflare-warpattribute set, instead handling the firewall rule vianetworking.firewall.allowedUDPPorts = lib.mkIf cfg.openFirewall [ cfg.udpPort ];. -
File:
modules/apps/cloudflare-warp.nix
In thecloudflare-warp-connectscript (Patch 3), if thewarp-cli registration organizationcommand fails or times out, themanaged_registrationvariable remains empty. This causesrefresh_statusto evaluate the device as having a "mismatched" registration and executewarp-cli disconnect, potentially killing a healthy, correctly enrolled tunnel.
Fix: Implement the "unknown" state logic alluded to in Patch 8 to skip disconnection unless a mismatch is explicitly confirmed. -
Truncated Diff: Patch 8
The implementation for Patch 8 ("disconnect only on a confirmed registration mismatch") is missing from the provided unified diff, despite being listed in the subject lines.
Fix: Provide the complete diff including the implementation for Patch 8.
The connect oneshot could not recover from a warp-svc crash: PartOf= propagates only explicit restart jobs, so upstream's Restart=always respawn left the Type=oneshot RemainAfterExit=true unit active and the host untunneled until the next rebuild. BindsTo= plus Upholds= on cloudflare-warp.service now stop and re-run it. Other defects in the same unit: * refresh_status left `connected` empty when the tunnel was up but `registration organization` never answered, so the loop spent all 30 attempts and the epilogue logged "connect never succeeded (daemon unreachable ...)" for a live tunnel. That state is terminal after three observations, and the epilogue now names the state that ended the run (unverified registration, wrong organization, refused connect, unreachable daemon). * refresh_registration captured the check with 2>&1 and compared it to the secret with string equality, so a banner on stderr would classify a correctly enrolled device as "mismatch" and disconnect it. It reads stdout alone, strips whitespace from both sides, and reports the exit status on failure. * refresh_registration ran twice per boot with only read-only calls in between. * enableStrictShellChecks = true routes the script through writeShellApplication, so ShellCheck 0.11.0 and errexit/nounset/pipefail now cover a fail-closed security guard that nixpkgs otherwise builds unchecked. Without secrets/cloudflare-warp.yaml the wrapper still started warp-svc: root, CAP_NET_ADMIN, an open UDP port, and no mdm.xml, which serves only consumer WARP. enable now installs warp-cli alone in that state and declares neither the daemon nor the connect oneshot. restartTriggers on cloudflare-warp.service duplicated the mdm template's restartUnits: sops compares the rendered template between generations, so both already fired on a serviceMode change, and under sops.useSystemdActivation the two mechanisms restart warp-svc twice per activation. The template is the only restart owner. An xmllint --noout ExecStartPre parses the rendered fragment before installing it, so a credential carrying an XML metacharacter fails the unit instead of silently degrading warp-svc to unmanaged mode. The flake check asserts package/enable/udpPort/openFirewall forwarding (headless selection lost its only test when modules/system76/services.nix stopped setting it directly), the restart owner, the unit wiring, the shellcheck gate, and the un-enrolled shape. Two splitString ordering assertions lacked the `lib.length parts > 1` guard and degraded to always-passing whole-script searches when a marker moved. The lookup uses nixosAppHelpers.getApp instead of a hand-rolled tree walker; config.flake.lib.nixos.getApp is unusable here because the same helper read back through the flake.lib option merge returns a wrapper whose functionArgs are empty, so the module system cannot inject pkgs. Validation: nix fmt; nix flake check path:. --accept-flake-config --no-build --offline; built unit-script-cloudflare-warp-connect-start (ShellCheck 0.11.0 passes); ran that script against a stub warp-cli across eight states (verified, unverifiable, wrong org, noisy stderr, whitespace org, refused connect, dead daemon, empty secret); rendered both units to confirm BindsTo/Upholds and the absence of X-Restart-Triggers; nix develop path:. -c pre-commit run --files <changed>.
The enrolled fixture only exercised openFirewall = false, so a hardcoded false in the production forwarding could pass while managed hosts use the default UDP 2408 rule. Assert the true default of mdmVariant and its allowedUDPPorts entry. Validation: nix eval path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval".name --raw; temporary production hardcode rejected with openFirewall = true must open udpPort; nix build path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"; nix develop path:. -c pre-commit run --files modules/apps/cloudflare-warp/module-check.nix.
Review resolutionImplemented:
Rejected with technical justification:
Decisions:
Consequential coverage and validation:
These changes keep lifecycle enforcement fail-closed on conclusive foreign enrollment while preserving the bounded readiness path for non-conclusive results. The paired firewall checks make both production forwarding arms observable before a host rebuild. |
Both DNS warning positives used serviceMode = "warp", so removing the mode predicate left tpnix tunnelonly behavior without a test and could emit a self-contradictory warning. Force a local resolver under tunnelonly and 1dot1. Validation: nix eval and nix build path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"; temporary removal of the mode gate and of the 1dot1 member each failed the target evaluation; nix flake check path:. --accept-flake-config --no-build --offline; scoped pre-commit hooks.
The operations prose still gave bare registration and status commands, although cloudflare-warp 2026.3.846.0 refuses a bare call at the ToS prompt. Align the operator instruction with the source and verification block. Validation: executable WARP command scan found no bare command; nix develop path:. -c lychee --no-progress --include-fragments docs/cloudflare/warp/operations.md; scoped pre-commit hooks; git diff --check.
Automated review round resolvedImplemented:
Rejected with technical justification:
Decisions:
Consequential reliability work:
Validation:
|
managed_org is read and stripped before the startup gate. That gate reports status and exits on an empty value before the retry loop reaches its only refresh_registration call, so the inner branch and registration_state writer cannot execute. Keep the causal cannot verify registration diagnostic: it names an unreadable or whitespace-only secret while the later line names the terminal no-connect action. Validation: nix-instantiate --parse modules/apps/cloudflare-warp.nix; targeted formatter; nix eval and nix build path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"; scoped pre-commit hooks; git diff --check.
Automated review round resolvedImplemented:
Rejected with technical justification:
Decisions:
Consequential consistency work:
Reliability:
Validation:
All open inline threads are resolved. |
A retained mismatch becomes unknown after a failed registration probe, but the ordinary unverified counter ended the run after three connected observations even after cleanup failed. A fresh successful registration result was then never reached to retry the existing mismatch cleanup. Keep the live state unknown and preserve the retained classification only as a bounded retry reason. It neither disconnects nor accepts the tunnel, while the existing 30-attempt and 120-second limits still end the run. Validation: parsed source and module check; targeted formatter; evaluated and built path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"; negative removal of the retained branch failed the evaluator; scoped pre-commit hooks; operations link check; git diff --check.
A failed registration query after a managed confirmation could leave a refused connect without another retry. With autoConnect = 0, the daemon then remained disconnected until a manual restart despite enrollment having been confirmed in the same run. Reuse confirmed_once only while mismatch_kind is clear. A later successful mismatch still blocks a retry, preserving the fail-closed consumer-WARP guard. Validation: nix-instantiate --parse modules/apps/cloudflare-warp.nix; nix-instantiate --parse modules/apps/cloudflare-warp/module-check.nix; targeted nix formatter; nix eval and nix build of apps/cloudflare-warp-module-eval; negative evaluator mutation removing mismatch_kind guard; scoped pre-commit hooks.
Cloudflare moved its service-token documentation from the service-auth path to service-credentials. The former reference returned HTTP 404, leaving the managed-enrollment README without a working credential reference. Validation: targeted nix formatter; lychee --no-progress --include-fragments docs/cloudflare/warp/README.md docs/cloudflare/warp/operations.md docs/cloudflare/warp/reference.md; curl verification of the replacement URL returned HTTP 200.
managed_org is rejected before the retry loop, but equality in refresh_registration could still classify two empty values as confirmed if a later control-flow change bypassed that exit. confirmed_once would then authorize consumer WARP. Require a nonempty managed organization at the confirmation producer. The existing startup diagnostic and exit remain in place; this only closes the future fail-open edge. Validation: nix-instantiate --parse modules/apps/cloudflare-warp.nix; nix-instantiate --parse modules/apps/cloudflare-warp/module-check.nix; targeted nix formatter; nix eval and nix build of apps/cloudflare-warp-module-eval; negative evaluator mutation removing the nonempty guard; scoped pre-commit hooks.
Automated review round resolvedImplemented
Rejected with technical justification
Decisions
Consequential consistency work
ReliabilityThe state machine now preserves bounded recovery after a failed cleanup without allowing historical mismatch state to disconnect a potentially re-enrolled device. It also avoids abandoning a valid same-run retry after transient daemon IPC failure, while a successful mismatch still blocks stale acceptance and connection attempts. Local nonempty confirmation makes consumer-WARP authorization fail closed even if future control flow changes the startup gate. Validation
All three newly addressed inline threads were replied to and resolved before push. |
Cloudflare retired the WARP diagnostic-log page. Its previous URL redirects to a 404, so the cheatsheet now links to the verified WARP client documentation root instead of promising a removed page. Validation: targeted nix formatter; lychee --no-progress --include-fragments across every WARP documentation page; git diff --check.
Documentation completeness fixImplemented
Rejected with technical justification
Decisions
Consequential consistency work
ReliabilityOperator diagnostics instructions now point to a live vendor reference instead of a dead page, so the recovery path remains usable when Validation
|
A successful empty registration response after a prior confirmation was treated like an unanswered query. The confirmed_once shortcut then left held_empty open past its three-answer window and could accept or retry an unverified tunnel. Count every successful empty response against the readiness window and reset it only on a fresh managed confirmation. Reuse confirmed_once after an unanswered query only when held_empty and mismatch_kind are clear. Validation: nix-instantiate --parse on the module and evaluator; targeted formatter; apps/cloudflare-warp-module-eval evaluation and build; focused negative mutations; strict shell state model; scoped pre-commit; Lychee; nix flake check path:. --accept-flake-config --no-build --offline.
|
Review resolution for #448 Implemented in
Rejected with technical justification: none. Decisions requested: none. The distinction between an unanswered probe and a successful empty response follows the existing bounded readiness-window and fail-closed mismatch contract. Consequential fixes: added a confirmation-time counter reset and exact negative evaluator coverage for the empty producer, both held-empty guards, and the reset. A strict shell state model covers settling, fourth-empty mismatch, timeout-only reuse, held-empty timeout preservation, retained mismatch, and reset behavior. Reliability: successful empty responses can no longer be reported as verified or trigger reconnects from stale confirmation. Failed probes remain non-destructive evidence, retained mismatches remain fail-closed, and fresh confirmation alone resolves the readiness hold. Validation passed for Nix parse, targeted evaluator and build, formatter, focused mutations, scoped hooks, WARP documentation links, and the offline flake check. |
Review resolutionImplemented
Rejected with technical justification
Decisions
Consequential consistency checks
Reliability resultThe bounded readiness window tolerates daemon settling without treating a successful empty registration result as permission to select consumer WARP. A fresh managed confirmation is the only way to reopen active connection authorization after that state, while a persistent empty result follows the existing mismatch cleanup path. |
Summary
Turns
cloudflare-warpfrom a package-on-PATH into a declarative Cloudflare Zero Trustenrollment, and moves enablement from the shared baseline to per-host modules.
Before this change the app module only put
warp-clion PATH, so enrolling a device meantrunning
warp-cli registerby hand and the Zero Trust team name landed in shell history.The shared baseline also switched every
shareCommonhost towarp-svcatmkOverride 1100, even though service mode is a per-host decision.Managed enrollment (
modules/apps/cloudflare-warp.nix)services.cloudflare-warpand renders/var/lib/cloudflare-warp/mdm.xmlfrom a sops template, so
organization,auth_client_id, andauth_client_secretreachwarp-svcwithout entering the Nix store.mdm.xmlis authoritative forservice_mode, soserviceMode,autoConnect, andswitchLockedare exposed as options andwarp-cli modeis never called.builtins.pathExists (secretsRoot + "/cloudflare-warp.yaml"). Withoutthe secret the host installs
warp-clialone and declares neitherwarp-svcnor theconnect oneshot: an unmanaged daemon holds
CAP_NET_ADMINand an open UDP port whileserving only consumer WARP. A tmpfiles rule clears any stale
mdm.xml, which wouldotherwise pin the old service mode and cached service token.
templates."cloudflare-warp-mdm".restartUnitsis the sole restart owner forcloudflare-warp.service. sops compares the rendered template between generations, so arotated credential and a changed
serviceMode/autoConnect/switchLockedboth restartwarp-svc. A secondrestartTriggershash on the unit would restart it twice peractivation on hosts running
sops.useSystemdActivation.ExecStartPreparses the rendered fragment withxmllint --nooutbefore installing it.Credentials are substituted after evaluation, so no Nix-side quoting can escape them; an
XML metacharacter in one fails the unit instead of degrading
warp-svcto unmanaged mode.xmllint's stderr is discarded and replaced with a fixed message, because it reports a parse
error by echoing the offending source line, which is the credential, and neither this module
nor upstream sets
StandardError.after/requiresuseflake.lib.security.sopsInstallSecretsDeps, which resolves to[]on activation-script hosts because
sops-install-secrets.serviceonly exists undersops.useSystemdActivation(issue bug: Services fail with 'Unit sops-install-secrets.service not found' #37).on
127.0.0.1:53fighting Gateway DNS, a missingsecrets/cloudflare-warp.yamlleaving thehost with the CLI and no daemon, and strict
checkReversePathdropping asymmetric returntraffic on the
CloudflareWARPinterface.services.dnscrypt-proxy.enabletriggers the resolverwarning independently. Its NetworkManager dnsmasq branch requires
networking.networkmanager.enablealongsidedns = "dnsmasq", since that option is declaredunconditionally but only reaches
NetworkManager.confundermkIf enable.switchLockeddeliberately remains documentation-only: it is a Cloudflare policy choice that legitimately
combines with
connectOnBoot, but it makes mismatch teardown report-only. The module summary,option description, reference row, and cheatsheet record that tradeoff.
Connect-on-boot guard
warp-svcstarts idle (auto_connectonly governs reconnect after a manual disconnect), soan enrolled device stays outside the tunnel until someone connects. A best-effort oneshot
performs that connect, gated so it can never select consumer WARP:
warp-cli connecton anun-enrolled daemon registers against Cloudflare's consumer service, a different security
posture than Zero Trust. The first
warp-cli connectrequireswarp-cli registration organizationto match thesops-provisioned organization. A later unanswered response may reuse that same-run confirmation only while no successful empty response is
held and no later successful response has recorded a mismatch. A successful empty response opens a bounded readiness
hold, so it cannot verify the tunnel or authorize another connect request. The script
disconnects a tunnel that reached
Connectedwithout a confirmed managed registration.registration_stateis tri-state (confirmed/mismatch/unknown) becausewarp-cli 2026.3.846.0 answers
registration organizationwith exit 0 and an empty string whenthe device carries no Teams registration. Collapsing the two would let one
timeout 5sfiringon a busy
warp-svcdisconnect a healthy managed tunnel, so onlymismatchdisconnects whileunknownleaves the tunnel up. A successful empty answer opens a bounded readiness hold only while no conclusive mismatch is retained. A fresh managed confirmation resets its three-answer window.A later failed query cannot cancel that hold, because it has no organization result to resolve the
state. This keeps an interleaved timeout from consuming the unverified budget before a fourth empty
answer reaches
mismatchand tears down consumer WARP. A retained mismatch instead keeps a later empty answer on the existing mismatch path, so it cannotsuppress enforcement of a foreign tunnel. After a failed cleanup, it also preserves the remaining
attempt and deadline budget for a fresh successful response to retry cleanup, without treating a
failed registration query itself as disconnect evidence. Command substitution captures stdout alone and
the check strips
whitespace from both sides, so a banner on stderr cannot turn a correctly enrolled device into a
mismatchwithout a redirect suppressing it; stderr is therefore left to reach the journal aswarp-cli's own reason for a failed check. An
empty managed organization, read once at startup, exits after the first status report instead
of spinning until the deadline. The confirmation branch also requires that configured organization
to be nonempty, so a future control-flow refactor cannot classify two empty strings as
confirmedand authorize consumer WARP.
The terminal log line normally names the state the run ended on. A conclusive mismatch remains
the exception: a later unanswered registration check cannot prove it resolved, so the terminal
diagnostic retains only its empty-or-foreign classification until a successful managed confirmation
clears it. This retained classification cannot turn a failed check into disconnect evidence, but it prevents an
earlier
confirmed_onceresult from accepting a laterConnectedtunnel or authorizing a laterconnectretry, and prevents a later successful empty answer from reopening the readiness hold untila successful managed confirmation clears the mismatch. With neither a pending empty-answer hold nor a retained mismatch,
unverifiedcountsunanswered checks against a live tunnel and is never reset. A hold persists until a successful registration response resolves
it, so intermittent failures cannot pre-empt the fourth empty answer reaching
mismatch.unverifiedalso cannot gate that report: one early unanswered check on a leftover tunnel wouldotherwise outrank a mismatch confirmed later, logging
tunnel is up ... left it connectedat<4>for a tunnel the run had just torn down, with nothing at
<3>to send the operator to thedashboard. The report branches on the live
registration_stateor that retained mismatchclassification, then on the last
warp-cli status, so the unverified line only prints while thetunnel really is up. A mismatch is split by its retained empty-or-foreign classification, because
a device whose enrollment never completed reports no registration rather than a foreign tenant,
and the two send an operator to different places. The empty-organization exit runs a status query
before it leaves, so that outcome is not the one path with no record of the tunnel in the journal.
The unit is
BindsTo=andUpholds=cloudflare-warp.service, which cover different triggers.An explicit restart, which is what sops issues for
restartUnits(try-restart), reaches theoneshot through
BindsTo=alone. An unexpected exit does not, becauseBindsTo=stops theoneshot ahead of the restart-dependency propagation for that same event, so
Upholds=is whatstarts it again.
PartOf=adds nothing either way and stays unset, sinceBindsTo=wins thatrace whether or not
PartOf=is also present. Verified against systemd 261.1 from this flake'spin, with
try-restart(the verbsops-install-secretsissues) as well asrestart.The oneshot also waits on
network-online.target. Upstream orderswarp-svconnetwork.target,which says nothing about an associated link, and 25.05 decoupled
multi-user.targetfromnetwork-online.target, so the retry budget could otherwise burn before Wi-Fi associates andauto_connect = 0would leave nothing to reconnect. Per-attempt states log at<4>and onlyunrecoverable or enforcement states at
<3>, sojournalctl -p errstays quiet on a healthyboot rather than reporting every warm-up as broken. Every connect-script
warp-clicall passes--accept-tos, which is a global option on 2026.3.846.0, so the fail-closeddisconnectcannotbe the one call that gets refused.
Bounds:
timeout -k 1s 5sper IPC call, since a baretimeoutsends only SIGTERM and waits, 30 attempts or a 120s deadline,TimeoutStartSec=180, andexit 0 after logging so a user who deliberately keeps WARP off does not boot into a failed
unit. With neither a pending empty-answer hold nor a retained mismatch, a live tunnel whose
registration goes unanswered three times is terminal, since nothing is left to request. A pending
hold remains until a successful response resolves it, and a retained mismatch keeps the existing
attempt/deadline budget open until a fresh successful response resolves or retries cleanup. The final log
line normally names the state that ended the run. A conclusive mismatch remains its terminal
diagnostic across later unanswered checks until a successful managed confirmation clears it
(unverified registration, no registration at all, wrong organization, refused connect, or
unreachable daemon).
enableStrictShellChecks = trueroutes the script throughwriteShellApplication, soShellCheck and
errexit/nounset/pipefailcover this fail-closed guard.Per-host enablement
warp(Full / Gateway with WARP)127.0.0.1:53, so Gateway DNS does not collide. The rawservices.cloudflare-warpblock inservices.nixis removed because a second definition ofpackagewould conflict.tunnelonlydns = "dnsmasq"for private-host mappings, whichwarpwould break. WARP still supplies the tunnel, HTTP filtering, network policies, and posture checks.tpnix gates
enableonflake.lib.nixos.hosts.tpnix.sopsRuntimeReady. The flag is true sincePR #305, so the gate is a kill switch rather than what keeps the host un-enrolled: losing the
runtime decryption key drops the WARP stack with it and keeps the
sops.secrets."cloudflare-warp/*"declarations from failing activation on an un-decryptablepayload.
Coverage
CI has no secrets submodule, so host evaluation only ever takes the un-enrolled path.
checks."apps/cloudflare-warp-module-eval"evaluates the module against an in-reponon-secret fixture to force the managed branch and against a missing path for the un-enrolled
branch, deep-forcing the sops template content, the install
ExecStartPre, and the connectscript text so a regression in the registration guard cannot reach
mainunnoticed. An exactsource-derived assertion requires a nonempty configured organization before confirmation assigns
confirmed_onceand resets the empty-response readiness window. It also asserts thatpackage/enable/udpPort/openFirewallreachservices.cloudflare-warp(headless selection has no other test now thatservices.nixno longersets it). The
enrolledfixture covers non-default UDP 24080 withopenFirewall = false;mdmVariantcovers default UDP 2408 withopenFirewall = trueand requires the port inallowedUDPPorts, so a hardcoded false cannot leave managed hosts closed. The check also assertsthat the template is the only restart owner, the
BindsTo/Upholdswiring, the shellcheck gate,and that the un-enrolled branch declares no daemon and no connect unit. The fixture holds
placeholder values only and is not sops-encrypted: the check needs the path to exist and the
three keys to resolve, not real credentials.
Both managed-branch warnings are conditional, so forcing only the template and script text left
their predicates unreachable by any evaluation in this repo: a rename of
services.dnscrypt-proxy.enable,networking.networkmanager.dns, ornetworking.firewall.checkReversePathwould have surfaced first on an operator's rebuild afterthe sops payload landed. The check now forces the managed
warningslist with independentfixtures: dnscrypt-proxy and NetworkManager dnsmasq under DNS-owning
serviceMode "warp"mustwarn; dnscrypt-proxy under
tunnelonlymust stay silent; and dnscrypt-proxy under1dot1mustwarn. The independent
checkReversePath = "strict"fixture also requires its warning. A baselinehost stays silent, and a
staleDnsfixture holds the NetworkManager negative case:dns = "dnsmasq"with NetworkManager off must not warn.The rendered
mdm.xmlis asserted against amdmVariantfixture that diverges onserviceMode,autoConnect, andswitchLocked, since the enrolled fixture holds all three at their optiondefaults and would pass against hardcoded output; the enrolled fixture pins the other arm of
switchLocked's Nixif.warp-clion PATH is asserted for the managed branch too, the onebranch where the module does not place the package itself.
Three further assertions pin properties that the previous substring matches let through. The
registration capture is matched whole, so neither
2>&1(which would compare an enrolleddevice as unmanaged) nor
2>/dev/null(which would drop warp-cli's reason for a failed checkfrom the journal) can return. The
xmllintguard must carry both the stderr redirect andexit 1, since suppressing the leak without the exit would install a malformedmdm.xmlsilently. The terminal report must preserve a conclusive mismatch across later failed checks
with an empty-or-foreign classification, and it must not reference
unverified. The scoped assertions prove that a retained mismatch blocks staleconfirmed_onceacceptance andreopening the readiness hold, preserves the attempt and deadline budget for fresh cleanup, but
cannot make a failed check enter the live
mismatch)disconnect enforcement. Readiness assertions pin the bounded empty-answer producer, reset it on a fresh confirmation,and preserve it across failed checks. They require
held_emptyto block staleconfirmed_onceacceptance and connection retries, retain mismatch gates, and prohibit
connected=1in the heldor ordinary unverified paths. Thus an interleaved timeout cannot pre-empt the fourth empty
response reaching
mismatchor accept consumer WARP. Three morecover the round after that: the
status query on the empty-organization exit, the split mismatch report, and a
connectOfffixturefor
connectOnBoot = false, the one managed sub-branch no fixture reached. Each was verified byinjecting the regression it targets and confirming it is the assertion that fails.
Docs
New pages under
docs/cloudflare/warp/: deployment (dashboard prerequisites, thesecrets/cloudflare-warp.yamlpayload, and pushing the submodule before bumping its pointer),modes (which mode each host runs and which collide with a local resolver), operations
(verifying registration, reading
cloudflare-warp-connectlogs, recovering a stalemdm.xml), reference (option tomdm.xmlkey mapping), and cheatsheet. Linked fromdocs/cloudflare/README.mdanddocs/index.md.Test plan
nix flake check path:. --accept-flake-config --no-build --offline(exit 0)nix build 'path:.#checks.x86_64-linux."apps/cloudflare-warp-module-eval"'(exit 0, bothsops branches forced)
unit-script-cloudflare-warp-connect-start; ShellCheck 0.11.0 passes on the connectscript
warp-cliacross eight states: verified tunnel,connected but unverifiable, connected to the wrong organization, banner on stderr,
whitespace in the organization value, connect requested and refused, unreachable daemon,
and empty organization secret
mismatch prevents a readiness hold and routes a
Connectedtunnel through existing mismatchcleanup; the first three initial and post-confirmation empty responses remain held, then a
fourth successful empty response routes to mismatch cleanup
unverifiedunchanged and keeps the existing attempt/deadline budget open for a fresh successfulresponse to retry cleanup without using stale state as disconnect evidence
connection retry only while neither a successful empty-response hold nor a mismatch is retained
acceptance or a connect retry, and the fourth success reaches mismatch cleanup
managed-organization clause from confirmation. Each mutation failed
apps/cloudflare-warp-module-evalon its focused assertion; the exact guards were restoredguard, and removed the confirmation reset. Each mutation failed
apps/cloudflare-warp-module-evalon its focused assertion; the exact state guards were restoredretry, held-empty plus timeout, retained mismatch, and readiness reset.
inherit (cfg) package udpPort openFirewall;temporarily withopenFirewall = false;:apps/cloudflare-warp-module-evalfailed withopenFirewall = true must open udpPort"1dot1"allow-list member; the targetevaluation failed on the tunnelonly and
1dot1assertions, respectivelywarp-clicommand omits--accept-tosBindsTo=/Upholds=and the absence ofX-Restart-Triggers=nix eval path:.#nixosConfigurations.system76.config.programs.cloudflare-warp.extended.serviceMode->
warpnix eval path:.#nixosConfigurations.tpnix.config.programs.cloudflare-warp.extended.serviceMode->
tunnelonlynix eval path:.#nixosConfigurations.tpnix.config.programs.cloudflare-warp.extended.enable->
truetreefmt --fail-on-change --no-cache(0 changed)Note:
secrets/cloudflare-warp.yamlis not committed, so both hosts currently build withwarp-cliinstalled, no daemon, and the build warning by design. Committing the sops payloaddescribed in
docs/cloudflare/warp/deployment.mdswitches them to managed enrollment with nofurther module change.