Skip to content

Smaller findings from the five user tests: lint offline, status precedence, snapshot silence, and friends #43

Description

@jordonpeterson

Collected low-severity findings from five user tests, kept in one issue so they are tracked without flooding the list. All pre-existing unless noted.

lint is unreachable offline, including its lookup-free stage. Rejoining @ org/mobile-core needs no network, but the whole run is held behind the token check that exists for owner removal. docs/LINTING.md also says "It names whichever one you left out"; with no token at all it went to the network and blamed the token's scopes instead. Related: docs/REFERENCE.md's A-4 row says dead rules are "deleted only by --lint --remove-stale-paths", which reads as available to a repo owner and is not.

Repo-level .status cannot distinguish "already converged" from "policy never landed". A fully-converged repo reports skipped if any single op skipped; the precedence ("any skip beats all-unchanged") is undocumented, and the jq remedy FLEET.md prescribes does not disambiguate — you have to count .ops[] | select(.status=="unchanged"). Suggest documenting the precedence and adding ops_unchanged as an unconditional key.

A comment contradicted by a newly inserted rule is not warned about. The rename case is warned (#33); a comment saying "/.github/workflows/ intentionally left to the catch-all until SEC-4412 lands" directly above a new /.github/workflows/ rule is not. Same class of stale prose, and cheaper to detect — the pattern is right there in changes[].pattern.

snapshot reports all-null ownership with no explanation for a zero-byte, BOM'd, or symlinked CODEOWNERS, at exit 0 with a reassuring codeowners_sha256. README says to reach for snapshot first; audit explains all three clearly, but nothing points there.

snapshot and sync disagree when CODEOWNERS is a directory. snapshot: "no CODEOWNERS file found … (use --file)", exit 3 — actively misleading. sync: "read …/.github/CODEOWNERS: is a directory", exit 2 — correct.

Raw git plumbing surfaces for ordinary repo states. A freshly git inited repo yields git ls-tree -r --name-only -z HEAD --: exit status 128: fatal: Not a valid object name HEAD; it never says "this repository has no commits yet".

A BOM on a policy file falls through to the raw JSON decoder (invalid character 'ï'), while a BOM on CODEOWNERS is handled beautifully (pattern "*" matches zero tracked files). The escaping machinery exists; this path doesn't reach it.

sync --out/--summary-out write failures are warnings at exit 0, while plan --out and snapshot --out fail loudly at exit 3 in the same situations. The sync behavior is deliberate and documented; the inconsistency across verbs is not.

on_empty is a single top-level policy field, so a dissolve wave whose repos need different answers cannot be one policy — you drop to per-repo --op and lose the reviewable artifact.

The --on-empty refusal names the modes but not their consequences — nothing hints that inherit on a catch-all leaves paths owned by nobody. Mitigated by plan, but the error is what an operator under time pressure reads.

verify is path-scoped; a reorg's intent is owner-scoped. "Did anyone lose access they should have kept" required hand-derived path globs and a bespoke jq. A verify --owner @org/legacy --retired mode would replace it.

Offline repair leaves litter. set_owners can insert a correct rule but no op deletes the broken line it supersedes, so repos accumulate duplicate patterns and permanent A-8/A-5 findings.

Cosmetic. set_owners(/x/, [@a, @a, @a]) passes check and writes the duplicates verbatim. audit on a 10k-rule file emits 10,002 unpaged lines. --checks ' ' errors while --checks ' a4 ' works. The human-format refusal prints its message twice. .repo in the JSON is the clone path, while FLEET.md's snippets print it as if it were org/name.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions