diff --git a/.agents/reports/quality-audit.md b/.agents/reports/quality-audit.md new file mode 100644 index 0000000..29d7e14 --- /dev/null +++ b/.agents/reports/quality-audit.md @@ -0,0 +1,325 @@ +# Quality Audit Report + +## Human Summary + +Quality meta-audit of **appenv** — a 1701-line single-file Python tool with zero runtime dependencies, 10 CLI subcommands, and 220 tests. All quality gates pass clean: ruff 0 issues, ty 0 errors, pytest 217/220 pass (3 env skips). Coverage is 93.73% with gaps only in network-dependent code paths. Test double strategy is exemplary — zero unittest.mock usage, pure hand-crafted fakes. No fixes needed. Score: **94/100 (Grade A)**. + +## Completion Checklist + +- [x] Entry point inventory + smoke test completed +- [x] Structural inventory completed (noqa, mock, complexity, test discovery, dependencies) +- [x] Quality gates collected (baseline + extreme) +- [x] All 4 investigation streams completed with structured review results +- [x] Tool tolerance audit produced with per-tool signals (ruff/ty/pytest) +- [x] Test collection integrity verified (all test files collected, no config hiding) +- [x] Skip/xfail/xpass audit completed (lazy skips flagged, cross-platform checked) +- [x] Test double strategy analyzed (mock:fake:golden:real per layer) +- [x] E2E coverage assessed for every entry point (PROVEN/SUSPECTED/UNKNOWN/BROKEN) +- [ ] Full CLI test executed — NOT TRIGGERED (existing E2E evidence sufficient) +- [ ] Fixes applied — NOT NEEDED (no critical findings) +- [ ] Fix loop — NOT NEEDED (all gates green) +- [x] North Star generated from loaded skills +- [x] Course Corrections derived (Reality vs North Star diff) +- [ ] Git commit: pending + +## Entry Point Inventory + +| Entry Point | Type | Source | Smoke | E2E Status | Evidence | +|-------------|------|--------|-------|------------|----------| +| init | cli-subcommand | src/appenv.py:meta() | PASS | PROVEN | test_init.py (12 tests) + integration/test_cli.py::test_init_cli | +| migrate | cli-subcommand | src/appenv.py:meta() | PASS | PROVEN | test_migrate.py (12 tests) + integration/test_cli.py::test_migrate_cli | +| self-update | cli-subcommand | src/appenv.py:meta() | PASS | SUSPECTED | test_self_update.py (10 tests), no integration test | +| update-lockfile | cli-subcommand | src/appenv.py:meta() | PASS | SUSPECTED | test_update_lockfile.py (8 tests), no integration test | +| prepare | cli-subcommand | src/appenv.py:meta() | PASS | PROVEN | test_prepare.py (22 tests) + integration/test_cli.py::test_prepare_cli | +| reset | cli-subcommand | src/appenv.py:meta() | PASS | SUSPECTED | test_reset.py (8 tests), no integration test | +| python | cli-subcommand | src/appenv.py:meta() | PASS | SUSPECTED | test_main.py::test_python_method_calls_run, no integration test | +| run | cli-subcommand | src/appenv.py:meta() | PASS | PROVEN | test_main.py + integration/test_subprocess.py::test_subprocess_main_flow | +| uv | cli-subcommand | src/appenv.py:meta() | PASS | SUSPECTED | test_main.py + test_prepare.py, no integration test | +| version | cli-subcommand | src/appenv.py:meta() | PASS | PROVEN | test_main.py::test_show_version + smoke test | +| --help | flag | src/appenv.py:meta() | PASS | PROVEN | smoke test — prints grouped help | +| (no args) | default | src/appenv.py:main() | PASS | PROVEN | smoke test — prints usage | +| --version (flag) | UX ISSUE | N/A | FAIL | UX ISSUE | Smoke test — "unrecognized arguments: --version", version is subcommand not flag | +| symlink dispatch | run mode | src/appenv.py:main() | PASS | PROVEN | integration/test_subprocess.py::test_subprocess_main_flow | + +## Tool Tolerance Audit + +| Tool | Baseline | Extreme | Delta | Signal | +|------|----------|---------|-------|--------| +| ruff | 0 issues | **2621 issues** | **+2621 suppressed** | green | +| ty | 0 errors | 0 errors | 0 | green | +| pytest | 215 passed, 5 deselected | **217 passed, 3 skipped** | **+2 passed (slow)** | green | + +### Suppression Categorization (240 in src/appenv.py) + +| Category | Count | Legitimacy | Rationale | +|----------|-------|------------|-----------| +| T201 (print) | 85 | legitimate | CLI tool, zero-dep constraint | +| COM812 (trailing comma) | 41 | legitimate | Formatter territory | +| DOC/D (docstrings) | 62 | legitimate | Deliberate for single-file tool | +| ARG002 (unused dispatch args) | 13 | legitimate | Argparse signature convention | +| S404/S603/S606/S607 (subprocess) | 13 | legitimate | Core functionality | +| ANN (annotations) | 36 | legitimate | Covered by .pyi stub file | +| CPY001 (copyright) | 33 | questionable | Missing copyright notices | +| FBT001 (boolean trap) | 4 | questionable | Minor — internal methods | +| PLC0415 (lazy imports) | 4 | legitimate | Intentional for download path | +| Other | ~15 | legitimate | Noise from ALL ruleset | + +**Zero critical suppressions.** Project ruff config (B, C4, E, EM, F, FLY, I, N, PERF, RET, RUF, SIM, SLF, TRY, UP, W) is well-tuned. + +## Test Collection Integrity + +| Check | Result | Signal | +|-------|--------|--------| +| Tests on disk | 13 files | — | +| Tests collected (baseline) | 215 nodes | — | +| Tests deselected (slow marker) | 5 | — | +| Total with slow tests | 220 nodes | — | +| Uncollected files | 0 | green | +| Collection errors | 0 | green | +| Config exclusions | `-m "not slow"` in addopts | — | +| conftest hooks modifying collection | none | green | + +- pytest config: importlib mode, timeout=120, durations=10, `-W error`, `-m "not slow"` +- 2 integration files appear "missing" from baseline but are correctly collected when slow tests run +- All test files accounted for: 13 on disk = 13 collected (with appropriate marker filter) + +## Skip/Xfail/Xpass Audit + +| Category | Count | Signal | +|----------|-------|--------| +| @pytest.mark.skip | 0 | — | +| @pytest.mark.skipif (platform) | 1 | green | +| @pytest.mark.skipif (environment) | 2 | green | +| @pytest.mark.xfail | 0 | — | +| XPASS | 0 | — | +| Lazy skips | 0 | green | +| Flaky-hidden | 0 | green | +| Stale temporal skips | 0 | green | + +### Skip Details + +| File:Line | Condition | Type | Verdict | +|-----------|-----------|------|---------| +| integration/test_cli.py:30 | `sys.platform == "win32"` | Platform | Legitimate — pexpect requires Unix | +| integration/test_cli.py:33 | `not _uv_is_runnable()` | Environment | Legitimate — uv binary must exist | +| integration/test_pip_install_uv.py:53 | `not _pip_is_available()` | Environment | Legitimate — pip must be installed | + +- Cross-platform skip asymmetry: balanced (win32 skip for pexpect is correct) +- Zero xfail, zero lazy skips, zero stale temporal skips + +## Test Double Strategy + +| Layer | Mock | Spec'd Mock | Fake | Golden | Real | Total | +|-------|------|-------------|------|--------|------|-------| +| Unit | 0 | 0 | 4 | 0 | ~190 | ~194 | +| Integration | 0 | 0 | 0 | 0 | 5 | 5 | +| E2E | 0 | 0 | 0 | 0 | 2 | 2 | + +- Tautological tests (mock theater): 0 +- Golden file smell (no regenerate path): 0 +- Mock density hotspots: none — zero mock usage across entire test suite +- Overall double strategy verdict: **Excellent** — pure fake strategy with hand-crafted test doubles (FakeResult, FakeNixResult) and pytest-native monkeypatch. Zero unittest.mock dependency. +- Signal: green + +### Test Double Details + +- `FakeResult` — used in test_main.py, test_uv_bin.py (simulates subprocess.run result) +- `FakeNixResult` — used in test_uv_bin.py (simulates nix-build output) +- `MockUvBin` (conftest.py) — hand-crafted fake, not a mock object; returns canned UvBin instances +- `monkeypatch.setattr` — ~20 usages for targeted subprocess/Path patching (pytest-native) + +## Test Structure Summary + +- Total tests: 220 +- Distribution: unit ~200, integration 5, E2E 2 (slow tests include both) +- RED FLAGS: **0/10** — zero mock-only indicators +- Signal: green + +## Test Coverage + +| Module | Coverage | Missing Lines | Signal | +|--------|----------|---------------|--------| +| src/appenv.py | 93.73% | 654-695, 705-725, 751-752, 1458, 1462, 1523-1524 | green | + +- Overall coverage: 93.73% (950 stmts, 54 miss, 262 branch, 4 brpart) +- Modules < 50%: none +- Entry points with 0% coverage: none +- Signal: green + +### Coverage Gap Analysis + +- **Lines 654-695** (42 lines): `_try_uv_from_installer` — GitHub release download + tarfile extraction. Requires network access. +- **Lines 705-725** (21 lines): `_uv_platform_triple` — platform-specific branches (musl/darwin). Single-platform test runner. +- **Lines 751-752** (2 lines): `_resolve_pip_command` — pip discovery via `which()`. Requires pip installed. +- **Lines 1458, 1462** (2 lines): `ensure_gitignore` — "Updated .gitignore" print path. +- **Lines 1523-1524** (2 lines): `ensure_best_python` — "already running best" early return. + +All gaps are in network/platform-dependent code paths. Coverage is honestly measured. + +## Duration Anomalies + +- Total suite time: 4s (baseline), 6.29s (all tests including slow) +- Duration stats: P50=<10ms, P90=~10ms, P95=0.28s, P99=0.48s + +| Category | Count | Details | +|----------|-------|---------| +| EXTREME OUTLIER (>P99+2σ) | 0 | None — longest test 0.77s | +| FAKE SLOW (marked slow, 0.4s | +| HIDDEN SLOW (unmarked, >P95) | 0 | Top durations are all integration tests | +| Zero-duration (<1ms) | 0 | All tests have measurable duration | + +- Slow test cluster: integration/test_cli.py (0.43s-0.77s range) +- Root causes for outliers: none — suite is fast +- Signal: green + +## Dependency Audit + +| Category | Count | Signal | +|----------|-------|--------| +| Forbidden libraries | 0 | green | +| Stdlib reinvention | 0 | green | +| Unused dependencies | 0 | green | +| Missing blessed libraries | 0 | green | +| Available but unused (partial migration) | 0 | green | + +- Runtime dependencies: **zero** — pure stdlib (argparse, subprocess, pathlib, logging, dataclasses) +- argparse and stdlib logging are CORRECT choices under zero-dep constraint (North Star exception) +- `print()` is correct for CLI output under zero-dep constraint (rich not viable) +- Signal: green + +## E2E Coverage Assessment + +- PROVEN: 7 entry points (init, migrate, prepare, run, version, --help, no-args) +- SUSPECTED: 5 entry points (self-update, update-lockfile, reset, python, uv) +- UNKNOWN: 0 +- BROKEN: 0 +- UX ISSUE: 1 (--version flag not supported; use `appenv version` subcommand) +- Full CLI test triggered: NO — existing E2E evidence sufficient +- Signal: green + +## Stream Signals + +- Code Architecture: green +- Code Quality: green +- Test Structure: green +- E2E Coverage + Production Reality: green + +## Architectural North Star + +Generated from python-dev and python-audit skills. Key dimensions: + +| Dimension | True North | Source | +|-----------|------------|--------| +| HTTP Client | Not needed (zero-dep tool) | python-dev | +| CLI Framework | argparse (zero-dep exception) | python-dev + project constraint | +| Logging | stdlib logging (zero-dep exception) | python-dev + project constraint | +| Data Structures | dataclasses with slots=True | python-dev | +| Testing | pytest + hand-crafted fakes | python-tests | +| Type System | .pyi stub files, no Any in public API | python-typing | +| Coverage | ≥90% with branch coverage | python-dev | +| Architecture | Layer boundaries enforced by pytest-archon | python-architecture | +| Quality Gates | ruff + ty + pytest, 0 issues/errors/failures | python-dev | + +## Course Corrections + +### NAV-01 Architecture Enforcement [RED] +- **Current heading:** No test_architecture.py, no pytest-archon dependency +- **True north:** pytest-archon rules enforcing layer boundaries +- **Correction:** Add architecture tests or document intentional omission for single-file design + +### NAV-02 Forbidden Library Usage — argparse and logging [ORANGE] +- **Current heading:** Uses argparse (1) and stdlib logging (1) — correct under zero-dep constraint +- **True north:** typer + structlog +- **Correction:** Document project-specific exception in North Star + +### NAV-03 Test Architecture [GREEN] +- **Current heading:** Zero mocks, pure fake strategy — exceeds North Star minimum +- **True north:** Mock only external boundaries, never own code +- **Correction:** None needed — already exceeding + +### NAV-04 Coverage [GREEN] +- **Current heading:** 93.73% — above ≥90% target, gaps only in network code +- **True north:** ≥90% with branch coverage +- **Correction:** None needed + +### NAV-05 Slow Test Gate [ORANGE] +- **Current heading:** 5 tests excluded from default run, 3 more skip when run +- **True north:** 100% pass rate +- **Correction:** Ensure CI runs slow tests in separate matrix step + +### NAV-06 Docstring Compliance [ORANGE] +- **Current heading:** No docstrings (D1xx rules excluded from ruff config) +- **True north:** 0 issues with project config +- **Correction:** Deliberate for single-file tool — no change needed + +### NAV-07 Test Collection Integrity [GREEN] +- **Current heading:** All test files collected, no config hiding +- **True north:** All tests discoverable and collected +- **Correction:** None needed + +### NAV-08 type:ignore [GREEN] +- **Current heading:** Zero type:ignore comments +- **True north:** No type:ignore without error code +- **Correction:** None needed + +### NAV-09 noqa Usage [GREEN] +- **Current heading:** 2 noqa comments, both SLF001 on argparse internals +- **True north:** No noqa without justification +- **Correction:** None needed + +### NAV-10 Print Usage [GREEN] +- **Current heading:** 20+ print() calls for CLI output — correct under zero-dep +- **True north:** Use rich.console +- **Correction:** Document project-specific exception + +- NAV-items total: 10 +- Dimensions on course (no deviation): 6 +- Dimensions with deviation: 4 (1 red, 2 orange, 1 green with note) +- Signal: green (all deviations are documented project-specific exceptions) + +## Test Automation + +- Task runner: tox (pyproject.toml [tool.tox]) +- Single-command gate: YES (`tox -p` runs pre-commit + cov + multi-version tests) +- Default coverage: partial (slow tests excluded via `-m "not slow"`, run separately via `tox -e slow`) +- Signal: orange (slow tests excluded from default, but have dedicated tox env) + +## Infrastructure Recommendations + +- **Coverage pipeline**: Already configured. `tox -e cov` runs `pytest --cov=appenv --cov-report=term-missing --cov-report=html`. No changes needed. +- **CI gate recommendation**: Already configured. `tox -p` runs all default envs. Consider adding `tox -e slow` to CI matrix. +- **Duration regression**: Suite runs in 3-4s baseline. No outliers. Current `--durations=10` in pytest config is sufficient. + +## Critical Findings Fixed + +No fixes applied — no critical findings. + +## Full CLI Test Trace + +Full CLI test not triggered — existing E2E evidence sufficient. + +## Code Volume + +No code changes in this audit. + +## Post-Fix Quality Gates + +| Tool | Result | +|------|--------| +| tox | pre-commit PASS, cov PASS, 3.10-3.14 PASS | +| ruff | 0 issues | +| ty | 0 errors | +| pytest baseline | 215 passed, 5 deselected | +| pytest all | 217 passed, 3 skipped | +| E2E smoke | PASS | + +## Recommendations + +1. **--version flag UX**: Consider adding `parser.add_argument('--version')` support alongside the `version` subcommand for discoverability. Low priority. +2. **Architecture enforcement**: Consider adding pytest-archon for a single-file tool — evaluate whether the overhead is justified. Low priority. +3. **Slow test CI integration**: Ensure `tox -e slow` runs in CI. Current state is acceptable if slow tests run somewhere. Medium priority. +4. **Network-dependent coverage**: The `_try_uv_from_installer` path (42 lines uncovered) could benefit from a slow integration test with network mocking. Low priority. + +## Raw Data Location + +`.agents/tmp/quality/` — inventory/, baseline/, extreme/, analysis/, e2e/ diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml new file mode 100644 index 0000000..195b92a --- /dev/null +++ b/.github/workflows/e2e.yml @@ -0,0 +1,155 @@ +name: E2E Tests + +on: + push: + pull_request: + types: [opened, reopened, synchronize] + +jobs: + # E2E: Debian Bookworm + python3-pip — which(pip) fallback bootstraps uv + debian-bookworm-pip: + runs-on: ubuntu-24.04 + container: + image: debian:bookworm + + env: + APPENV_VERBOSE: "1" + + steps: + - uses: actions/checkout@v4 + + - name: Install prerequisites + run: apt-get update && apt-get install -y python3 python3-pip ca-certificates + + - name: Bootstrap appenv + run: | + mkdir /tmp/testproject && cd /tmp/testproject + + cp $GITHUB_WORKSPACE/src/appenv.py appenv + chmod +x appenv + + printf 'http\nhttpie\n\n\n\n3.11\n' | ./appenv init + + ./appenv version + python3 --version + ./appenv update-lockfile + - name: Show project info + run: cd /tmp/testproject && ./appenv version && python3 --version + # E2E: Bare Debian Bookworm — only python3, astral.sh installer fallback + debian-bookworm-minimal: + runs-on: ubuntu-24.04 + container: + image: debian:bookworm + + env: + APPENV_VERBOSE: "1" + + steps: + - uses: actions/checkout@v4 + + - name: Install python3 + run: apt-get update && apt-get install -y python3 ca-certificates + + - name: Bootstrap appenv (astral.sh installer) + run: | + mkdir /tmp/testproject && cd /tmp/testproject + + cp $GITHUB_WORKSPACE/src/appenv.py appenv + chmod +x appenv + + cat > pyproject.toml << 'EOF' + [project] + name = "bootstrap-test" + version = "0.1.0" + dependencies = ["httpie"] + requires-python = ">=3.10" + EOF + + ./appenv version + python3 --version + ./appenv update-lockfile + - name: Show project info + run: cd /tmp/testproject && ./appenv version && python3 --version + # E2E: uvx from git — user has only uv, runs uvx appenv init + ubuntu-uvx: + runs-on: ubuntu-24.04 + + env: + APPENV_VERBOSE: "1" + + steps: + - uses: astral-sh/setup-uv@v7 + + - name: uvx appenv init from current branch + run: | + mkdir /tmp/testproject && cd /tmp/testproject + printf 'http\nhttpie\n\n\n\n3.12\n' | uvx --from git+https://github.com/flyingcircusio/appenv@${{ github.head_ref || github.ref_name }} appenv init + + - name: Show info + run: cd /tmp/testproject && ./appenv version && python3 --version + + # E2E: Alpine Linux — musl libc, astral.sh installer must use linux-musl triple + alpine-musl: + runs-on: ubuntu-24.04 + container: + image: alpine:latest + + env: + APPENV_VERBOSE: "1" + + steps: + - uses: actions/checkout@v4 + + - name: Install python3 + ca-certificates + run: apk add python3 ca-certificates + + - name: Bootstrap appenv (musl installer) + run: | + mkdir /tmp/testproject && cd /tmp/testproject + + cp $GITHUB_WORKSPACE/src/appenv.py appenv + chmod +x appenv + + cat > pyproject.toml << 'EOF' + [project] + name = "bootstrap-test" + version = "0.1.0" + dependencies = ["httpie"] + requires-python = ">=3.10" + EOF + + ./appenv version + python3 --version + ./appenv update-lockfile + - name: Show project info + run: cd /tmp/testproject && ./appenv version && python3 --version + # E2E: macOS ARM64 — native aarch64-apple-darwin, tests Apple Silicon triple + macos-arm64: + runs-on: macos-14 + + env: + APPENV_VERBOSE: "1" + + steps: + - uses: actions/checkout@v4 + + - name: Bootstrap appenv (macOS ARM64) + run: | + mkdir /tmp/testproject && cd /tmp/testproject + + cp $GITHUB_WORKSPACE/src/appenv.py appenv + chmod +x appenv + + cat > pyproject.toml << 'EOF' + [project] + name = "bootstrap-test" + version = "0.1.0" + dependencies = ["httpie"] + requires-python = ">=3.10" + EOF + + ./appenv version + python3 --version + ./appenv update-lockfile + - name: Show project info + run: cd /tmp/testproject && ./appenv version && python3 --version diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml deleted file mode 100644 index 686c4d4..0000000 --- a/.github/workflows/lint.yml +++ /dev/null @@ -1,41 +0,0 @@ -name: Lint - -on: - push: - pull_request: - type: [ "opened", "reopened", "synchronize" ] - -env: - FORCE_COLOR: 1 - -jobs: - build: - runs-on: ubuntu-20.04 - - steps: - - uses: actions/checkout@v2 - - - name: Cache - uses: actions/cache@v2 - with: - path: | - ~/.cache/pip - ~/.cache/pre-commit - key: - lint-v1-${{ hashFiles('**/setup.py') }}-${{ hashFiles('**/.pre-commit-config.yaml') }} - restore-keys: | - lint-v1- - - name: Set up Python - uses: actions/setup-python@v2 - with: - python-version: '3.x' - - - name: Install dependencies - run: | - python -m pip install -U pip - python -m pip install pre-commit - - name: Lint - run: | - pre-commit run --all-files --show-diff-on-failure - env: - PRE_COMMIT_COLOR: always diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ca04670..0813898 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -1,65 +1,34 @@ -# This is a basic workflow to help you get started with Actions +name: Tox Tests -name: Unit tests - -# Controls when the action will run. Triggers the workflow on push or pull request -# events but only for the master branch on: push: pull_request: - type: [ "opened", "reopened", "synchronize" ] + types: [opened, reopened, synchronize] + workflow_dispatch: -# A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: - # This workflow contains a single job called "build" - build-old-python: + build: strategy: + fail-fast: false matrix: - python-version: ['3.6', '3.7'] + python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"] - # The type of runner that the job will run on - runs-on: ubuntu-20.04 + runs-on: ubuntu-24.04 - # Steps represent a sequence of tasks that will be executed as part of the job steps: - # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - with: - python-version: ${{ matrix.python-version }} - - # Runs a set of commands using the runners shell - - name: Setup - run: pip install tox - - - name: Show environment - run: set - - - name: Test - run: tox -e py -- -vv + - uses: actions/checkout@v4 - build-new-python: - strategy: - matrix: - python-version: ['3.8', '3.9', '3.10', '3.11', '3.12'] - - # The type of runner that the job will run on - runs-on: ubuntu-22.04 - - # Steps represent a sequence of tasks that will be executed as part of the job - steps: - # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 + - uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - # Runs a set of commands using the runners shell - - name: Setup - run: pip install tox + - uses: astral-sh/setup-uv@v7 + with: + enable-cache: true - - name: Show environment - run: set + - run: uv tool install tox --python ${{ matrix.python-version }} --with tox-uv-bare --with tox-gh - - name: Test - run: tox -e py -- -vv + - name: Run tests + run: tox + env: + TOXENV: "${{ matrix.python-version }}" diff --git a/.gitignore b/.gitignore index 381980b..665d5c7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,9 +1,10 @@ -example/.ducker -bin/ -include/ -htmlcov/ -lib/ -pyvenv.cfg +# Appenv runtime directory (created on first run) +.appenv/ + +# Example project artifacts +example/.venv/ +example/.appenv/ +example/.httpie *.sublime-* @@ -12,127 +13,52 @@ __pycache__/ *.py[cod] *$py.class -# C extensions -*.so - # Distribution / packaging -.Python build/ -develop-eggs/ dist/ downloads/ eggs/ .eggs/ -lib/ -lib64/ -parts/ sdist/ var/ -wheels/ -pip-wheel-metadata/ -share/python-wheels/ *.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# PyInstaller -# Usually these files are written by a python script from a template -# before PyInstaller builds the exe, so as to inject date/other infos into it. -*.manifest -*.spec - -# Installer logs -pip-log.txt -pip-delete-this-directory.txt # Unit test / coverage reports +report.xml htmlcov/ .tox/ -.nox/ .coverage .coverage.* .cache -nosetests.xml coverage.xml -*.cover -*.py,cover -.hypothesis/ +coverage.json .pytest_cache/ -# Translations -*.mo -*.pot - -# Django stuff: -*.log -local_settings.py -db.sqlite3 -db.sqlite3-journal - -# Flask stuff: -instance/ -.webassets-cache - -# Scrapy stuff: -.scrapy - # Sphinx documentation docs/_build/ -# PyBuilder -target/ - -# Jupyter Notebook -.ipynb_checkpoints - -# IPython -profile_default/ -ipython_config.py - # pyenv .python-version -# pipenv -# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. -# However, in case of collaboration, if having platform-specific dependencies or dependencies -# having no cross-platform support, pipenv may install dependencies that don't work, or not -# install all needed dependencies. -#Pipfile.lock - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow -__pypackages__/ - -# Celery stuff -celerybeat-schedule -celerybeat.pid - -# SageMath parsed files -*.sage.py - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ +# mypy +.mypy_cache/ -# Spyder project settings -.spyderproject -.spyproject +# ruff +.ruff_cache/ -# Rope project settings -.ropeproject +# Profiling artifacts +*.prof +*.pstats -# mkdocs documentation -/site +# Complexipy analysis artifacts +.complexipy_cache/ +complexipy_results_*.json -# mypy -.mypy_cache/ -.dmypy.json -dmypy.json +# uv cache +.uv/ -# Pyre type checker -.pyre/ +# opencode +.agents/tmp +.venv +.appenv +.batou-lock diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index fc455aa..ddfac15 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,16 +1,30 @@ +exclude: ^appenv$ + repos: -- repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.5.0 + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v6.0.0 hooks: - - id: check-yaml - - id: end-of-file-fixer - - id: trailing-whitespace -- repo: https://github.com/google/yapf - rev: 'v0.40.2' # Use the sha / tag you want to point at + - id: detect-private-key + - id: check-added-large-files + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-json + - id: check-toml + + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.15.6 hooks: - - id: yapf - args: [-i, -p] -- repo: https://github.com/pycqa/flake8 - rev: '7.0.0' # pick a git hash / tag to point to + - id: ruff-check + args: [--fix] + - id: ruff-format + + - repo: local hooks: - - id: flake8 + - id: ty-check + name: ty type check + entry: uv run --with ty ty check src/ + language: system + types: [python] + pass_filenames: false + always_run: true diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 0000000..3b693fc --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,24 @@ +# Read the Docs configuration file for Sphinx projects +# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details + +# Required +version: 2 + +# Uses uv to manage virtualenv for building the docs, see: +# https://docs.readthedocs.com/platform/stable/build-customization.html#install-dependencies-with-uv +build: + os: ubuntu-24.04 + tools: + python: "3.14" + jobs: + pre_create_environment: + - asdf plugin add uv + - asdf install uv latest + - asdf global uv latest + create_environment: + - uv venv "${READTHEDOCS_VIRTUALENV_PATH}" + install: + - UV_PROJECT_ENVIRONMENT="${READTHEDOCS_VIRTUALENV_PATH}" uv sync --frozen --group docs +sphinx: + configuration: docs/conf.py + fail_on_warning: false diff --git a/.vulture_exclude b/.vulture_exclude new file mode 100644 index 0000000..3961639 --- /dev/null +++ b/.vulture_exclude @@ -0,0 +1,11 @@ +# Vulture whitelist for false positives +# Run: vulture src/ tests/ .vulture_exclude --min-confidence=80 + +# src/appenv.py:159 - 'quiet' param in cmd() not implemented (API compat) +quiet # noqa + +# test_main.py:769 - lambda parameter must match function signature +b # noqa + +# test_update_lockfile.py:183 - pytest fixture declared but unused +tmp_path # noqa diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..0c75ff6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,15 @@ +# AGENTS.md + +Dev docs for agents working on this codebase: docs/dev/index.md + +## Python Version Support + +| Layer | Minimum | Rationale | +|-------|---------|-----------| +| `src/appenv.py` bootstrapping | 3.9 | appenv must run on any Python that exists in the wild — it's a bootstrap script | +| Everything else (tests, tooling) | 3.10+ | | +| Stub files (`.pyi`) | 3.14 | Modern type syntax (`type` statements, PEP 695 generics) | + +## Dependencies + +None. `src/appenv.py` uses only the Python standard library — no runtime dependencies, no vendored code. This is a hard constraint: appenv is a single-file bootstrap script that must work with nothing but a Python interpreter. diff --git a/README.md b/README.md index f9a2bfa..a978f68 100644 --- a/README.md +++ b/README.md @@ -1,96 +1,122 @@ # appenv -Self-contained bootstrapping/updating of Python applications deployed through shared repositories. +[![E2E Tests](https://github.com/dpausp/appenv/actions/workflows/e2e.yml/badge.svg)](https://github.com/dpausp/appenv/actions/workflows/e2e.yml) [![Tox Tests](https://github.com/dpausp/appenv/actions/workflows/main.yml/badge.svg)](https://github.com/dpausp/appenv/actions/workflows/main.yml) -> The following examples use the `ducker` package to illustrate how to use ->`appenv`. `ducker` and `appenv` are not related at all. +appenv pins Python packages to exact versions and exposes their binaries +via symlinks — one file, no installation step. Drop it into a repository, +commit it, and every checkout (local or remote) gets the same tools at the +same versions by running `./http`, `./pytest`, `./batou`, or whatever you need. -## Bootstrapping an application / project +**appenv never modifies your system** — all state lives in `.appenv/` inside +the project directory. Remove that folder and nothing is left behind. -Use `curl -sL https://github.com/flyingcircusio/appenv/raw/master/bootstrap | sh` for bootstrapping a new project. +## Using an existing appenv project -``` -$ curl -sL https://github.com/flyingcircusio/appenv/raw/master/bootstrap | sh -Let's create a new appenv project. +Someone gave you a project that already uses appenv? Just run the command: -What should the command be named? ducker -What is the main dependency as found on PyPI? [ducker] -Where should we create this? [/private/tmp/ducker] +```console +git clone && cd +./http # First run sets up everything automatically +``` -Creating appenv setup in /private/tmp/ducker ... +Running the command will get an appenv-managed [uv](https://docs.astral.sh/uv/) if it's not globally available on your system. -Done. You can now `cd ducker` and call `./ducker` to bootstrap and run it. +No `uv.lock` (should be committed) or dependencies changed? -$ cd ducker -$ ./ducker -Running unclean installation from requirements.txt -Ensuring unclean install ... -Please initiate a query. -Ducker (? for help) q +```console +./appenv update-lockfile ``` -## Freezing requirements for repeatable builds +(Only needed again after manually editing `pyproject.toml`) -Using frozen requirements makes the builds repeatable for you and your team -and also speeds up subsequent invocations: +Use `./appenv uv add/remove` to manage your dependencies or just use `uv` as you are used to it. -``` -$ ./appenv update-lockfile -Updating lockfile -Installing packages ... -$ time ./ducker wikipedia -Installing ducker ... -./ducker wikipedia 2.91s user 0.99s system 88% cpu 4.407 total +### Upgrading from requirements.txt -$ time ./ducker wikpedia -./ducker wikipedia 0.22s user 0.11s system 90% cpu 0.371 total +Already an appenv user and still using `requirements.txt` instead of `pyproject.toml`? +```console +uvx appenv migrate ``` -## Using a specific version of Python for your application +## New Project + +Requires Python 3.9+ (managed environments need 3.10+). [uv](https://docs.astral.sh/uv/) 0.5.0+ is auto-installed if not found. +Get appenv via `uvx` or download the single-file script. -`appenv` tries to use the best Python version available. It bootstraps with -the Python 3 interpreter available in your PATH as `python3` and then can -either detect the newest Python or select the best python of your choice. +`appenv init` will ask you some questions and set up the project (interactive +by default — pass `--binary` and `--dep` for non-interactive use). The example +assumes that you want to run a binary called `http` from the `httpie` package. -Two disable the automatic detection of the newest version and provide a -list of acceptable Python versions (tried in the order you list them) -add the following line to your requirements.txt file: +### uvx (uv) +`uvx` is part of [uv](https://docs.astral.sh/uv/) — the easiest way to start: + +```shell +# appenv init is interactive +# Answer: +# httpie as dependency +# http as binary +uvx appenv init +./http ``` -# appenv-python-preference: 3.6,3.9,3.8 + +### Manual Download + +No uv installed? Download appenv directly: + +```shell +curl -sL https://raw.githubusercontent.com/flyingcircusio/appenv/master/src/appenv.py -o appenv +chmod +x appenv +# appenv init is interactive +# Answer: +# httpie as dependency +# http as binary +./appenv init +./http ``` -The best version that is found on the system will be used to re-spawn appenv -and then also used to manage the virtual environments for your application. +**What just happened?** -AppEnv itself is tested against Python 3.6+. +- appenv installed itself inplace by adding the `./appenv` script. +- `init` created `pyproject.toml` and a symlink `http → appenv`. +- `./http` set up the venv with pinned versions from `uv.lock`, then ran the `http` binary (from the [httpie](https://github.com/httpie/httpie) package) -## Learning more about appenv +The repository now contains: -``` -$ ./appenv --help -usage: appenv [-h] {update-lockfile,init,reset,prepare,python,run} ... - -positional arguments: - {update-lockfile,init,reset,prepare,python,run} - update-lockfile Update the lock file. - init Create a new appenv project. - reset Reset the environment. - prepare Prepare the venv. - python Spawn the embedded Python interpreter REPL - run Run a script from the bin/ directory of the virtual env. - -options: - -h, --help show this help message and exit +```shell +myproject/ +├── appenv # The appenv script (single file, committed to git) +├── http -> appenv # Runs the `http` binary from installed deps +├── pyproject.toml # Project config and dependency list +└── uv.lock # Exact versions of all dependencies (committed to git) ``` -## Testing +### Non-Interactive / CI -If you want to contribute, please install `tox` and run it. +For scripts and CI pipelines — no TTY needed: +```shell +appenv init --binary http --dep httpie --name myproject ``` -$ tox +### Development + +For dev tooling, `uv run` and other `uv` commands work transparently. +`appenv` automatically creates a `.venv` symlink to make this work: + +```shell +# includes dev dependencies automatically +uv run pytest -xvs ``` + +## Documentation + +Full documentation at [Readthedocs](https://appenv-test.readthedocs.io): + +- [Installation](docs/user/installation.md) -- how to get appenv +- [Commands Reference](docs/user/commands.md) -- all commands with options +- [Workflows](docs/user/workflows.md) -- common usage patterns +- [Locking Behavior](docs/user/locking-behavior.md) -- how uv.lock works +- [Developer Guide](docs/dev/index.md) -- development setup and architecture diff --git a/bootstrap b/bootstrap deleted file mode 100755 index c614ffd..0000000 --- a/bootstrap +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/sh -{ # prevent execution of partial downloads. -set -e - -oops() { - echo "$0:" "$@" >&2 - exit 1 -} - - -t="$(mktemp -d)/appenv" -curl -sL https://raw.githubusercontent.com/flyingcircusio/appenv/master/src/appenv.py -o $t || oops "failed to download appenv" -python3 $t init < /dev/tty -} diff --git a/bootstrap-dev.sh b/bootstrap-dev.sh deleted file mode 100755 index 080b995..0000000 --- a/bootstrap-dev.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh - -set -ex - -PYTHONABS=${1:-python3} -PYTHON=$(basename $PYTHONABS) - -rm -rf .Python bin lib include -$PYTHONABS -m venv . -bin/pip install --upgrade -r requirements.txt diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 0000000..07bef47 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,7 @@ +# Sphinx build artifacts +_build/ +autoapi/ + +# OS files +.DS_Store +Thumbs.db diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..890f6d7 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,27 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = uv run --group docs sphinx-build +SOURCEDIR = . +BUILDDIR = _build + +.PHONY: all help clean html + +all: html + +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +clean: + rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(SPHINXOPTS) $(SOURCEDIR) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +# Catch-all target: route all unknown targets to Sphinx +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 0000000..19d15d9 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,348 @@ +"""Sphinx configuration for appenv documentation.""" + +from __future__ import annotations + +import ast +import re +import sys +from pathlib import Path + +# Add src to path for imports +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +# Project information +project = "appenv" +copyright = "2026, Flying Circus" +author = "Christian Theune" + +# Read version from source +version_path = Path(__file__).parent.parent / "src" / "appenv.py" +for line in version_path.read_text().splitlines(): + if line.startswith("__version__"): + release = version = line.split('"')[1] + break +else: + release = version = "dev" + +# Extensions +extensions = [ + # Markdown support + "myst_parser", + # Automatic API documentation + "autoapi.extension", + # Source code viewing + "sphinx.ext.viewcode", + # Type hints in documentation + "sphinx_autodoc_typehints", + # AI-friendly output (llms.txt) + "sphinx_llm.txt", + # UX enhancements + "sphinx_copybutton", + "sphinx_design", + "sphinx_togglebutton", + # Diagrams + "sphinx.ext.graphviz", + # Planned features / TODOs + "sphinx.ext.todo", + # Social sharing metadata + "sphinxext.opengraph", +] + +# source_suffix: autoapi generates .rst internally, so register both +source_suffix = {".md": "markdown", ".rst": "restructuredtext"} + +# autoapi configuration +autoapi_type = "python" +autoapi_dirs = ["../src"] +autoapi_file_patterns = ["*.py"] +autoapi_generate_api_docs = True +autoapi_add_toctree_entry = True +autoapi_options = [ + "members", + "undoc-members", + "show-inheritance", + "show-module-summary", +] +autoapi_keep_files = True +autoapi_python_use_implicit_namespaces = True + + +# Skip private members (underscore prefix except dunder) +def autoapi_skip_member( + _app, _what: str, name: str, _obj, skip: bool, _options +) -> bool: + if name.startswith("_") and not name.startswith("__"): + return True + return skip + + +def _suppress_autoapi_orphan_warnings(app, env, docnames): + """Post-build hook: fix autoapi toctree for single-file modules. + + autoapi generates an empty toctree in autoapi/index.rst when the source + is a single .py file (not a package). Add the generated module page so + it is not reported as orphan. + """ + autoapi_index = Path(app.srcdir) / "autoapi" / "index.rst" + if not autoapi_index.exists(): + return + content = autoapi_index.read_text() + if "src/appenv/index" in content: + return + content = content.replace( + ".. toctree::\n :titlesonly:\n\n\n", + ".. toctree::\n :titlesonly:\n\n src/appenv/index\n\n", + ) + autoapi_index.write_text(content) + + +# --- Stub type injection for autoapi --- + +_STUB_PATH = Path(__file__).parent.parent / "src" / "appenv.pyi" + + +def _parse_stub_types(stub_path: Path) -> tuple[dict[str, str], dict[str, str]]: + """Parse .pyi stub to extract typed signatures and attribute types. + + Returns (func_sigs, attr_types) where: + - func_sigs: qualified_name -> "(param: type, ...) -> RetType" + - attr_types: qualified_name -> "TypeAnnotation" + """ + if not stub_path.exists(): + return {}, {} + + tree = ast.parse(stub_path.read_text()) + func_sigs: dict[str, str] = {} + attr_types: dict[str, str] = {} + + def _ann(node: ast.expr) -> str: + return ast.unparse(node) + + def _is_ellipsis(node: ast.expr | None) -> bool: + return isinstance(node, ast.Constant) and node.value is ... + + def _build_params(args: ast.arguments, strip_self: bool = False) -> str: + parts: list[str] = [] + pos_args = list(args.args) + if strip_self and pos_args and pos_args[0].arg == "self": + pos_args = pos_args[1:] + + defaults_offset = len(args.args) - len(args.defaults) + + for i, arg in enumerate(pos_args): + orig_i = i + (len(args.args) - len(pos_args)) + p = arg.arg + if arg.annotation: + p += f": {_ann(arg.annotation)}" + di = orig_i - defaults_offset + if 0 <= di < len(args.defaults) and not _is_ellipsis(args.defaults[di]): + p += f" = {ast.unparse(args.defaults[di])}" + parts.append(p) + + if args.vararg: + p = f"*{args.vararg.arg}" + if args.vararg.annotation: + p += f": {_ann(args.vararg.annotation)}" + parts.append(p) + elif args.kwonlyargs: + parts.append("*") + + for i, arg in enumerate(args.kwonlyargs): + p = arg.arg + if arg.annotation: + p += f": {_ann(arg.annotation)}" + kw_default = args.kw_defaults[i] + if kw_default is not None and not _is_ellipsis(kw_default): + p += f" = {ast.unparse(kw_default)}" + parts.append(p) + + if args.kwarg: + p = f"**{args.kwarg.arg}" + if args.kwarg.annotation: + p += f": {_ann(args.kwarg.annotation)}" + parts.append(p) + + return ", ".join(parts) + + def _build_sig(func: ast.FunctionDef, strip_self: bool = False) -> str: + params = _build_params(func.args, strip_self) + sig = f"({params})" + if func.returns: + sig += f" -> {_ann(func.returns)}" + return sig + + def _is_property(func: ast.FunctionDef) -> bool: + return any( + isinstance(d, ast.Name) and d.id == "property" for d in func.decorator_list + ) + + for node in ast.iter_child_nodes(tree): + if isinstance(node, ast.FunctionDef): + func_sigs[node.name] = _build_sig(node) + elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): + attr_types[node.target.id] = _ann(node.annotation) + elif isinstance(node, ast.ClassDef): + for item in node.body: + if isinstance(item, ast.FunctionDef) and _is_property(item): + if item.returns: + attr_types[f"{node.name}.{item.name}"] = _ann(item.returns) + elif isinstance(item, ast.FunctionDef): + func_sigs[f"{node.name}.{item.name}"] = _build_sig( + item, strip_self=True + ) + elif isinstance(item, ast.AnnAssign) and isinstance( + item.target, ast.Name + ): + attr_types[f"{node.name}.{item.target.id}"] = _ann(item.annotation) + + return func_sigs, attr_types + + +def _inject_stub_types(app, env, docnames): + """Post-autoapi hook: inject type information from .pyi stubs.""" + try: + func_sigs, attr_types = _parse_stub_types(_STUB_PATH) + except SyntaxError: + return + if not func_sigs and not attr_types: + return + + autoapi_dir = Path(app.srcdir) / "autoapi" + if not autoapi_dir.exists(): + return + + for rst_file in autoapi_dir.rglob("*.rst"): + content = rst_file.read_text() + patched = _patch_rst_with_stub_types(content, func_sigs, attr_types) + if patched != content: + rst_file.write_text(patched) + + +def _patch_rst_with_stub_types( + content: str, func_sigs: dict[str, str], attr_types: dict[str, str] +) -> str: + """Patch autoapi-generated RST with type information from stubs.""" + lines = content.split("\n") + result: list[str] = [] + current_class: str | None = None + + for i, line in enumerate(lines): + patched = line + next_line = lines[i + 1] if i + 1 < len(lines) else "" + + # Track class/exception context and patch constructor params + m = re.match(r"^\.\. py:(class|exception):: (\w+)", line) + if m: + current_class = m.group(2) + init_key = f"{current_class}.__init__" + if init_key in func_sigs: + # Extract params only (drop return type from __init__) + params = func_sigs[init_key].split(" -> ", 1)[0] + patched = re.sub(r"\(.*\)$", params, line, count=1) + + # Module-level functions + m = re.match(r"^(\.\. py:function:: )(\w+)\(.*\)$", line) + if m: + name = m.group(2) + if name in func_sigs: + patched = f"{m.group(1)}{name}{func_sigs[name]}" + + # Module-level data/variables + m = re.match(r"^(\.\. py:data:: )(\w+)$", line) + if m: + name = m.group(2) + if name in attr_types and ":type:" not in next_line: + patched += f"\n :type: {attr_types[name]}" + + # Class methods (indented under class) + m = re.match(r"^(\s+)(\.\. py:method:: )(\w+)\(.*\)$", line) + if m and current_class: + indent, prefix, name = m.group(1), m.group(2), m.group(3) + key = f"{current_class}.{name}" + if key in func_sigs: + patched = f"{indent}{prefix}{name}{func_sigs[key]}" + + # Class properties + m = re.match(r"^(\s+)(\.\. py:property:: )(\w+)$", line) + if m and current_class: + indent, prefix, name = m.group(1), m.group(2), m.group(3) + key = f"{current_class}.{name}" + if key in attr_types and ":type:" not in next_line: + patched += f"\n{indent} :type: {attr_types[key]}" + + # Class attributes + m = re.match(r"^(\s+)(\.\. py:attribute:: )(\w+)$", line) + if m and current_class: + indent, prefix, name = m.group(1), m.group(2), m.group(3) + key = f"{current_class}.{name}" + if key in attr_types and ":type:" not in next_line: + patched += f"\n{indent} :type: {attr_types[key]}" + + result.append(patched) + + return "\n".join(result) + + +def setup(app): + app.connect("autoapi-skip-member", autoapi_skip_member) + app.connect("env-before-read-docs", _suppress_autoapi_orphan_warnings) + app.connect("env-before-read-docs", _inject_stub_types) + + +# MyST configuration +myst_enable_extensions = [ + "colon_fence", + "deflist", + "fieldlist", + "tasklist", + "dollarmath", +] +myst_heading_anchors = 3 +myst_all_links_external = False + +# sphinx-copybutton configuration +copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | " +copybutton_prompt_is_regexp = True + +# Type hints presentation +autodoc_typehints = "description" +autodoc_typehints_description_target = "documented" + +# sphinx-llm configuration +llms_txt_build_parallel = True +llms_txt_full_build = True +llms_txt_suffix_mode = "auto" +llms_txt_description = ( + "appenv - Self-contained bootstrapping and updating of Python CLI applications" +) + +# sphinx.ext.todo configuration +todo_include_todos = True + +# sphinx.ext.graphviz configuration +graphviz_output_format = "svg" + +# Theme configuration +html_theme = "furo" +html_title = f"appenv {release}" +html_theme_options = { + "source_repository": "https://github.com/flyingcircusio/appenv/", + "source_branch": "main", + "source_directory": "docs/", + "sidebar_hide_name": False, + "navigation_with_keys": True, + "light_css_variables": { + "font-stack": "Inter, sans-serif", + }, +} + +# Templates path +templates_path = ["_templates"] + +# Exclude patterns +exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] + +# Suppress specific warnings that are known and harmless +suppress_warnings = [ + "myst.header", +] diff --git a/docs/dev/index.md b/docs/dev/index.md new file mode 100644 index 0000000..d2593a0 --- /dev/null +++ b/docs/dev/index.md @@ -0,0 +1,340 @@ +# Developer Guide + +How appenv works internally and how to contribute. + +## Architecture + +How appenv's components fit together and why. + +### Design Philosophy + +appenv is a single-file Python CLI that pins packages to exact versions and exposes their binaries via symlinks, using [uv](https://docs.astral.sh/uv/) for environment management. The single-file constraint is deliberate: appenv gets copied into project repositories as a self-contained bootstrap script with zero runtime dependencies. Commit it alongside `pyproject.toml` and `uv.lock`, and every checkout — local or on a remote deployment target — gets the same tools at the same versions by running `./http` or `./batou`. + +This shapes every architectural decision: + +- **Zero system modification**: appenv never installs anything outside the project directory. All state lives in `.appenv/` — venv, cached uv binary, logs. No system packages, no global bin directories, no PATH modifications. Drop the script, remove `.appenv/`, and the system is unchanged. +- **Symlink dispatch**: When invoked as `./http` (a symlink to `appenv`), it prepares the venv and runs `.appenv/venv/bin/http`. When invoked as `./appenv`, it parses subcommands via argparse. The script detects its own filename (`Path(__file__).stem`) to choose the mode. Multiple symlinks can coexist to expose different binaries from the same venv. +- **Type hints in stubs only**: Implementation lives in `appenv.py` with minimal typing. Complete type annotations (public and private methods) live in `appenv.pyi`. Any signature change must update both files. See [](#type-annotations) for the full policy. +- **Conditional tomllib**: `init` uses `tomllib` (stdlib, Python 3.11+) to parse existing `pyproject.toml` when a `[project]` section already exists. The import is guarded with try/except — no third-party dependency. On Python 3.9–3.10, `init` falls back to refusing to modify an existing `[project]` section. This is a conscious trade-off: full functionality on modern Python, zero cost on older interpreters. +- **Guard-then-act pattern**: `ensure_*` functions validate preconditions and exit with specific error codes if unsatisfied. The main flow only proceeds after all guards pass. + +### Command Dispatch + +The entry point `main()` does three things: + +1. **Clear PYTHONPATH** — prevents host environment contamination in the venv +2. **Select best Python** via `ensure_best_python()` (see [](#python-version-selection)) +3. **Dispatch based on filename**: + - Filename is `appenv` → `meta()` (argparse subcommand handling) + - Filename is anything else → `run()` (exec the venv binary) + +#### Run Mode + +`./http arg1 arg2` → appenv prepares the venv, then `os.execv()` replaces the current process with `.appenv/venv/bin/http arg1 arg2`. The original Python process is gone — no wrapper, no subprocess overhead. + +#### Meta Mode + +`./appenv ` → argparse dispatches to handler methods grouped by category (Project, Venv, Tools, Debug). Subcommands are defined in `AppEnv.meta()` with `func` defaults pointing to handler methods. + +### Python Version Selection + +(python-version-selection)= + +`ensure_best_python()` runs before any other setup. It reads `requires-python` from `pyproject.toml`, scans PATH for `python3.X` binaries (newest first), and re-execs with the best match via `os.execv()`. A guard (`APPENV_BEST_PYTHON` env var) prevents infinite re-exec loops. + +**Base directory**: `APPENV_BASEDIR` overrides the project location. By default, appenv uses `Path(__file__).parent` — meaning appenv must be located next to `pyproject.toml`. This is intentional: the bootstrap script lives beside the project it manages. + +If no compatible Python is found, it lists available versions and exits with `EXIT_CODE_DATAERR` (65). + +Each candidate is probed by running `python -c "print(1)"` to verify the binary actually works — important on systems like NixOS where symlink targets may have been garbage-collected. + +### uv Management + +#### Discovery Chain + +`UvBin` discovers the `uv` binary through a six-step cascade. Each step validates the version before accepting — see `UvVersion.minimum()` in the API reference for the current minimum: + +1. **PATH** — `shutil.which("uv")`, validate version +2. **Cached binary** — `.appenv/.uv/bin/uv` from a previous nix/pip install +3. **Nix channel** — `nix-build -A uv` into `.appenv/.uv` +4. **Nix flake** — `nix build nixpkgs#uv` into `.appenv/.uv` (more expensive, fresher packages) +5. **pip install** — `pip install uv -t .appenv/.uv` as last resort +6. **Direct download** — download the uv binary tarball from [astral-sh/uv GitHub releases](https://github.com/astral-sh/uv/releases) via `urllib` + `tarfile` (stdlib only), extract to `.appenv/.uv/bin/uv`. See [](#platform-detection) for supported platforms. + +When a PATH uv is valid, any previously cached `.appenv/.uv` is cleaned up automatically. The cascade handles environments where uv may not be pre-installed (CI, NixOS, minimal containers). + +#### Platform Detection + +(platform-detection)= + +The direct-download step (6) needs to know the correct release archive for the current platform. `_uv_platform_triple()` in `src/appenv.py` maps `platform.machine()` and `sys.platform` to the archive naming convention used by astral-sh/uv: + +| Architecture | Linux (glibc) | Linux (musl/Alpine) | macOS | +| ------------ | -------------------------- | -------------------------- | --------------------- | +| x86_64 | `x86_64-unknown-linux-gnu` | `x86_64-unknown-linux-musl` | `x86_64-apple-darwin` | +| aarch64 | `aarch64-unknown-linux-gnu` | `aarch64-unknown-linux-musl` | `aarch64-apple-darwin` | +| armv7 | `armv7-unknown-linux-gnu` | `armv7-unknown-linux-musl` | — | + +Linux glibc vs. musl is detected by checking for `/etc/alpine-release` (Alpine uses musl). Unsupported platforms (unrecognized architecture or OS) cause the direct-download step to be skipped silently. + +#### Version Enforcement + +`ensure_uv()` wraps `UvBin` construction and exits with `EXIT_CODE_UNAVAILABLE` (68) if the discovered binary doesn't meet the minimum version. This guard runs before any venv operations. + +### Venv Lifecycle + +The venv lives at `.appenv/venv` — a real virtual environment managed by uv. `.venv` is a symlink to `.appenv/venv` for IDE and tool compatibility (editors, linters, debuggers that expect `.venv` by convention). + +#### Creation + +`_prepare_venv()` handles the full lifecycle: + +1. Run guards: `ensure_pyproject()`, `ensure_lock_file()`, `ensure_uv()` +2. Set `UV_PROJECT_ENVIRONMENT` to `.appenv/venv` so uv targets the right directory +3. **Corruption recovery**: if `.appenv/venv` exists but `bin/python` is missing (NixOS garbage collection), the venv is removed and recreated +4. **Stale version recovery**: if the venv's Python version doesn't satisfy `requires-python` (e.g., after a constraint change), the venv is removed and recreated with a one-line message to stdout +5. Create venv with `uv venv --python ` — explicitly uses the current Python to prevent uv from downloading its own (which breaks on NixOS) +6. Sync production dependencies via `uv sync --no-dev --frozen` +7. Update `.venv` symlink (removed and recreated if stale) + +#### Sync Modes + +- **Production** (`prepare`, symlink dispatch): `uv sync --no-dev --frozen` — only production dependencies, lockfile must exist and be unchanged. +- **Run** (`run`): Delegates directly to `uv run` with appenv's configured environment. Does not enforce frozen — uv run handles its own sync. + +### Project Layout Conventions + +appenv expects specific files relative to the project root. Paths are conventions, not configuration: + +`pyproject.toml` +: Project definition with `[project]` section and `requires-python`. Required for all operations. + +`uv.lock` +: Dependency lockfile created by `./appenv update-lockfile`. Required before `prepare`. + +`appenv` +: The bootstrap script — a copy of `src/appenv.py`. `migrate` updates this file when the running appenv version differs from the one on disk. `init` warns when a version mismatch is detected. + +: Symlink to `appenv`. Running `./` executes the `` binary from the installed dependencies. See [](#command-dispatch) for how dispatch works. + +`.appenv/` +: Internal state directory (venv, cached uv binary, logs). Managed entirely by appenv. + +`.venv` +: Symlink to `.appenv/venv`. Created for tool compatibility — do not delete manually. + +### Error Handling Strategy + +appenv uses BSD sysexits.h exit codes to communicate specific failure modes. See {ref}`exit-codes` for the complete reference. + +### Logging Architecture + +Each command gets its own log file at `.appenv/logs/.log`. Logs use `TimedRotatingFileHandler` with daily rotation and 7-day retention. + +Verbose mode (`APPENV_VERBOSE=1`) adds a console handler with dimmed caller info (`funcName:lineno`) prepended to each message. Useful during development without cluttering normal output. + +The logger is a module-level `logging.getLogger("appenv")` singleton, configured per-command by `setup_logging()`. All components use it for structured debug output. + +## Contributing + +How to set up a development environment and submit changes. + +### Development Setup + +```console +git clone https://github.com/flyingcircusio/appenv.git +cd appenv +``` + +Repository layout: + +``` +. +├── src/appenv.py # The entire tool — single file +├── tests/ # Test suite +├── docs/ # Sphinx documentation +│ ├── user/ # User-facing docs (commands, workflows) +│ └── dev/ # Developer guide (this page) +├── pyproject.toml # Project config, dependencies, tool settings +└── uv.lock # Pinned dependency versions +``` + +Run tests: + +```console +uv run pytest +``` + +All CI checks (lint, format, type-check, test) run via: + +```console +tox +``` + +### Code Style + +appenv follows the style defined in `pyproject.toml` under `[tool.ruff]`. Run `uv run ruff check --fix .` and `uv run ruff format .` to apply. + +#### Type Annotations + +(type-annotations)= + +Type annotations live in `.pyi` stub files, not in `.py` source files. The `src/appenv.pyi` stub is the complete type surface — it must include all public *and* private methods. Ruff's `ANN` rules are dropped because they ignore `.pyi` files entirely. + +Any method signature change requires updating both `src/appenv.py` and `src/appenv.pyi`. Validate with `uv run ruff check --select PYI src/appenv.pyi`. + +The `src/py.typed` marker file signals PEP 561 compliance to type checkers. + +Test files follow the same stub-only policy — every `.py` in `tests/` has a matching `.pyi`. See [](#test-type-stubs) for patterns and fixture types. + +#### Exit Codes + +Use BSD sysexits.h constants (`EXIT_CODE_USAGE`, `EXIT_CODE_DATAERR`, `EXIT_CODE_NOINPUT`, `EXIT_CODE_UNAVAILABLE`) defined at module level. See [](#error-handling-strategy) for the full error handling strategy. + +#### Spec Comments + +Non-trivial error handling uses `# SPEC:` comments to trace requirements: + +```python +# SPEC: SRS-F001-cmd-wrapper - Enrich subprocess errors with command output context +try: + result = subprocess.check_output(cmd_list, stderr=subprocess.STDOUT) +except subprocess.CalledProcessError as e: + raise ValueError(e.output.decode("utf-8", "replace")) from e +``` + +### Running Tests + +```console +# All tests (excludes slow by default) +uv run pytest + +# Specific file +uv run pytest tests/test_prepare.py + +# With coverage report +uv run pytest --cov=appenv --cov-report=term-missing + +# Include slow tests +uv run pytest -m '' +``` + +#### Two-Tier Model + +appenv has no natural seam for an integration tier — `uv` is either mocked (unit) or real (E2E). Tests fall into exactly two tiers: + +**Unit tests** (`tests/test_*.py`) +: Mock `uv` via `MockUvBin`. Fast, no external dependencies. Covers logic branches, error handling, and argument construction. + +**E2E tests** (`tests/integration/`) +: Real `uv`, real subprocess via `pexpect`. Exercises the full bootstrap workflow end-to-end. Requires `uv` installed on the system. + +The standard 70/20/10 pyramid model does not apply — test expansion beyond unit coverage is E2E-only. + +#### Slow Marker + +Any test consistently exceeding 2 seconds gets `@pytest.mark.slow`. The threshold is objective: measure new tests and apply the marker if they cross it. + +- Default `pytest` configuration excludes slow tests (`-m "not slow"`) +- `tox` cov environment runs all tests including slow +- E2E tests should target under 2s to avoid needing the marker + +(test-type-stubs)= +#### Test Type Stubs + +Test stubs live alongside their `.py` files in `tests/`. Every test function, helper, and class has a corresponding stub entry. Markers (`@pytest.mark.slow`, `@pytest.mark.parametrize(...)`) are preserved in stubs. + +**Signature patterns:** + +```python +def test_example(monkeypatch: MonkeyPatch, tmp_path: Path) -> None: ... +def helper(verbose: bool = ...) -> None: ... +``` + +**Builtin pytest fixture types:** + +| Fixture | Type | +|---------|------| +| `monkeypatch` | `MonkeyPatch` | +| `capsys` | `CaptureFixture[str]` | +| `tmp_path` | `Path` | +| `request` | `FixtureRequest` | +| `caplog` | `LogCaptureFixture` | + +**Project fixture types** (from `tests/conftest.pyi`): + +| Fixture | Return type | +|---------|-------------| +| `workdir` | `Path` | +| `mock_uv` | `MockUvBin` | +| `mock_uv_version` | `None` | +| `subprocess_run_fail` | `None` | +| `app_env` | `Callable[..., AppEnv]` | +| `make_mock_uv` | `Callable[..., MockUvBin]` | +| `no_ensure_python` | `None` | +| `mock_cmd_python` | `None` | +| `create_venv` | `Callable[..., Path]` | +| `mock_uv_lock` | `None` | +| `mock_logdir` | `Path` | +| `clean_uv_project_env` | `None` | +| `make_pyproject` | `Callable[..., None]` | +| `capture_appenv_logs` | `logging.Logger` | +| `test_settings` | `Callable[..., AppEnvSettings]` | +| `patterns` | pytest-patterns plugin type | + +**Validation:** + +```console +uv run ruff check --select PYI tests/ +uv run ty check tests/ +``` + +Stubs must stay in sync with their `.py` counterparts — any signature change updates both files. Ty runs on `src/` only in CI; test stub validation is manual until a follow-up enables it. + +#### Test Strategy + +- **Existing code with gaps**: write tests first (tests-after) to cover uncovered paths +- **New features with code changes**: write tests alongside or before implementation +- **Existing features with no code changes** (e.g., untested subcommands): write E2E tests first (E2E-first) since the feature already works + +### Quality Gates + +All of these must pass before submitting a PR: + +| Gate | Command | What it checks | +|------|---------|----------------| +| Lint | `uv run ruff check .` | Code quality rules | +| Format | `uv run ruff format --check .` | Formatting consistency | +| Types | `uv run ty check .` | Type correctness via `.pyi` stubs | +| Dead code | `uv run vulture .` | Unused code detection | +| Tests | `uv run pytest` | All tests pass | +| Full CI | `tox` | All environments (fix, cov, multiple Python versions) | + +Pre-commit hooks run these automatically. + +### Documentation + +Docs are built with Sphinx using MyST markdown and autoapi: + +```console +tox -e docs +``` + +- **User docs**: `docs/user/` — usage and workflows +- **Dev docs**: `docs/dev/` — this guide +- **API reference**: auto-generated from source by autoapi — do not write API docs by hand + +### Pull Request Process + +1. Fork and create a feature branch +2. Make changes with accompanying tests +3. Run `tox` — all environments must pass +4. Update documentation if behavior changed +5. Submit pull request + +**PR checklist:** + +- [ ] `tox` passes cleanly +- [ ] New behavior has tests +- [ ] Documentation updated if applicable +- [ ] No `# noqa` without justification diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..52ea436 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,32 @@ +# appenv + +appenv is a single Python file that pins packages to exact versions and exposes +their binaries via symlinks. Drop it into a repository, commit it, and every +checkout gets the same tools at the same versions — locally and on remote machines. + +Built on [uv](https://docs.astral.sh/uv/) for environment management. +appenv never modifies your system — all state lives in `.appenv/` inside the +project directory. Remove that folder and nothing is left behind. + + +## Core Concepts + +- **Single-file deployment**: `appenv.py` is the entire tool — drop it into any repository +- **Symlink dispatch**: `./http` (where `http → appenv`) runs the `http` binary from the pinned venv +- **Reproducible everywhere**: commit `appenv`, `pyproject.toml`, and `uv.lock` — every checkout gets identical versions +- **Multiple binaries**: create additional symlinks to expose more tools from the same venv + +Requires Python 3.9+ (managed environments need 3.10+). [uv](https://docs.astral.sh/uv/) 0.5.0+ is auto-installed if not found. `pyproject.toml` must sit next to the appenv script. + +## Getting Started + +New to appenv? Start with the {doc}`user/workflows` — practical examples for setting up +a project, adding dependencies, and integrating with CI/CD. + +For reference material, see {doc}`user/commands` and {doc}`user/locking-behavior`. + +```{toctree} +:hidden: +user/index +dev/index +``` diff --git a/docs/llms.txt b/docs/llms.txt new file mode 100644 index 0000000..163b467 --- /dev/null +++ b/docs/llms.txt @@ -0,0 +1,91 @@ +# appenv Documentation + +Self-contained bootstrapping and updating of Python CLI applications using pyproject.toml and uv. + +**Test Coverage: 91%** - Tier 1: Full Documentation Priority + +## Overview + +appenv is a single-file Python application bootstrapping mechanism for CLI applications. It manages virtual environments using uv and supports pyproject.toml-based workflows. + +Requirements: +- Python 3.10 or later +- uv 0.5.0 or later (auto-installed if not found) + +## User Guide + +### Installation +- Bootstrap Script: `curl -sL https://github.com/flyingcircusio/appenv/raw/master/bootstrap | sh` +- Clone Repository: `git clone https://github.com/flyingcircusio/appenv.git` +- Copy appenv.py: `curl -sL https://github.com/flyingcircusio/appenv/raw/master/src/appenv.py -o appenv` + +### Quick Start +- Interactive setup with `./appenv init` +- First run automatically installs dependencies +- Automatic Python version selection based on requires-python + +### Commands +| Command | Description | +|---------|-------------| +| `./appenv prepare` | Create venv with production deps | +| `./appenv update-lockfile` | Update uv.lock | +| `./appenv reset` | Remove virtual environment | +| `./appenv python` | Start Python REPL | +| `./appenv run