Single-shot, hardware-aware autodE installer#5
Open
gabegomes wants to merge 1 commit into
Open
Conversation
One command (`./install.sh`) sets up autodE for the detected hardware — x86 CPU, x86 GPU (boltzmann), or ARM64 GPU (GH200) — with checkpoint/resume, rich (optional) UI, and backend auto-wiring. Bootstrap (`install.sh`): - prefers the shared cluster uv + micromamba on BeeGFS (works on no-egress nodes); tries both `aarch64` and `arm64` staged-dir names. - online → hands off under `uv run --with rich` (full UI); offline (GH200 compute nodes have no egress) → runs the driver with a system python and the driver degrades to plain output. `--no-project` so an ambient .venv can't hijack the interpreter. Driver (`installer/autode_install.py`): - detects arch / glibc / GPU (nvidia-smi) / egress; 8 checkpointed steps (`--resume` after failure, `--fix STEP`, `--only STEP`). - CPU env via micromamba; GPU env is a venv layered on the pyscf CUDA venv's site-packages via `.pth` (production skala recipe) so in-process GPU4PySCF imports; editable install recompiles the top-level `cconf_gen` C-extension per python+glibc+arch. - offline path: stdlib-free `uv venv` (no python3-venv/ensurepip needed) on a header-carrying base python + a staged aarch64 wheelhouse (`--no-index --find-links`, `--no-build-isolation`). - checks external ORCA (path/version) with manual-install guidance; wires UMA / GPU4PySCF / VeloxChem / g-xTB / MetalloGen for the hardware; writes an `activate-autode.sh` with all backend env vars. Validated end-to-end: x86 CPU (gpg-head), x86 GPU (boltzmann, incl. GPU4PySCF import + UMA + glibc-2.28 compile), ARM64 CPU offline (GH200 cpu-gh, wheelhouse + aarch64 compile). Resume/--fix/activation all exercised. ARM64 GPU queued. INSTALL.md documents prerequisites, where-to-run (egress+arch), backends, ORCA manual install, and resume/fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q2nSxXjYDpoBXiHDGRW9Mv
There was a problem hiding this comment.
Pull request overview
Introduces a new “single-shot” installation flow intended to set up autodE across heterogeneous cluster hardware (x86/ARM64, CPU/GPU) with resumable/checkpointed steps, optional rich UI, and automatic backend wiring.
Changes:
- Adds a Python installer driver (
installer/autode_install.py) implementing hardware detection, checkpoint/resume, environment creation, package installation, and backend wiring. - Adds a bootstrap shell entrypoint (
install.sh) that locates/bootstrapsuv+micromambaand runs the driver with rich UI when online. - Adds installation documentation (
INSTALL.md) and ignores installer runtime artifacts (.gitignore).
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| installer/autode_install.py | New installer driver: hardware detection, step engine with checkpoint/resume, env creation (micromamba/venv layering), install + verification, backend wiring, activation script generation. |
| install.sh | New bootstrap script to find/install uv + micromamba, detect egress, and hand off to the Python driver. |
| INSTALL.md | New user-facing documentation for the single-shot installer workflow and options. |
| .gitignore | Ignores installer state/log artifacts and related runtime directories. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+133
to
+135
| PY_FOR_ARCH = {"x86_64": "3.11", "aarch64": "3.12"} | ||
| CORE_DEPS = ["rdkit", "numpy>=1.26,<3", "networkx", "matplotlib", "pillow>=9.5.0", | ||
| "cython", "scipy", "loky", "mendeleev", "nvidia-ml-py3"] |
Comment on lines
+530
to
+541
| lines = ["#!/usr/bin/env bash", | ||
| "# autodE environment — source me. Generated by install.sh.", | ||
| f'export PATH="{env}/bin:$PATH"'] | ||
| if orca_dir: | ||
| lines += [f'export ORCA_DIR="{orca_dir}"', | ||
| f'export PATH="$ORCA_DIR:$PATH"'] | ||
| mpi = CLUSTER["openmpi_arm64" if c.hw.is_arm else "openmpi_x86"] | ||
| lines += [f'export PATH="{mpi}/bin:$PATH"', | ||
| f'export LD_LIBRARY_PATH="{mpi}/lib:${{LD_LIBRARY_PATH:-}}"'] | ||
| for _, kv in wired.items(): | ||
| for k, v in kv.items(): | ||
| lines.append(f'export {k}="{v}"') |
Comment on lines
+598
to
+621
| ap.add_argument("--no-editable", action="store_true", help="install autodE as a copy, not -e") | ||
| ap.add_argument("--skip-backends", action="store_true") | ||
| ap.add_argument("--yes", "-y", action="store_true", help="non-interactive (assume yes)") | ||
| ap.add_argument("--hf-token", default=os.environ.get("HF_TOKEN")) | ||
| a = ap.parse_args(argv) | ||
|
|
||
| repo = Path(a.repo).resolve() | ||
| prefix = Path(a.prefix).resolve() if a.prefix else repo | ||
| logs = prefix / "installer" / "logs" | ||
| logs.mkdir(parents=True, exist_ok=True) | ||
| state = State(prefix / ".autode_install_state.json") | ||
|
|
||
| hw = detect_hardware() | ||
| state.data["hardware"] = asdict(hw) | ||
| state.save() | ||
| banner(hw, prefix, a.resume or bool(a.fix)) | ||
|
|
||
| if not (a.yes or a.resume or a.fix or a.only): | ||
| if not Confirm.ask("Proceed with install?", default=True): | ||
| console.print("aborted."); return 1 | ||
|
|
||
| ctx = Ctx(repo=repo, micromamba=a.micromamba, hw=hw, state=state, prefix=prefix, | ||
| yes=a.yes, editable=not a.no_editable, skip_backends=a.skip_backends, | ||
| hf_token=a.hf_token, logs=logs) |
Comment on lines
+31
to
+33
| for t in bash uname; do need_cmd "$t" || MISSING+=("$t"); done | ||
| # a C toolchain is required for the Cython extensions | ||
| if ! need_cmd cc && ! need_cmd gcc; then MISSING+=("gcc/cc (C compiler)"); fi |
Comment on lines
+8
to
+10
| git clone https://github.com/gomesgroup/autode.git | ||
| cd autode | ||
| ./install.sh |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
One command —
./install.sh— sets up autodE correctly for whatever hardware you're on (x86 CPU, x86 GPU, or ARM64 GPU), with checkpoint/resume, a rich (optional) UI, and automatic backend wiring. Replaces the ad-hoc per-envmicromamba create+pip install -e+ manual.pyrsync + manual env-var wiring.How it works
install.sh): finds the shared clusteruv+micromambaon BeeGFS (works on no-egress nodes; tries bothaarch64/arm64dir names). Online → hands off underuv run --with rich; offline (GH200 compute nodes) → runs the driver on a system python and it degrades to plain output.installer/autode_install.py): detects arch/glibc/GPU/egress; 8 checkpointed steps (--resume,--fix STEP,--only STEP)..pth(production skala recipe) so in-process GPU4PySCF imports.cconf_genC-extension per python+glibc+arch (keeps boltzmann's glibc-2.28 build separate from the Ubuntu glibc-2.39 nodes).uv venv(nopython3-venv/ensurepip needed) on a header-carrying base python + a staged aarch64 wheelhouse (--no-index --find-links,--no-build-isolation).activate-autode.sh.Tested
--fix, activation.cpu-gh): full, incl. wheelhouse install + aarch64 compile with header-base fallback.gpu-gh): queued (nodes GPU-saturated); its two halves — offline mechanics and pyscf layering — are each validated above.🤖 Generated with Claude Code