Skip to content

beast host sdh disk hardware failure kills long-running Incus container processes mid-boot #871

Description

@ryanbreen

What

While proving the #865 host-wide x86 QEMU lock on beast (container breenix-x86), every docker/qemu/run-x86-boot-tests.sh 1 / run-x86-prod-profile-boot-test.sh invocation that ran past roughly a minute of wall-clock got silently SIGKILLed -- the whole process tree (the bash script AND its qemu-system-x86_64 child) vanished from ps with no ERR-trap output, no FAIL line, no facts line, mid-write to its own serial capture. This happened 4 times across 2 independent gate scripts, at 4 different points in each boot's own sequence ([init] Breenix init starting (PID 1), and three separate times right after a [RECLAIM_DRAIN:...] line deep in the userspace test cohort) -- ruling out any single code-level marker or line as the trigger.

Evidence

sudo journalctl -k --since "<window>" on the beast HOST (not the container) shows an ACTIVE, ongoing hardware fault on physical disk sdh at the same times as every one of the 4 deaths, e.g.:

Sep 06 06:48:51 beast kernel: device offline error, dev sdh, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Sep 06 06:48:51 beast kernel: Buffer I/O error on dev sdh, logical block 0, async page read
Sep 06 06:48:51 beast kernel: ldm_validate_partition_table(): Disk read failed.
...
Sep 06 06:49:18 beast kernel: sd 4:0:3790:0: Power-on or device reset occurred

and, ~10 minutes earlier:

Sep 06 06:38:56 beast kernel: sd 4:0:3785:0: attempting task abort!scmd(0x00000000ac59a87e), outstanding for 30317 ms & timeout 30000 ms

A 5-minute journalctl window at the time of investigation carried 88 lines mentioning sdh. This is a real, ongoing physical-disk problem on the beast host, not a container- or Breenix-scoped issue -- it can affect any Incus container's workload on this host, not only breenix-x86.

Impact

Any beast-hosted gate/build whose runtime crosses a bad I/O window is at risk of being silently killed mid-run with zero diagnostic output (a hard SIGKILL bypasses every set -e/ERR-trap/EXIT-trap in the killed script, so nothing it would normally print on failure gets a chance to run). This is a false-negative risk for every x86 gate on this host, independent of any Breenix code change.

Ask

Beast host storage health needs operator attention (likely a failing/failed drive in the array backing sdh -- Power-on or device reset occurred and task abort cycles are typical pre-failure SMART-adjacent symptoms, not a transient blip). Not something a Breenix PR can fix; filing so it's tracked and the beast-hosted gate battery's next red isn't mis-attributed to code.

Not a Breenix code defect

The #865 host-wide x86 QEMU lock itself worked correctly across 2 independent concurrent-invocation trials run on this same host before this disk issue's proof was pinned down: a second invocation correctly blocked on QEMU HOST LOCK: waiting for /root/.cache/breenix/x86-qemu.lock for 90s and again for 420+s (9+ distinct 30s-interval wait messages) while a first invocation held the lock, and correctly acquired and booted once the first released it. This issue is scoped to the host's storage hardware, not the lock or gate logic.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions