Push-to-talk, a native macOS engine, and the placement work behind them

.github/workflows/ci.yml

@@ -88,3 +88,46 @@ jobs:
       # does first and no unit test does.
       - name: the entry point starts
         run: uv run flow --help
+
+  # The one part of Flow that is not Python, on the only runner that can compile it.
+  #
+  # `native/flow_stt.swift` is the macOS on-device decoder — the answer for machines that
+  # cannot reach huggingface.co, where faster-whisper's weights are the only official
+  # copy. It is written on Windows, where there is no Swift toolchain, so without this
+  # leg the compiler is a person on a laptop pasting errors back. The first version was
+  # exactly that and shipped with a hard one in it: Swift allows top-level statements
+  # only in a file called `main.swift`, and this is not one.
+  #
+  # Compile only, and that is the honest limit of what CI can say here. Actually
+  # *running* it needs a granted Speech Recognition permission and an on-device model
+  # that only arrives when a human enables Dictation — neither of which a headless runner
+  # has, and faking them would make this leg green about something it never checked.
+  # `--probe` would exit 2 on the runner for exactly that reason, which is the right
+  # answer and a useless test.
+  helper:
+    name: swift helper compiles
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+
+      - name: swiftc version
+        run: xcrun swiftc --version
+
+      # `-warnings-as-errors` because this file is edited by people who cannot run it.
+      # A warning here is the only signal that reaches them before a user does.
+      - name: compile the macOS decoder
+        run: xcrun swiftc -O -parse-as-library -warnings-as-errors -o /tmp/flow-stt native/flow_stt.swift
+
+      # It links against AVFoundation and Speech, so a binary that builds but cannot
+      # resolve a symbol is still a broken helper. Asking it for its usage line proves
+      # the dynamic linker is satisfied without needing a microphone or a permission.
+      - name: it links and runs far enough to refuse
+        run: |
+          set +e
+          out=$(/tmp/flow-stt --nonsense 2>&1)
+          code=$?
+          echo "$out"
+          # 1 is `die("usage: ...")`. Anything else — a link failure, a crash, or a
+          # silent success on an argument that is not valid — is a broken build.
+          if [ "$code" -ne 1 ]; then echo "expected exit 1, got $code"; exit 1; fi
+          case "$out" in *usage*) ;; *) echo "expected a usage line"; exit 1 ;; esac

README.md

@@ -52,10 +52,16 @@ mode are all there, unchanged. What changes is the last inch: **Send copies the
 you press Ctrl+V yourself.**
 
 Four things it does not do — exclusions, not gaps: no injection into another
-application's window, no global hotkeys (arming is a click on the pill), no auto-paste,
-and no target-window awareness. What that buys is the property full Flow cannot have:
-**nothing to grant but the microphone** — no accessibility permission, no input
-monitoring, no trusted-application prompt.
+application's window, no global hotkeys, no auto-paste, and no target-window awareness.
+What that buys is the property full Flow cannot have: **nothing to grant but the
+microphone** — no accessibility permission, no input monitoring, no trusted-application
+prompt.
+
+**Push-to-talk works here anyway, and it does not need a hotkey.** Hold the pill, speak,
+let go — the words land on your clipboard. A quick click still toggles listening, and
+dragging still moves the pill. It is the same gesture Windows gets from `ctrl+win`, on a
+button Flow already draws, which is why it costs no permission: a system hotkey is the
+part that needs Accessibility and Input Monitoring, and this is not one.
 
 Two requirements Lite cannot meet, named rather than dropped: P7 (safe paste into a
 terminal) is a promise about a paste Flow performs, and Lite performs none; and P9's loop
@@ -158,7 +164,7 @@ where Flow mishears them. That is the one thing I cannot measure alone.
 ```bash
 git clone https://github.com/samartomar/flow && cd flow
 uv sync && uv run flow                          # run it
-uv run python -m unittest discover -s tests     # 1,881 tests, ~40 s, no mic needed
+uv run python -m unittest discover -s tests     # 1,965 tests, ~42 s, no mic needed
 uv run python scripts/selfdrive.py              # the end-to-end harness
 ```
 

docs/analysis.md

@@ -173,9 +173,15 @@ Guards that enforce R11 ("no heavy lifting"), **as built**:
 - The **input device** is health-checked every 5 s and reopened if it dies. The decode
   worker instead swallows and reports per-decode exceptions, so it cannot die and needs
   no restart.
-- Idle > 5 min → **unload the model only**. The mic stays open, a deliberate narrowing
+- Idle > 30 min → **unload the models only**. The mic stays open, a deliberate narrowing
   of the "release the mic" idea above: releasing it would leave the app unable to hear
-  its own wake-up, and the mic is cheap while the model is 141 MB.
+  its own wake-up, and the mic is cheap while the models are ~605 MB (`base.en` 141 MB
+  for partials plus `small.en` 464 MB for finals — this line said "the model is 141 MB"
+  while there were two tiers resident, understating its own case fourfold). Was 5 min,
+  which sat *inside* the gaps of an ordinary session: the common case was not reclaiming
+  memory from somebody who had left, it was paying a reload in the middle of their first
+  sentence back. The chord now also warms on press-down, so the load happens during the
+  hold rather than inside the first utterance.
 - Undo history is bounded by **both** snapshot count and total characters, since 30
   copies of a long draft is where undo quietly becomes megabytes.
 

docs/architecture.md

@@ -1500,14 +1500,14 @@ Only the ones with a measurement or a failure behind them. Everything else is in
 |---|---|---|
 | `MAX_UTTERANCE_SEC` | 24.0 s | Whisper pads to one 30 s mel window, so cost is flat below it and climbs past it. Cut before the boundary keeps latency constant in a long session |
 | `PARTIAL_MIN_GROWTH_SEC` | 0.7 s | Paired with the worker-idle check, this is what bounds partial latency |
-| `IDLE_UNLOAD_SEC` | 300 s | Release the models, keep the mic. Releasing the mic would leave the app unable to hear its own wake-up |
+| `IDLE_UNLOAD_SEC` / `WARM_GRACE_SEC` | 1800 s / 60 s | Release the models (~605 MB: `base.en` 141 + `small.en` 464), keep the mic. Releasing the mic would leave the app unable to hear its own wake-up. 1800 rather than the original 300 because five minutes is inside the gaps of a working session, so the unload's common case was a reload in the middle of somebody's first sentence back rather than memory reclaimed from somebody gone. The grace is what the chord's press-down buys: `Session.warm` starts the load during the hold, and holds off the idle unload for a minute so the health pump cannot drop the models between the press and the release that arms. A window rather than a touch of `_last_activity`, because `ctrl+win` is also Windows' `ctrl+win+arrow` prefix and letting a desktop switch reset the idle clock would retire the unload for anyone who uses virtual desktops |
 | `STALL_SEC` | 2.0 s | The liveness *backstop*: how long PortAudio may go without handing over a block before the stream counts as dead whatever it says about itself. Not a silence heuristic — what is timed is delivery, and a quiet room delivers blocks at exactly the rate a loud one does. Measured here: the gap between callbacks is a median **63.0 ms** (p99 79 ms, max 79 ms) idle, and a worst case of **453 ms** over ~109 s under four CPU-bound threads standing in for a decode. 2.0 s is 4.4x that worst case and 31 block periods; a false positive tears down a working stream mid-sentence, so the bias is toward late. The actual detector is `Pa_IsStreamActive`, polled every frame at **0.43 us** a call |
 | `MIC_RETRY_SEC` | 1.0 s | Spacing between reopen attempts. Measured: `sd._terminate()` + `sd._initialize()` costs **12.2 ms** (11.5-13.8 over six rounds), the open behind it **20.6 ms** (20.1-23.0), and the first block arrives **111-266 ms** after the open begins — a full `Mic.restart()` on real hardware measured **103 ms**. So a second is roughly 4x the worst end-to-end cost of an attempt: long enough that a retry is a fresh chance at a device still settling rather than the same failure re-timed, short enough that the whole ordeal fits inside `AUTO_ASK_SEC`. Spent as a deadline the pump checks, never as a sleep |
 | `MIC_RETRIES` | 3 | Attempts, counting the immediate one. The first covers the commonest failure, which is not a device dying but the *default moving* — a headset plugged in, the old stream orphaned, the replacement already there. The other two cover a device that is coming back but is not back yet (USB re-enumeration, a Bluetooth re-pair). Past ~2 s a device is not settling, and the honest end is the disarmed state a failed startup produces: pill off, reason on screen, clicking it opens fresh against whatever is plugged in by then. This replaced a 5 s heartbeat that reopened *forever*, one note and one error every five seconds, over a pill still claiming to be armed |
 | `FORCE_NEXT_TTL_SEC` | 30 s | A Refine/Continue chip means "the next thing I say"; after this long the next thing someone says is a different thought. The chips also toggle, because a one-way door that lasts 30 s reads as the app being stuck |
 | `AUTO_ASK_SEC` | 4 s | Converse mode only. Measured: the pauses a speaker leaves between separate spoken items run 1.4–3.3 s (median 2.5 s) on the one recording where every item was located, and each gap also contains a spoken item number, so real silence is shorter — under ~3.3 s fires mid-thought. R5 still holds where it matters: pasting into a window is irreversible and stays manual, asking is not |
 | `ui.SENT_LINGER_SEC` | 4 s | How long the bubble holds what a dictate-mode Send just handed over, with the chip that puts it back. Deliberately **not** `AUTO_ASK_SEC`, which is also 4 s and is a different four seconds: that one is how long a settled draft waits before asking itself, this is how long words stay recoverable after they have gone, and either could move without the other. The number it replaces was zero — the bubble was withdrawn on Send, so a Send that went nowhere and a Send that worked left the same empty screen |
-| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1600 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1600 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 56 (measured: 3 160 characters wrapped to 56 lines at 352 px) — an average rather than a layout, because the layout is the cost |
+| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1750 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1750 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 62 (re-measured at the shipped 392 px column: 3 160 characters wrapped to 51 lines; the earlier 56 was the same prose at the pre-Phase-6 352 px column) — an average rather than a layout, because the layout is the cost |
 | `ui.EDGE_AIR` | 8 px | Air between the bubble and every edge of the work area — one number, because the window is *fitted* to `work − 2 × air` and *clamped* by `air`, and those two have to agree. They did not: item 37's cap bounded the draft body and the reply path kept its full-text probe, so a 4 000-character answer sized the window **1 459 px** and a 12 000-character artifact **4 179 px** on a 672 px desktop, and `reposition` pinned both at `top + 8` and let the rest run off the bottom — **12 of 36 corner placements outside the work area**, chip row at screen y 1 427 and 4 147. Fitting the height in `_render` is what makes the clamp a guarantee rather than a best effort: **0 of 36** after, chip row at 624. The top edge was never the breach, at any corner or in any state |
 | `ui.DOT_SEC` | 0.4 s | One dot of the indeterminate-wait animation. The bubble renders on events and a wait has no events, so the frame is computed and compared before anything is drawn — at this cadence that is ~2.5 repaints a second instead of the 33 that redrawing every pump would cost. Same discipline as the auto-ask countdown |
 | `DEAF_DB` | −120.0 | What `level_db` reports while the microphone is not evidence. Below any real room — a quiet room with a good USB mic measures −96.7 dB — so every meter maps it to silence without having to know why |

docs/development.md

@@ -26,7 +26,7 @@ flow/
   hotkey.py    RegisterHotKey on its own message-loop thread (ctypes)
   diag.py      the wordless trace, and the identity block every benchmark records
 scripts/       benchmarks, probes, the soak test and the self-drive harness
-tests/         1,881 tests: routing, state machine, filters, phonetics, resilience
+tests/         1,965 tests: routing, state machine, filters, phonetics, resilience
 docs/          what Flow is for, the roadmap, the analysis, the recording kit
 ```
 
@@ -42,7 +42,7 @@ the event stream and the tuning constants with the measurements behind them.
 uv run python -m unittest discover -s tests
 ```
 
-1,881 tests, ~40 s, no microphone or model required — the fakes are injectable precisely so
+1,965 tests, ~42 s, no microphone or model required — the fakes are injectable precisely so
 the routing logic, where the subtle bugs live, can be tested without either.
 
 **The interpreter is pinned.** `.python-version` holds `3.12`, which is what CI installs and