diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f51a9cf..57ce8e5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -88,3 +88,46 @@ jobs: # does first and no unit test does. - name: the entry point starts run: uv run flow --help + + # The one part of Flow that is not Python, on the only runner that can compile it. + # + # `native/flow_stt.swift` is the macOS on-device decoder — the answer for machines that + # cannot reach huggingface.co, where faster-whisper's weights are the only official + # copy. It is written on Windows, where there is no Swift toolchain, so without this + # leg the compiler is a person on a laptop pasting errors back. The first version was + # exactly that and shipped with a hard one in it: Swift allows top-level statements + # only in a file called `main.swift`, and this is not one. + # + # Compile only, and that is the honest limit of what CI can say here. Actually + # *running* it needs a granted Speech Recognition permission and an on-device model + # that only arrives when a human enables Dictation — neither of which a headless runner + # has, and faking them would make this leg green about something it never checked. + # `--probe` would exit 2 on the runner for exactly that reason, which is the right + # answer and a useless test. + helper: + name: swift helper compiles + runs-on: macos-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + + - name: swiftc version + run: xcrun swiftc --version + + # `-warnings-as-errors` because this file is edited by people who cannot run it. + # A warning here is the only signal that reaches them before a user does. + - name: compile the macOS decoder + run: xcrun swiftc -O -parse-as-library -warnings-as-errors -o /tmp/flow-stt native/flow_stt.swift + + # It links against AVFoundation and Speech, so a binary that builds but cannot + # resolve a symbol is still a broken helper. Asking it for its usage line proves + # the dynamic linker is satisfied without needing a microphone or a permission. + - name: it links and runs far enough to refuse + run: | + set +e + out=$(/tmp/flow-stt --nonsense 2>&1) + code=$? + echo "$out" + # 1 is `die("usage: ...")`. Anything else — a link failure, a crash, or a + # silent success on an argument that is not valid — is a broken build. + if [ "$code" -ne 1 ]; then echo "expected exit 1, got $code"; exit 1; fi + case "$out" in *usage*) ;; *) echo "expected a usage line"; exit 1 ;; esac diff --git a/README.md b/README.md index f1c99da..5eff6c3 100644 --- a/README.md +++ b/README.md @@ -52,10 +52,16 @@ mode are all there, unchanged. What changes is the last inch: **Send copies the you press Ctrl+V yourself.** Four things it does not do — exclusions, not gaps: no injection into another -application's window, no global hotkeys (arming is a click on the pill), no auto-paste, -and no target-window awareness. What that buys is the property full Flow cannot have: -**nothing to grant but the microphone** — no accessibility permission, no input -monitoring, no trusted-application prompt. +application's window, no global hotkeys, no auto-paste, and no target-window awareness. +What that buys is the property full Flow cannot have: **nothing to grant but the +microphone** — no accessibility permission, no input monitoring, no trusted-application +prompt. + +**Push-to-talk works here anyway, and it does not need a hotkey.** Hold the pill, speak, +let go — the words land on your clipboard. A quick click still toggles listening, and +dragging still moves the pill. It is the same gesture Windows gets from `ctrl+win`, on a +button Flow already draws, which is why it costs no permission: a system hotkey is the +part that needs Accessibility and Input Monitoring, and this is not one. Two requirements Lite cannot meet, named rather than dropped: P7 (safe paste into a terminal) is a promise about a paste Flow performs, and Lite performs none; and P9's loop @@ -158,7 +164,7 @@ where Flow mishears them. That is the one thing I cannot measure alone. ```bash git clone https://github.com/samartomar/flow && cd flow uv sync && uv run flow # run it -uv run python -m unittest discover -s tests # 1,881 tests, ~40 s, no mic needed +uv run python -m unittest discover -s tests # 1,965 tests, ~42 s, no mic needed uv run python scripts/selfdrive.py # the end-to-end harness ``` diff --git a/docs/analysis.md b/docs/analysis.md index 73df480..5b94c59 100644 --- a/docs/analysis.md +++ b/docs/analysis.md @@ -173,9 +173,15 @@ Guards that enforce R11 ("no heavy lifting"), **as built**: - The **input device** is health-checked every 5 s and reopened if it dies. The decode worker instead swallows and reports per-decode exceptions, so it cannot die and needs no restart. -- Idle > 5 min → **unload the model only**. The mic stays open, a deliberate narrowing +- Idle > 30 min → **unload the models only**. The mic stays open, a deliberate narrowing of the "release the mic" idea above: releasing it would leave the app unable to hear - its own wake-up, and the mic is cheap while the model is 141 MB. + its own wake-up, and the mic is cheap while the models are ~605 MB (`base.en` 141 MB + for partials plus `small.en` 464 MB for finals — this line said "the model is 141 MB" + while there were two tiers resident, understating its own case fourfold). Was 5 min, + which sat *inside* the gaps of an ordinary session: the common case was not reclaiming + memory from somebody who had left, it was paying a reload in the middle of their first + sentence back. The chord now also warms on press-down, so the load happens during the + hold rather than inside the first utterance. - Undo history is bounded by **both** snapshot count and total characters, since 30 copies of a long draft is where undo quietly becomes megabytes. diff --git a/docs/architecture.md b/docs/architecture.md index 8418d2a..be465ca 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1500,14 +1500,14 @@ Only the ones with a measurement or a failure behind them. Everything else is in |---|---|---| | `MAX_UTTERANCE_SEC` | 24.0 s | Whisper pads to one 30 s mel window, so cost is flat below it and climbs past it. Cut before the boundary keeps latency constant in a long session | | `PARTIAL_MIN_GROWTH_SEC` | 0.7 s | Paired with the worker-idle check, this is what bounds partial latency | -| `IDLE_UNLOAD_SEC` | 300 s | Release the models, keep the mic. Releasing the mic would leave the app unable to hear its own wake-up | +| `IDLE_UNLOAD_SEC` / `WARM_GRACE_SEC` | 1800 s / 60 s | Release the models (~605 MB: `base.en` 141 + `small.en` 464), keep the mic. Releasing the mic would leave the app unable to hear its own wake-up. 1800 rather than the original 300 because five minutes is inside the gaps of a working session, so the unload's common case was a reload in the middle of somebody's first sentence back rather than memory reclaimed from somebody gone. The grace is what the chord's press-down buys: `Session.warm` starts the load during the hold, and holds off the idle unload for a minute so the health pump cannot drop the models between the press and the release that arms. A window rather than a touch of `_last_activity`, because `ctrl+win` is also Windows' `ctrl+win+arrow` prefix and letting a desktop switch reset the idle clock would retire the unload for anyone who uses virtual desktops | | `STALL_SEC` | 2.0 s | The liveness *backstop*: how long PortAudio may go without handing over a block before the stream counts as dead whatever it says about itself. Not a silence heuristic — what is timed is delivery, and a quiet room delivers blocks at exactly the rate a loud one does. Measured here: the gap between callbacks is a median **63.0 ms** (p99 79 ms, max 79 ms) idle, and a worst case of **453 ms** over ~109 s under four CPU-bound threads standing in for a decode. 2.0 s is 4.4x that worst case and 31 block periods; a false positive tears down a working stream mid-sentence, so the bias is toward late. The actual detector is `Pa_IsStreamActive`, polled every frame at **0.43 us** a call | | `MIC_RETRY_SEC` | 1.0 s | Spacing between reopen attempts. Measured: `sd._terminate()` + `sd._initialize()` costs **12.2 ms** (11.5-13.8 over six rounds), the open behind it **20.6 ms** (20.1-23.0), and the first block arrives **111-266 ms** after the open begins — a full `Mic.restart()` on real hardware measured **103 ms**. So a second is roughly 4x the worst end-to-end cost of an attempt: long enough that a retry is a fresh chance at a device still settling rather than the same failure re-timed, short enough that the whole ordeal fits inside `AUTO_ASK_SEC`. Spent as a deadline the pump checks, never as a sleep | | `MIC_RETRIES` | 3 | Attempts, counting the immediate one. The first covers the commonest failure, which is not a device dying but the *default moving* — a headset plugged in, the old stream orphaned, the replacement already there. The other two cover a device that is coming back but is not back yet (USB re-enumeration, a Bluetooth re-pair). Past ~2 s a device is not settling, and the honest end is the disarmed state a failed startup produces: pill off, reason on screen, clicking it opens fresh against whatever is plugged in by then. This replaced a 5 s heartbeat that reopened *forever*, one note and one error every five seconds, over a pill still claiming to be armed | | `FORCE_NEXT_TTL_SEC` | 30 s | A Refine/Continue chip means "the next thing I say"; after this long the next thing someone says is a different thought. The chips also toggle, because a one-way door that lasts 30 s reads as the app being stuck | | `AUTO_ASK_SEC` | 4 s | Converse mode only. Measured: the pauses a speaker leaves between separate spoken items run 1.4–3.3 s (median 2.5 s) on the one recording where every item was located, and each gap also contains a spoken item number, so real silence is shorter — under ~3.3 s fires mid-thought. R5 still holds where it matters: pasting into a window is irreversible and stays manual, asking is not | | `ui.SENT_LINGER_SEC` | 4 s | How long the bubble holds what a dictate-mode Send just handed over, with the chip that puts it back. Deliberately **not** `AUTO_ASK_SEC`, which is also 4 s and is a different four seconds: that one is how long a settled draft waits before asking itself, this is how long words stay recoverable after they have gone, and either could move without the other. The number it replaces was zero — the bubble was withdrawn on Send, so a Send that went nowhere and a Send that worked left the same empty screen | -| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1600 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1600 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 56 (measured: 3 160 characters wrapped to 56 lines at 352 px) — an average rather than a layout, because the layout is the cost | +| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1750 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1750 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 62 (re-measured at the shipped 392 px column: 3 160 characters wrapped to 51 lines; the earlier 56 was the same prose at the pre-Phase-6 352 px column) — an average rather than a layout, because the layout is the cost | | `ui.EDGE_AIR` | 8 px | Air between the bubble and every edge of the work area — one number, because the window is *fitted* to `work − 2 × air` and *clamped* by `air`, and those two have to agree. They did not: item 37's cap bounded the draft body and the reply path kept its full-text probe, so a 4 000-character answer sized the window **1 459 px** and a 12 000-character artifact **4 179 px** on a 672 px desktop, and `reposition` pinned both at `top + 8` and let the rest run off the bottom — **12 of 36 corner placements outside the work area**, chip row at screen y 1 427 and 4 147. Fitting the height in `_render` is what makes the clamp a guarantee rather than a best effort: **0 of 36** after, chip row at 624. The top edge was never the breach, at any corner or in any state | | `ui.DOT_SEC` | 0.4 s | One dot of the indeterminate-wait animation. The bubble renders on events and a wait has no events, so the frame is computed and compared before anything is drawn — at this cadence that is ~2.5 repaints a second instead of the 33 that redrawing every pump would cost. Same discipline as the auto-ask countdown | | `DEAF_DB` | −120.0 | What `level_db` reports while the microphone is not evidence. Below any real room — a quiet room with a good USB mic measures −96.7 dB — so every meter maps it to silence without having to know why | diff --git a/docs/development.md b/docs/development.md index 5ad34f3..c99b904 100644 --- a/docs/development.md +++ b/docs/development.md @@ -26,7 +26,7 @@ flow/ hotkey.py RegisterHotKey on its own message-loop thread (ctypes) diag.py the wordless trace, and the identity block every benchmark records scripts/ benchmarks, probes, the soak test and the self-drive harness -tests/ 1,881 tests: routing, state machine, filters, phonetics, resilience +tests/ 1,965 tests: routing, state machine, filters, phonetics, resilience docs/ what Flow is for, the roadmap, the analysis, the recording kit ``` @@ -42,7 +42,7 @@ the event stream and the tuning constants with the measurements behind them. uv run python -m unittest discover -s tests ``` -1,881 tests, ~40 s, no microphone or model required — the fakes are injectable precisely so +1,965 tests, ~42 s, no microphone or model required — the fakes are injectable precisely so the routing logic, where the subtle bugs live, can be tested without either. **The interpreter is pinned.** `.python-version` holds `3.12`, which is what CI installs and diff --git a/docs/guide.md b/docs/guide.md index 689d8b2..91b4f54 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -179,6 +179,34 @@ screen is the CLI that will be called, not the first one on PATH. `codex` now measures 6.6–8.5 s here for a one-word answer, so a long question can breach it. +### Which model, and how hard it thinks + +**Effort is `low` unless you say otherwise**, and that is the setting most worth knowing +about. These calls are a *rewrite* — take what was dictated and make it read like a +written prompt — not a reasoning problem, and the whole time one runs you are watching a +spinner between finishing a sentence and having your words. **Effort** in the right-click +menu offers every level the CLIs do: `low`, `medium`, `high`, `xhigh`, `max`. + +It reaches the CLIs that offer the choice, which is `claude` and `kiro-cli`. `codex` has +no effort flag — its only route is a config key that does not appear in its help, and +Flow will not write down a flag nobody has seen it print. + +Choosing a **model** takes one run of `--cli-model`: + +``` +uv run python -m flow --cli-model gpt-5.6-luna +``` + +After that it is a click, under **Model** in the same menu, alongside every other name +you have used and **The CLI's own default**. It has to arrive by flag once because no CLI +will list its models — `codex exec --help` says `-m, --model ` and stops — and +Flow has no text field to type one into. Settings is a menu, not a dialog. + +Both apply to whichever CLI answers, including a fallback: a walk that reverted to the +CLI's own defaults the moment the first candidate failed would be slowest exactly when +you are already waiting longest. A model name means nothing to a CLI that takes no +`--model`, and is dropped for that CLI rather than breaking the call. + ## Running it ```bash @@ -222,12 +250,14 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--final-model X` | stronger model for the pasted text (default `small.en`) | | `--model X` | pin BOTH tiers to one model, for a low-memory machine | | `--decode-device {auto,cuda,cpu}` | where decoding runs (default `auto`: the GPU when there is a working one) | +| `--engine {auto,whisper,native}` | which decoder. `whisper` is faster-whisper and needs model files; `native` is macOS on-device speech, which needs no download at all. Default `auto`: whisper unless its models are not on the machine and the native engine is ready — see [Without HuggingFace](#without-huggingface) | | `--lexicon PATH` | personal terms file (default `~/.flow/lexicon.txt`) | | `--no-lexicon` | ignore that file without deleting it | | `--device N` | input device index; list them with `scripts/devices.py`. **Pinned**: if it goes away mid-session Flow retries *this* index and never substitutes another — see [When the microphone goes away](#if-the-microphone-goes-away-mid-session) | | `--arm` | start listening immediately, no click needed | | `--no-paste` | print the draft to stdout instead of pasting it | | `--no-hotkeys` | skip global hotkey registration | +| `--no-chord` | skip the modifier-only chord, so no low-level keyboard hook is installed ([The chord](#the-chord-ctrlwin)) | | `--calibrate` | measure this room and this voice, store the profile, and exit ([P8](#calibration-p8)) | | `--no-profile` | ignore the stored profile and learn nothing this session | | `--converse` | start in converse mode: Send asks the agent CLI instead of pasting ([P9](#converse-mode-p9)) | @@ -235,9 +265,12 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--voice X` | voice for spoken replies: a name, part of one, or `male`/`female` | | `--no-auto-ask` | in converse mode, wait for the Ask button instead of a pause | | `--cli NAME` | pin the agent CLI (`codex` or `claude`) instead of trying each in turn | +| `--no-warm` | do not load the model at startup; wait until it is first needed | +| `--cli-model NAME` | ask the CLI for this model; remembered, and blank clears it | +| `--cli-effort LEVEL` | how hard the CLI may think: `low`, `medium`, `high`, `xhigh`, `max` (default `low`) | | `--cli-timeout SEC` | how long to wait for one CLI call (default 20) | | `--cwd PATH` | the project converse-mode questions are asked from; overrides the stored `workspace` ([P9](#converse-mode-p9)) | -| `--lite` | clipboard-out mode: Send copies the draft instead of pasting it, and no hotkeys are registered (automatic off Windows — see [Install](#install)) | +| `--lite` | no global hotkeys and no target-window tracking (automatic off Windows — see [Install](#install)). On Windows it also makes Send copy instead of paste; on a Mac Send still pastes, through System Events | | `--version` | print `flow X.Y.Z` and exit. The same number the startup block names, and the one Help shows at the bottom of the sheet | | `--check-update` | ask GitHub once whether a newer release exists, print one line, and exit. Manual only: nothing in Flow ever checks on its own, and the request carries no version, no identifier and no account — see [What leaves the machine](architecture.md#what-leaves-the-machine) | | `--stats` | print how much has been dictated — today and all time — and exit, without loading a model or opening the microphone. See [The numbers](#the-numbers). With `--no-profile` it reads nothing and says so | @@ -406,6 +439,297 @@ for you to fix. `--no-hotkeys` and [Lite](#install) register nothing with the OS, so a `hotkeys` block is not read on those launches and nothing is said about it. +### Hold the pill (every platform) + +**Hold the pill, speak, let go.** Same gesture as the chord below, on a button Flow +already draws rather than on a system hotkey — so it works in Lite, on macOS and Linux, +where there is no chord at all and nothing to grant but the microphone. + +One button, three gestures, each judged on what you actually did: + +| | | +|---|---| +| **Quick click** | toggles listening, as it always has | +| **Hold ~300 ms, then speak** | push-to-talk — release sends | +| **Press and move** | drags the pill somewhere else | + +Moving the mouse *while already talking* does not cancel anything: once capture is open +the pointer is irrelevant, and a sentence lost to a twitch would be the gesture betraying +you. The 300 ms and the 4 px of slop are `PILL_HOLD_SEC` and `PILL_DRAG_SLOP`. + +This also fixed something older: `_toggle` was bound to the button *press*, so **every +drag of the pill used to toggle listening on the way past**. A click is judged on +release now, like a button anywhere else. + +### The chord (ctrl+win) + +**Settings ▸ Chord (ctrl+win)** picks which of two gestures it is. Both ship, because +neither replaces the other, and you can switch between them while Flow is running — the +change takes effect on the next press, with no restart. + +| | | +|---|---| +| **Push to talk - hold to speak, release to send** | The shipped default. Press both keys, speak while they are down, let go. What you said is pasted into whatever window has focus — no third key, no clicking anything, no second shortcut to send. | +| **Toggle - press to start, press again to stop** | The original. A clean press-and-release starts listening and the next one stops it, exactly like the `toggle` hotkey. | + +**Pick the hold for a sentence and the toggle for a paragraph.** The hold needs no +decision about when you are finished and cannot leave a microphone running by accident. +The toggle is the only one of the two that survives a long thought with pauses in it, a +phone call you are transcribing, or hands that would rather not hold two keys down for a +minute. It is remembered in `profile.json` as `"gesture": "toggle"`. + +The rest of this section describes the hold, which is the default. + +The press-down does two things at once: it opens the microphone, and it starts loading +the models if they had been idle long enough to be released. That second half is why the +gesture is a *hold* rather than a tap — the second or so you spend pressing the keys and +drawing breath is the second the models need to come back, so it costs nothing instead of +landing in the middle of your first sentence. + +It exists because every combo in the table above ends in a key. `RegisterHotKey`, the +Windows call behind them, takes a virtual key and has no way to say "these modifiers and +nothing else" — so `ctrl+alt+space` is a shape your left hand makes *plus* a reach. Ctrl +and Win are neighbours, and holding two neighbours is one movement. + +**The `toggle` hotkey still works and still toggles.** The two are for different things +now rather than being two doors to the same room: hold the chord for a sentence, press +`ctrl+alt+space` for a paragraph you want to say hands-free. + +#### What happens when Windows wants ctrl+win too + +Windows uses ctrl+win as a prefix — `ctrl+win+d` makes a virtual desktop, `ctrl+win+←/→` +switch between them. Those still work, and Flow gets out of the way as soon as it sees +the third key: the capture stops on that keystroke, and **nothing is ever pasted**. + +Under the old toggle gesture Flow could refuse these outright, because nothing had +started until you let go. Push-to-talk opens the microphone on the press-down, so a +desktop switch does open it for a moment. Two things bound what that costs. The +microphone closes on the third key rather than at the release, so holding the keys +through several desktop switches does not record them. And if you *had* already started +speaking when the third key landed, your words are kept — they go to the draft, where the +Send chip picks them up — rather than being thrown away or pasted somewhere you did not +mean. + +Adding a modifier is a different chord and starts nothing at all: ctrl+shift+win does +nothing here. + +#### The two things that stop on their own + +Both are ceilings on a failure, not limits on you: + +- **A hold of two minutes ends itself.** A key release can genuinely go missing — a lock + screen or a remote-desktop session takes the keyboard mid-hold, or Windows drops the + keyboard hook for being slow. Without a ceiling that leaves a microphone open. What you + said is kept and put on screen; it is not pasted, because two minutes later you are + somewhere else. +- **A paste waits up to fifteen seconds for the decode.** Transcribing the final takes + from under a second to several, so the release cannot paste immediately — it waits. + Past fifteen seconds Flow stops waiting and tells you the words are in the draft. It + never pastes late: text arriving a minute after the gesture would land in whatever + window you had moved to. + +Change it or turn it off with `chord` in `~/.flow/profile.json`, beside the `hotkeys` +table: + +```json +{ + "schema": 1, + "chord": "ctrl+shift" +} +``` + +Two or three of `ctrl`, `alt`, `shift` and `win`. **One is refused** — a chord of one +modifier fires every time you tap that key and let go, which is a thing hands do all day +without meaning anything by it. Four is refused as well, because it is not a shape you +can hold. `"chord": ""` turns it off entirely, and is deliberately different from +deleting the line: Flow writes every field back when it saves, so a deleted `chord` +comes back as the default. + +**What this costs, stated plainly.** A modifier-only chord cannot be done with +`RegisterHotKey`, so Flow installs a low-level keyboard hook (`WH_KEYBOARD_LL`) — which +means Windows calls into Flow for every keystroke on the machine, not just Flow's own. +Three things are true about what that code does, and they are the reason it is +considered acceptable here: + +- **It never learns which key you pressed.** The key code is compared against eleven + modifier constants and against nothing else. Anything not in that set flips a single + true/false — not stored, not logged, not compared to anything, and it never leaves the + function. It is about sixty lines in `flow/hotkey.py` (`Chord`), written to be read. +- **It never swallows a keystroke.** Every event is passed straight on, including the + release that ends it. Ctrl and Win keep their normal jobs. +- **Nothing is sent anywhere.** Same as the rest of Flow: no network, no file. + +If you would rather not have that hook at all, `--no-chord` skips it for one launch and +`"chord": ""` skips it for good. The `toggle` hotkey is unaffected either way. + +If the OS refuses the hook — some policies and some elevated desktops do — the startup +block says so on one line and Flow carries on with the registered combos: + +``` +chord unavailable (keyboard hook refused); the toggle hotkey still works +``` + +### Without HuggingFace + +faster-whisper's model files are published on HuggingFace and nowhere else official — +SYSTRAN's GitHub ships the library, not the weights. On a network that blocks +`huggingface.co` there are three ways through, and they are in this order for a reason. + +**An internal proxy, if your organisation runs one.** Nothing to move, nothing to host: + +```bash +export HF_ENDPOINT=https:///huggingface +``` + +**Any local directory.** `--model`, `--partial-model` and `--final-model` take a path as +happily as a name, so a folder holding `config.json`, `model.bin`, `tokenizer.json` and +`vocabulary.txt` is all Flow needs: + +```bash +uv run python -m flow --model ~/flow-models/base.en +``` + +`--model` pins **both** decoder tiers to one model — 138 MB for `base.en` instead of 599 +for the usual pair. Finals come out a little weaker, since `small.en` is what that tier +exists for, and everything else is unchanged. + +Copying a HuggingFace cache between machines needs one flag: the cache stores every file +as a symlink into `blobs/`, so a plain copy arrives as four broken links. Dereference +them: + +```bash +cp -RL ~/.cache/huggingface/hub/models--Systran--faster-whisper-base.en +``` + +**The macOS engine, which needs no model files at all.** + +```bash +uv run python -m flow --engine native +``` + +macOS has an on-device recogniser with models the OS downloads through **System Settings +▸ Keyboard ▸ Dictation** — enable it once and there is nothing else to fetch, ever. Flow +talks to it through a small Swift helper it compiles on first use, which needs Xcode +Command Line Tools (`xcode-select --install`) and one grant under **Privacy & Security ▸ +Speech Recognition**. + +**`--engine auto`, the default, will not switch a machine that is working.** It reaches +for the native engine in exactly one case: the Whisper models are not on the machine and +cannot be fetched. That is deliberate — Apple's recogniser is a *different* engine rather +than a spare one. It reports no `no_speech_prob`, so Flow's hallucination filter falls +back to a much narrower check; it has one quality tier where Whisper has two; and it +cannot be biased toward your lexicon, so the re-listen that rescues a mis-heard command +comes back unbiased. Those are real differences, and changing what Flow hears without +being asked would be the wrong default. The startup line always says which engine you +got and why. + +**When something looks wrong on a Mac**, one command answers it: + +```bash +uv run --with pillow python scripts/mac_report.py +``` + +It drives the real windows, screenshots them against a neutral backdrop, renders the +numbers beside them, and writes a single `~/flow-mac-report.png` — platform and Tk build, +what each work-area method answers, where the stack is therefore placed and by how much +it misses centre, and whether the native engine is ready. One file, and the geometry in +it is exact rather than a photo of a screen. + +macOS asks for **Screen Recording** the first time, because that is the permission a +screenshot of other windows needs. The report says so out loud if the capture came back +black, rather than leaving you with an empty picture. + +Before wiring it into anything, you can judge it on your own voice: + +```bash +swiftc -O -parse-as-library -o flow-stt native/flow_stt.swift +./flow-stt --file some-recording.wav +``` + +### Where the panels open + +**Bottom centre of whichever monitor your mouse is on.** That is the shipped placement, +and it changed: Flow used to sit in the bottom-right corner of the primary display. + +Two things were wrong with the corner. The bottom right is the busiest part of a Windows +desktop — the tray lives there, every toast notification opens there, and plenty of apps +park their own status chrome there — so the one place Flow had reserved for itself was +the place most likely to be covered. And the primary display is not necessarily the one +you are working on: the work area was read once at launch from `SystemParametersInfoW`, +which only ever answers for the primary monitor, so on a two-monitor desk everything +Flow drew could land on the screen you were not looking at. + +The panel now follows the pointer's monitor and re-places itself when you move between +displays. Set `"place": "corner"` in `profile.json` to put it back in the bottom right; +it is kept for anybody who has spent months with it there, and only the position +changes. + +`"place"` takes `"bottom"` or `"corner"`. Anything else falls back to `"bottom"` rather +than refusing to launch — this is a file people edit by hand. + +**A hidden panel is moved off-screen, not closed.** It is parked past the far corner of +every monitor you have and pulled back when it is needed, so a push-to-talk hold shows +the draft in the time a window takes to move rather than the time one takes to open. +Nothing about that is visible except the speed. + +### Panel size + +**Settings ▸ Panel size** draws the draft bubble and the conversation card wider: +**Regular** (420 px, the shipped width), **Large** (520) or **Larger** (640). It applies +straight away — no restart — and is remembered in `profile.json` as `"panel": "large"`. + +**There is no "small", and the reason is the chip row rather than restraint.** The +bubble's five-chip row — Refine, Continue, Edit, Was a command, Send — measures 345 px, +and the card's runs to 377. Below 420 the row loses its gaps and then loses a label, and +the first label to go is Send. A draft panel must never put its own exit off the edge, so +420 is a floor: a hand-edited width below it is clamped back up rather than honoured. + +Wider panels lay out proportionally more text per line, so the tail of a long draft stays +as full at 640 px as it is at 420 — the number of lines handed to the canvas is what is +held constant, which is what keeps render cost flat on a two-hour dictation. + +### Per-app notes + +**A standing instruction that depends on which app you are dictating into.** Add an +`apps` table to `~/.flow/profile.json`, keyed by the executable name: + +```json +{ + "schema": 1, + "apps": { + "slack.exe": "Keep it conversational. No headings, no bullet lists.", + "code.exe": "Be terse. Prefer imperative mood.", + "outlook.exe": "Use British spelling and a full greeting." + } +} +``` + +The name is the executable of the window in front — the same one Flow already looks at +to tell a terminal from an editor. Case does not matter. An entry for an app you have not +installed is not an error and is not reported: there is no list of every program in the +world to check a name against, so a key that never matches simply never fires. `""` as +the instruction switches one app off without deleting the line you wrote. + +**It applies to rewrites, not to plain dictation.** Both the semantic rewrite and the +prompt-shaping polish ("make it a proper prompt") pick it up. Ordinary dictation never calls a +CLI at all, so there is nothing there for a note to change. + +**It cannot out-shout what you just said.** The note is phrased as a destination — *"this +text is going into slack.exe, bear this in mind, without letting it override anything +asked for below"* — and it is placed *before* your instruction rather than after it. That +ordering is the point: a per-app note is a standing preference, and speaking an +instruction is how you override a standing preference on this one occasion. Asking for +something formal in Slack gets you something formal. + +When a note applies, the pill says so, on the same line that names the CLI: + +``` +using your slack.exe note +``` + +Nothing is said when none applies, which is almost every rewrite until you write a table. +[Lite](#install) has no target-window awareness at all, so no note ever fires there. + ### The pill and the bubble Right-click the pill for **Listen / Stop listening**, **Send**, **Converse/Dictate @@ -413,8 +737,9 @@ mode**, **Clear draft** and **Quit**, plus any corrections Flow is offering. Lis the same toggle as clicking the pill, given a label — and it is the way in when a Hyper-V console or an RDP session keeps every hotkey for its guest and the mouse is what still reaches Flow. Everything you set once — **Trigger -word**, **Agent CLI**, **Voice**, **Mute/Speak replies** (only when a speech engine was -found), the auto-ask toggle and **Open settings folder** — lives under **Settings ▸**, and +word**, **Panel size**, **Agent CLI**, **Voice**, **Mute/Speak replies** (only when a +speech engine was found), the auto-ask toggle and **Open settings folder** — lives under +**Settings ▸**, and **Help ▸** has the command sheet and this guide. Drag the pill anywhere — it stays inside the desktop work area. @@ -559,6 +884,68 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. +### The strip above the draft + +Three things sit above the words whenever a panel is up, so the settings that matter +mid-task do not cost a right-click: + +**Dictate ⌄** / **Converse ⌄** — a chip, and clicking it switches. This is the one that +changes what Send does, and it is the one people switch mid-task. + +**workshop:** and **voice:** — values rather than controls. Their worth is being visible +— knowing which project Ask is running in without opening anything — and clicking either +opens the same list the right-click menu has, tick and all. The workspace is shown by the +last part of its path, because that is the part that names the project. + +**It appears with the panel and never at rest.** Those three only mean anything once +there is something to send, and an always-on strip would cost 22 px of the idle row, +which is the part of this surface worth keeping small. FluidVoice does not pay it either +— its bar belongs to the app being dictated into, not to the overlay. + +### Hiding it + +**Settings → Hide to tray** parks the window and leaves an icon in the Windows +notification area. The chord still works while hidden — it is a global hook and does not +care what is on screen — so dictating is unchanged and only the window goes. Pressing it +brings the overlay back for that utterance. + +Left-click the icon to show Flow; right-click it for **Show Flow** and **Quit Flow**. + +**It refuses to hide if the icon does not appear.** `Shell_NotifyIcon` can fail — a shell +still starting, a notification area that will not take another icon — and hiding anyway +would leave you with no window and nothing to click, reachable only through Task Manager. +So the icon is registered first and its answer is believed: no icon, no hiding, and the +bubble says why. + +Windows only. macOS has a menu bar item and Linux has whatever the desktop environment +offers, and neither is `Shell_NotifyIcon`, so the menu entry is simply absent there. + +### On a Mac + +Send pastes there too, as of this version. It used to copy and stop, which made every +other Mac fix beside the point — the whole idea is to speak into the window you are +already in, and "now press Cmd-V yourself" is the step Flow exists to remove. + +The mechanism is different because the platform is. There is no `SendInput` and no window +handle: Flow puts the text on the pasteboard with `pbcopy` and asks **System Events** to +type Cmd-V into whatever app is frontmost. It does not need to aim, because Flow's own +windows are built without a title bar and never take focus, so the app you were working in +is still the frontmost one. + +**macOS will refuse until you allow it.** Synthesising keystrokes needs Accessibility, and +macOS grants that to the *responsible* process — the terminal you started Flow from, not +Flow and not Python. So: + +**System Settings → Privacy & Security → Accessibility**, and switch on your terminal. + +Looking for "Flow" in that list is a dead end. Until it is granted, Send says so in the +bubble and names the setting; the text is on the clipboard either way, so Cmd-V works in +the meantime. `--no-paste` puts the old copy-only behaviour back if you would rather Flow +did not synthesise keystrokes at all. + +`enter boom` works the same way — the Return is sent as a key code rather than a typed +character, in the same script as the paste so nothing can come forward between them. + ### Sending it without touching anything Two words press Send, so the last step of the loop does not need the mouse: @@ -1296,7 +1683,7 @@ Verified on this machine while writing this document: | | | |---|---| -| Test suite | **1,881 tests, 39.6 s**, no mic or model needed | +| Test suite | **1,965 tests, 42.1 s**, no mic or model needed | | End-to-end | `scripts/selfdrive.py`, **64/64 checks**, live CLI round trip | | Build | `uv build` → wheel + sdist; wheel installs into a clean venv and its `flow` command runs | | Dependencies | 3 declared, **28 installed**, 243.9 MB venv | diff --git a/flow/__main__.py b/flow/__main__.py index 08d3ae0..077919f 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -28,7 +28,7 @@ from .lexicon import DEFAULT_PATH, NUL_PATH, Lexicon from .refine import MAX_TIMEOUT_SEC from .refine import TIMEOUT_SEC as REFINE_TIMEOUT_SEC -from .refine import CANDIDATES, available, named, unverified, unverified_note +from .refine import CANDIDATES, EFFORT_DEFAULT, EFFORTS, available, named, unverified, unverified_note from .session import AUTO_ASK_SEC, Session from .stats import TYPING_WPM from .stats import report as stats_report @@ -44,9 +44,23 @@ #: Said on every Lite launch, before anything else, in ASCII for the reason `say()` gives. #: It names the platform because the two ways into Lite are a flag and an OS, and someone #: who did not type `--lite` deserves to be told which one they got. +#: What a Mac is told instead, because on a Mac the sentence above is no longer true. +#: Send pastes there now — the one thing Lite was defined by. Everything else it says +#: still holds: no injection DLL, no global hotkeys, the pill is the gesture. +#: +#: Accessibility is named at startup rather than at the first failed paste, because it is +#: a thing to go and do once and finding out at the moment you needed it to work is the +#: worst time to be told. +MAC_LINE = ( + "Flow on {platform}: Send pastes into whatever app is in front, using System Events " + "- grant your terminal Accessibility in System Settings > Privacy & Security. " + "No global hotkeys: hold the pill to talk, let go to send." +) + LITE_LINE = ( "Flow Lite on {platform}: Send copies the draft and you paste it - no injection, " - "no global hotkeys, nothing to grant but the microphone." + "no global hotkeys, nothing to grant but the microphone. " + "Hold the pill to talk, let go to send; a quick click toggles listening." ) @@ -86,6 +100,140 @@ def _timeout_arg(text: str) -> float: return value + +def _chord(profile, hotkeys, Chord, parse_chord, echo, ignored_line, + unavailable, default): + """Install the modifier-only chord, and say what happened either way. + + Separated from `main()` for the reason the rest of that function is not: this is the + one startup step that can fail *silently and invisibly*. A hotkey that could not + register has `Hotkeys.failed` to name it; a chord whose hook was refused looks + exactly like a chord nobody pressed. So every path out of here prints a line, and the + lines sit in the `hotkey` block where somebody already looks when a shortcut is dead. + + Returns the `Chord` or None. The caller does not need the value — `hotkeys.chord` + holds it, which is what keeps the callback alive and gets it torn down — but a + returned object is what a test can look at. + """ + wanted = profile.chord if profile is not None else default + # The empty string is how somebody turns it off in the file, and it is deliberately + # not the same as deleting the key: the next save writes every field back, so a + # deleted `chord` would reappear as the default and the setting would look ignored. + if not (wanted or "").strip(): + return None + mods, reason = parse_chord(wanted) + if mods is None: + say(ignored_line.format(combo=echo(wanted), reason=reason)) + return None + chord = Chord(hotkeys.presses, mods, + gesture=getattr(profile, "gesture", None) if profile else None) + if not chord.start(): + # The hook is refusable — by policy, by another process, by an elevated window + # this process cannot see into. Not fatal and not silent: the registered toggle + # is still there, and the line says so rather than leaving a shape that does + # nothing with no explanation. + say(unavailable) + return None + hotkeys.chord = chord + # "hold" and not "toggle": the word in this column is what the shortcut *does*, and + # the chord stopped doing the same thing as the toggle hotkey when it became + # push-to-talk. A startup block that still said toggle would be the only place a + # user could check, quietly describing the gesture they no longer have. + # The word in this column is what the shortcut *does*, and the chord does two + # different things now depending on a setting. A startup block that named only one + # of them would be the one place a user could check, describing a gesture half of + # them do not have. + if chord.gesture == "hold": + say(f"chord {'hold':8s} {chord.describe()} (hold to talk, release to send)") + else: + say(f"chord {'toggle':8s} {chord.describe()} (press to start, again to stop)") + return chord + + +def _native_transcriber(): + """Import late, so a Windows launch never touches the macOS-only module.""" + from .native import NativeTranscriber + + return NativeTranscriber() + + +def _engine(args, partial_name: str, final_name: str) -> tuple[str, str]: + """Which decoder this launch gets, and the clause explaining why. + + `--engine` decides when it is asked to. `auto` is the interesting one, and its rule + is deliberately conservative: **Whisper unless it cannot run.** Apple's recogniser + is a real engine but a different one — no `no_speech_prob`, so `clean.py` drops to + the narrow filler check it documents for a non-Whisper engine, one quality tier + where Whisper has two, and no hotword biasing for the rescue path. Switching to it + silently, on a machine where Whisper was working, would change what Flow hears for + reasons the user never asked about. + + So `auto` reaches for it in exactly one situation: the Whisper models are not on + this machine and cannot be fetched. That is the situation it was written for — a + network that blocks huggingface.co, where the alternative is not a worse engine but + no dictation at all. + + Every path returns a clause for the startup line, because the engine decides what + Flow can hear and a silent choice would be the one thing nobody could check. + """ + if args.engine == "whisper": + return "whisper", "" + if sys.platform != "darwin": + if args.engine == "native": + say("--engine native is macOS only; using whisper") + return "whisper", "" + + from .native import available as native_available + + if args.engine == "native": + ok, why = native_available() + if ok: + return "native", "" + say(f"--engine native unavailable: {why}") + return "whisper", "" + + # auto. Ask the cheap question first: are the models here? + if _models_present(partial_name, final_name): + return "whisper", "" + # `compile_if_missing=False`, and a short probe. A launch is not the place to + # discover how slow `swiftc` is, and the probe blocks on a permission dialog that + # nobody has been shown yet — measured at a full minute per launch before this. + ok, why = native_available(compile_if_missing=False, timeout=10.0) + if ok: + return "native", " (whisper models not found on this machine)" + return "whisper", f" (not found locally, and no native engine: {why})" + + +def _models_present(*names: str) -> bool: + """Whether every named model is already on disk, without reaching the network. + + A directory holding `model.bin` is what `--model /some/path` gives, and a name like + `base.en` is present when huggingface_hub has it cached. Asked with the hub in + offline mode so this cannot become the download it is testing for. + """ + import os + from pathlib import Path + + for name in names: + path = Path(name) + if (path / "model.bin").exists(): + continue + was = os.environ.get("HF_HUB_OFFLINE") + os.environ["HF_HUB_OFFLINE"] = "1" + try: + from huggingface_hub import snapshot_download + + snapshot_download(f"Systran/faster-whisper-{name}", local_files_only=True) + except Exception: + return False + finally: + if was is None: + os.environ.pop("HF_HUB_OFFLINE", None) + else: + os.environ["HF_HUB_OFFLINE"] = was + return True + + def main(argv: list[str] | None = None) -> int: # First, before anything resolves anything. Windows searches the current directory # ahead of PATH for a bare executable name, and Flow is launched *inside* project @@ -121,6 +269,12 @@ def main(argv: list[str] | None = None) -> int: "--decode-device", default=DEVICE, choices=("auto", "cuda", "cpu"), help="where decoding runs (default auto: the GPU when there is a working one)", ) + ap.add_argument( + "--engine", default="auto", choices=("auto", "whisper", "native"), + help="which decoder: whisper (faster-whisper, needs model files) or native " + "(macOS on-device speech, no download at all). Default auto: whisper " + "unless its models are missing and the native engine is ready", + ) ap.add_argument( "--lexicon", default=None, help=f"personal terms to bias decoding toward (default {DEFAULT_PATH})", @@ -142,6 +296,10 @@ def main(argv: list[str] | None = None) -> int: ap.add_argument( "--no-hotkeys", action="store_true", help="skip global hotkey registration" ) + ap.add_argument( + "--no-chord", action="store_true", + help="skip the modifier-only chord (no low-level keyboard hook)", + ) ap.add_argument( "--no-paste", action="store_true", help="print the draft instead of pasting" ) @@ -183,6 +341,25 @@ def main(argv: list[str] | None = None) -> int: help="pin the agent CLI instead of trying each in turn " f"({', '.join(c.name for c in CANDIDATES if c.verified)})", ) + ap.add_argument( + "--no-warm", action="store_true", + # For a launcher that starts with the machine, where paying a model load at login + # is the wrong trade — and for measuring the cold path on purpose. + help="do not load the model at startup; wait until it is first needed", + ) + ap.add_argument( + "--cli-model", default=None, metavar="NAME", + # The only way a model name gets into Flow: the settings menu has no text field + # and is not growing one, so a name arrives here once, is remembered, and is a + # click from then on. Not validated against a list because no CLI will print one + # - `codex exec --help` says `-m, --model ` and stops there. + help="ask the agent CLI for this model (remembered; blank clears it)", + ) + ap.add_argument( + "--cli-effort", default=None, choices=EFFORTS, metavar="LEVEL", + help=f"how hard the CLI may think, where it offers the choice " + f"({', '.join(EFFORTS)}; default {EFFORT_DEFAULT})", + ) ap.add_argument( "--cli-timeout", type=_timeout_arg, default=REFINE_TIMEOUT_SEC, metavar="SEC", help=f"how long to wait for a CLI call (default {REFINE_TIMEOUT_SEC:.0f})", @@ -246,8 +423,13 @@ def main(argv: list[str] | None = None) -> int: # while the platform *was* the problem; it is not one any more (decisions.md, # "Flow Lite"). lite = args.lite or sys.platform != "win32" + #: Set when an injector is imported below. `None` means Send has nothing to paste + #: with and falls back to the clipboard, which is what `--no-paste` and every + #: platform without an injector get. + paste = take_warnings = None if lite: - say(LITE_LINE.format(platform=sys.platform)) + say((MAC_LINE if sys.platform == "darwin" and not args.no_paste + else LITE_LINE).format(platform=sys.platform)) if args.no_paste: # Accepted rather than refused: a launcher shared between two machines should # not fail on a flag that has simply run out of things to suppress. @@ -261,9 +443,19 @@ def main(argv: list[str] | None = None) -> int: say(f"version: {version()} (nothing checks for updates on its own; " "--check-update asks GitHub)") if not lite: - from .hotkey import BAD_BLOCK_LINE, DEFAULT_BINDINGS, Hotkeys + from .hotkey import ( + BAD_BLOCK_LINE, CHORD_IGNORED_LINE, CHORD_UNAVAILABLE, + DEFAULT_BINDINGS, Chord, Hotkeys, _echo, parse_chord, + ) + from .profile import CHORD_DEFAULT from .inject import paste, take_warnings - from .ui import Pill + elif sys.platform == "darwin" and not args.no_paste: + # Lite is about hotkeys and window handles, not about whether Flow can put the + # words where they are going. A Mac has no `SendInput` and no `hwnd`, and it does + # have `osascript` — so it gets a real send while staying Lite in every other + # respect. See `inject_mac` for why the permission is the interesting part. + from .inject_mac import paste, take_warnings + from .ui import Pill, apply_panel_width, apply_place, panel_width from .asr import WhisperTranscriber @@ -322,18 +514,39 @@ def main(argv: list[str] | None = None) -> int: default_partial, default_final = default_models(decode_device) partial_name = args.model or args.partial_model or default_partial final_name = args.model or args.final_model or default_final - if partial_name == final_name: - say(f"model: {final_name}, for partials and finals both") + engine, engine_why = _engine(args, partial_name, final_name) + if engine == "native": + say(f"engine: macOS on-device speech{engine_why}") + elif partial_name == final_name: + say(f"model: {final_name}, for partials and finals both{engine_why}") else: - say(f"models: {partial_name} for partials, {final_name} for finals") + say(f"models: {partial_name} for partials, " + f"{final_name} for finals{engine_why}") from .diag import Diag - from .profile import Profile, resolve_workspace + from .profile import CLI_MODEL_CAP, Profile, resolve_workspace # Tied to the same flag as the profile, and deliberately: --no-profile means # "write nothing about me this session", and a trace is a thing written about # somebody even when it holds none of their words. profile = None if args.no_profile else Profile() + # Written to the profile before the session reads it, so `--cli-model` is a *setting* + # and not a one-run override: the settings menu has no way to type a name, so a flag + # that vanished at exit would leave the menu permanently empty. `--cli-model ""` + # clears it, which is why this tests for None rather than truthiness. + if profile is not None and args.cli_model is not None: + profile.cli_model = args.cli_model.strip() + if profile.cli_model and profile.cli_model not in profile.cli_models: + profile.cli_models = (*profile.cli_models, profile.cli_model)[-CLI_MODEL_CAP:] + profile.save() + if profile is not None and args.cli_effort is not None: + profile.cli_effort = args.cli_effort + profile.save() + if profile is not None: + if profile.cli_model: + say(f"model: {profile.cli_model}") + say(f"effort: {profile.cli_effort} (where the CLI offers the choice)") + diag = None if args.no_profile else Diag() learned = profile.learned_terms if profile is not None else None if profile is not None and profile.calibrated: @@ -410,11 +623,11 @@ def main(argv: list[str] | None = None) -> int: profile.save() session = Session( - asr=WhisperTranscriber( + asr=(_native_transcriber() if engine == "native" else WhisperTranscriber( partial_name, final_name, lexicon=lexicon, baseline=profile.confidence if profile is not None else None, device=args.decode_device, - ), + )), device=args.device, speaker=speaker, profile=profile, @@ -468,12 +681,20 @@ def main(argv: list[str] | None = None) -> int: "(--no-auto-ask to press it yourself)") else: say("auto-ask: off - press Ask when you are ready") - elif lite: + elif paste is None: say("mode: DICTATE - Send copies the draft, and you paste it " "(--converse, or the right-click menu, to ask instead)") else: say("mode: DICTATE - Send pastes into the focused window " - "(--converse, or ctrl+alt+M, to ask instead)") + f"(--converse, or {'the right-click menu' if lite else 'ctrl+alt+M'}, " + "to ask instead)") + + # Before anything is drawn, and that is the whole contract: `apply_panel_width` + # rebinds module globals that two window classes and the chrome functions read, so a + # call after the first frame would leave a window whose parts disagree about how wide + # it is. Here is the last moment that is safely true. + apply_panel_width(panel_width(profile.panel if profile is not None else None)) + apply_place(profile.place if profile is not None else "bottom") hotkeys = None if not args.no_hotkeys and not lite: @@ -499,6 +720,10 @@ def main(argv: list[str] | None = None) -> int: else: say("hotkey thread did not start; continuing without hotkeys") hotkeys = None + if hotkeys is not None: + if not args.no_chord: + _chord(profile, hotkeys, Chord, parse_chord, _echo, + CHORD_IGNORED_LINE, CHORD_UNAVAILABLE, CHORD_DEFAULT) # Assigned rather than passed: the session is built before `RegisterHotKey` has been # asked for anything, and what the session needs is the answer, not the request. It # reads this only to say what still works when voice stops working. @@ -557,11 +782,29 @@ def on_send(text: str, target: int | None = None, submit: bool = False) -> str: # exist, so the menu is sent to the real settings folder instead: the profile lives # there either way, and creating a template beside the source is nobody's idea of # settings. + # What the mode notes read, and the same fact `on_send` is keyed off. + session.pastes = paste is not None pill = Pill( - session, on_send=None if lite else on_send, hotkeys=hotkeys, arm=args.arm, + # Keyed off whether an injector was imported, not off `lite`. The two came + # apart the day a Mac got a paste path: it is Lite in every other sense and can + # still put the words in the other window. + session, on_send=on_send if paste is not None else None, + hotkeys=hotkeys, arm=args.arm, settings_path=DEFAULT_PATH if args.no_lexicon else lexicon.path, lite=lite, ) + # **Loaded now, not at the first word.** "loading the model" used to be the first + # thing a fresh Flow said back, in the bubble, while somebody was already speaking — + # and the load lands *inside* that first utterance rather than in front of it, so the + # first partial measured 1 230 ms against ~570 ms for the four behind it. The chord's + # press-down has warmed the models since push-to-talk shipped, which covers the + # second use and not the first. + # + # After the pill is built and before the loop runs, so the window is on screen while + # the disk does its work rather than after it. `warm()` returns immediately — it is + # single-flight and does its loading on a thread of its own — so nothing here waits. + if not args.no_warm: + session.warm() try: pill.mainloop() except KeyboardInterrupt: diff --git a/flow/help.py b/flow/help.py index 08d4502..6990970 100644 --- a/flow/help.py +++ b/flow/help.py @@ -138,6 +138,10 @@ def auto_ask_notice(seconds: float) -> str: #: cannot find here is worse than one described badly. _ACTIONS = { "toggle": "start and stop listening", + # The chord's word, and the one row here that describes a *hold* rather than a + # press. Spelled out to the end — release and all — because the half people get + # wrong is that there is no second shortcut to send. + "talk": "hold to talk, release to send", "send": "hand the draft over (Send, or Ask in converse mode)", "cancel": "clear the draft, and cut a spoken reply short", "mode": "switch between dictate and converse", @@ -218,6 +222,16 @@ def _hotkey_rows(hotkeys) -> list[tuple[str, str, str]]: failed = list(getattr(hotkeys, "failed", []) or []) rows = [("pair", combo, _ACTIONS.get(action, action)) for action, combo in chosen.items()] + # The chord goes first among the rows it duplicates, because it is the one somebody + # is looking for: it does not appear in `chosen` — nothing registered it — so a sheet + # built only from what `RegisterHotKey` accepted would describe a machine where the + # shape they have been using does not exist. Absent when the hook was refused or + # `--no-chord` was passed, which is the same rule the rest of this function follows: + # the sheet says what works on this machine this launch. + chord = getattr(hotkeys, "chord", None) + if chord is not None: + rows.insert(0, ("pair", f"{chord.describe()} (held)", + _ACTIONS.get(chord.action, chord.action))) # Named as unavailable rather than left out. A shortcut that silently does nothing is # the defect `Hotkeys.failed` was built to report, and a sheet that omitted it would # send somebody looking for a key that cannot exist on this machine. diff --git a/flow/hotkey.py b/flow/hotkey.py index df6d23d..09aa553 100644 --- a/flow/hotkey.py +++ b/flow/hotkey.py @@ -95,6 +95,13 @@ def __init__(self, bindings: dict[str, list[tuple[int, int]]], self.failed: list[str] = [] #: action name -> the combo that actually registered, e.g. "ctrl+alt+space" self.chosen: dict[str, str] = {} + #: The modifier-only `Chord`, once one has been installed, or None. + #: + #: It hangs here rather than being tracked separately because the pill already + #: calls `hotkeys.stop()` on the way out and there is exactly one thing that + #: should own the teardown of "global key input". A chord left installed after + #: the window is gone is a hook the OS still calls into a dead interpreter. + self.chord = None self._ids: dict[int, str] = {} self._tid: int | None = None self._ready = threading.Event() @@ -105,6 +112,8 @@ def start(self, timeout: float = 2.0) -> bool: return self._ready.wait(timeout) def stop(self) -> None: + if self.chord is not None: + self.chord.stop() if self._tid is not None: user32.PostThreadMessageW(self._tid, WM_QUIT, 0, 0) @@ -347,3 +356,374 @@ def overridden( continue out[action].insert(0, binding) return out, ignored + + +# -- the modifier-only chord ------------------------------------------------ +# +# `RegisterHotKey` cannot express "ctrl+win, and no third key". It takes a virtual key +# and there is no VK for "nothing" — so every combo above the fold has to end in a key +# somebody presses with their other hand. That third key is the whole complaint: ctrl, +# shift and win sit together at the bottom-left and are one *shape*, while ctrl+shift+z +# is a shape plus a reach. +# +# Which leaves the low-level keyboard hook, and R16 said no to exactly that — no +# `keyboard`, no `pynput`, nothing sitting on the global input path. This narrows R16 +# rather than reversing it, and the narrowing is the thing to check when reading: +# +# **It never learns which key you pressed.** `vkCode` is compared against `_CHORD_VKS` +# — eleven modifier constants — and against nothing else. A key that is not in that set +# sets a *boolean*. It is not stored, not logged, not compared to anything, and does +# not leave the callback. That is a materially smaller claim than "Flow can see your +# keystrokes", and it is small enough to audit in one sitting, which is the point. +# +# **It is allocation-free on the hot path.** This callback runs on the input path of +# every keystroke on the machine, and a slow one makes *all* typing feel late — the +# worst possible failure for a dictation tool, because it would look like the OS. So +# the steady state is integer comparisons and attribute writes. The one allocation is +# `presses.put`, which happens when the chord actually fires. +# +# **It never swallows anything.** `CallNextHookEx` is called on every event, including +# the ones that fire the chord. Ctrl and Win are real keys with real jobs and Flow does +# not get to keep them. +# +# **Why the "some other key" flag is correctness and not only privacy.** Windows already +# uses ctrl+win as a *prefix*: ctrl+win+d makes a virtual desktop, ctrl+win+left and +# +right switch between them. Every one of those presses a third key, so requiring a +# clean release — both modifiers held, nothing else touched, then let go — is what keeps +# switching desktops from also starting dictation. It is the same flag serving both ends. +# +# Start never opens on the release either, and that is Windows' own rule rather than +# something arranged here: the Start menu comes up on a Win keyup only when Win was +# pressed alone, and holding Ctrl is already enough to suppress it. + +WH_KEYBOARD_LL = 13 +WM_KEYDOWN, WM_KEYUP, WM_SYSKEYDOWN, WM_SYSKEYUP = 0x0100, 0x0101, 0x0104, 0x0105 + +VK_SHIFT, VK_CONTROL, VK_MENU = 0x10, 0x11, 0x12 +VK_LWIN, VK_RWIN = 0x5B, 0x5C +VK_LSHIFT, VK_RSHIFT = 0xA0, 0xA1 +VK_LCONTROL, VK_RCONTROL = 0xA2, 0xA3 +VK_LMENU, VK_RMENU = 0xA4, 0xA5 + +#: vk -> which modifier name it is. The hook reads `vkCode` against this and nothing +#: else; see the block above for why that sentence is the design and not a detail. +#: +#: Both the generic constants (`VK_CONTROL`) and the sided ones (`VK_LCONTROL`) are in +#: here because the hook is fed the *sided* code for a physical press, while an injected +#: keystroke — `SendInput`, which `flow/inject.py` itself uses — may carry the generic +#: one. Listening for only one of the two would make the chord's behaviour depend on +#: whether a human or a program pressed it. +_CHORD_VKS: dict[int, str] = { + VK_CONTROL: "ctrl", VK_LCONTROL: "ctrl", VK_RCONTROL: "ctrl", + VK_LWIN: "win", VK_RWIN: "win", + VK_SHIFT: "shift", VK_LSHIFT: "shift", VK_RSHIFT: "shift", + VK_MENU: "alt", VK_LMENU: "alt", VK_RMENU: "alt", +} + +#: The shipped chord — two modifiers, one hand, no reach, which is the entire reason +#: this code path exists rather than a sixth entry in `DEFAULT_BINDINGS` — lives in +#: `flow/profile.py` as `CHORD_DEFAULT`, because that module can be imported on a Mac +#: and this one cannot. What a chord *means* is judged here; what it defaults to is +#: written there, the same split `hotkeys` already uses. + +#: Modifier names a chord may be written from, in the order `describe_chord` prints them +#: so that "win+ctrl" and "ctrl+win" report as the same thing. +CHORD_NAMES = ("ctrl", "alt", "shift", "win") + +#: The two gestures a chord can be. `"hold"` is push-to-talk — press to start capturing, +#: speak while it is down, release to send what was said. `"toggle"` is the original: +#: a clean press-and-release starts hands-free listening, and the next one stops it. +#: +#: Both ship because they are good at different things and neither replaces the other. +#: A hold is the better gesture for a sentence — it needs no decision about when you are +#: finished, and it cannot leave a microphone running. A toggle is the only one of the +#: two that survives a paragraph, a long thought with pauses in it, or a pair of hands +#: that cannot hold two keys down for a minute. Shipping only the hold, which is what +#: this did first, took the second case away from everybody who had it. +GESTURES = ("hold", "toggle") +GESTURE_DEFAULT = "hold" + +#: Said when the hook cannot be installed. Shaped like the `hotkey` lines beside it, +#: because that block is where somebody looks when a shortcut is dead — and unlike a +#: taken combo, this one has a working answer to point at. +CHORD_UNAVAILABLE = ("chord unavailable (keyboard hook refused); " + "the toggle hotkey still works") + +#: Said when the `chord` value in profile.json could not be read. Same shape as +#: `IGNORED_LINE` and for the same reason: a setting that silently reverts is +#: indistinguishable from one that never saved (P2). +CHORD_IGNORED_LINE = "chord in profile.json ignored: {combo} - {reason}" + + +class _KBDLLHOOKSTRUCT(ctypes.Structure): + _fields_ = [ + ("vkCode", wintypes.DWORD), + ("scanCode", wintypes.DWORD), + ("flags", wintypes.DWORD), + ("time", wintypes.DWORD), + ("dwExtraInfo", ctypes.POINTER(wintypes.ULONG)), + ] + + +_HOOKPROC = ctypes.WINFUNCTYPE( + wintypes.LPARAM, ctypes.c_int, wintypes.WPARAM, wintypes.LPARAM +) + +user32.SetWindowsHookExW.argtypes = [ + ctypes.c_int, _HOOKPROC, wintypes.HINSTANCE, wintypes.DWORD +] +user32.SetWindowsHookExW.restype = wintypes.HHOOK +user32.UnhookWindowsHookEx.argtypes = [wintypes.HHOOK] +user32.UnhookWindowsHookEx.restype = wintypes.BOOL +user32.CallNextHookEx.argtypes = [ + wintypes.HHOOK, ctypes.c_int, wintypes.WPARAM, wintypes.LPARAM +] +user32.CallNextHookEx.restype = wintypes.LPARAM + + +def parse_chord(text) -> tuple[frozenset, str]: + """`{"ctrl", "win"}` for "ctrl+win", or `None` and why not. + + Two returns for the reason `parse` has two: the caller has to print what was wrong, + and a reason assembled at the call site is one that will eventually disagree with the + check that produced it. + + **One modifier is refused**, and it is the refusal worth explaining. A single-modifier + "chord" fires every time that key is tapped and released cleanly — and a bare Ctrl tap + is a thing hands do constantly while thinking. The result would be a dictation app + that starts recording at random, which is a defect nobody would attribute to a + setting they typed. + + **Four is refused for the opposite reason**: ctrl+alt+shift+win is not a shape, and a + chord that cannot be held is a chord that never fires. + """ + if not isinstance(text, str): + return None, "not a string" + parts = [p.strip().lower() for p in text.split("+") if p.strip()] + if not parts: + return None, "empty" + seen = set() + for part in parts: + if part not in CHORD_NAMES: + return None, f"{_echo(part)} is not a modifier" + seen.add(part) + if len(seen) < 2: + return None, "a chord needs two modifiers" + if len(seen) > 3: + return None, "a chord of four modifiers cannot be held" + return frozenset(seen), "" + + +def describe_chord(mods) -> str: + return "+".join(name for name in CHORD_NAMES if name in mods) + + +class Chord: + """Fires `action` when `mods` are held together and released with nothing else hit. + + Writes into a queue the caller owns — `Hotkeys.presses` in practice — so the session + drains one stream and cannot tell a chord from a registered combo. The chord is a + second *way in*, not a second thing to handle, and every consumer downstream already + knows what "toggle" means. + """ + + def __init__(self, presses, mods=frozenset({"ctrl", "win"}), + action: str = "talk", warm_action: str = "warm", + end_action: str = "talk-end", break_action: str = "talk-break", + gesture: str = None, toggle_action: str = "toggle") -> None: + self.presses = presses + self.mods = mods + #: Put when the chord forms: start capturing, the hold is the utterance. + self.action = action + #: Put immediately before `action`, so the models load during the hold rather + #: than inside the first sentence. `Session.warm` carries that argument in full. + self.warm_action = warm_action + #: Put when a chord modifier is released and the hold was clean — stop, and send + #: what was said. + self.end_action = end_action + #: Put when a third key lands *during* a hold that had already started. Windows + #: owns `ctrl+win+d` and `ctrl+win+arrow`, so this is the common case and not the + #: exotic one, and it has to stop the capture the press-down opened. Distinct + #: from `end_action` because what happens to the audio differs: this one does not + #: paste, and the session decides between dropping it and keeping it on screen + #: based on how much of it there is. + self.break_action = break_action + #: Which gesture this chord *is* — `"hold"` for push-to-talk, `"toggle"` for the + #: press-and-release that starts and stops hands-free listening. + #: + #: **A plain attribute, read inside the hook, so it can be changed at runtime.** + #: Rebuilding the `Chord` to switch would mean tearing down a `WH_KEYBOARD_LL` + #: hook and installing another one, which is the one operation in this file the + #: OS is entitled to refuse — and being refused *while changing a setting* would + #: leave somebody with no chord at all and no obvious way back. One string + #: assignment cannot fail. + #: + #: The two are genuinely different gestures rather than one with a flag, and the + #: reason they are both here is that they are good at different things: a hold is + #: better for a sentence, and a toggle is the only one of the two that works for + #: a paragraph, a phone call, or anybody who cannot hold two keys down. + self.gesture = gesture if gesture in GESTURES else GESTURE_DEFAULT + #: What a `"toggle"` chord puts, on the release and nowhere else. Kept separate + #: from `action` so the two gestures do not have to agree about a word: a hold + #: starts capture and a toggle flips it, and calling both "talk" would make the + #: dispatch table lie about one of them. + self.toggle_action = toggle_action + self.installed = False + #: Which of `mods` are down right now. A set is the honest shape and the wrong + #: one here — this is touched on the input path of every keystroke on the + #: machine, so it is a plain dict of name -> bool, written in place. + self._down = {name: False for name in mods} + #: True once a key outside `mods` went down while the chord was forming. The only + #: thing this file ever learns about that key. + self._other = False + #: The modifiers this chord does *not* want, and whether each is held right now. + #: + #: Tracked separately from `_other` because the two answer different questions, + #: and one flag answering both got it wrong in exactly one direction. `_other` is + #: "was something pressed *during* this hold", and it has to reset when the chord + #: forms — otherwise holding Ctrl to click a link, tapping a key, then adding Win + #: would be refused for a keystroke that had nothing to do with the chord. But a + #: modifier that is *still down* when the chord forms is not history, it is part + #: of the shape: ctrl+shift+win is a different chord from ctrl+win, and telling + #: them apart means asking what is held, not what was pressed. + #: + #: Only modifiers get this treatment, and that is the privacy line holding: a + #: modifier's identity is already in `_CHORD_VKS`, while every other key on the + #: board still collapses to the single boolean above. + self._extra = {name: False for name in CHORD_NAMES if name not in mods} + #: True while every modifier in `mods` is held. Latched rather than recomputed so + #: that releasing them one at a time still fires exactly once. + self._armed = False + #: True between the press-down that started capturing and whatever ends it — a + #: release, or a third key. Separate from `_armed` because a hold that formed + #: with an unwanted modifier already down arms without ever starting, and the end + #: has to know whether there is anything to end. Nothing about it is a keystroke: + #: it is the same single boolean `_other` is, and for the same reason. + self._talking = False + self._hook = None + self._tid = None + self._ready = threading.Event() + #: The callback is handed to the OS, which does not keep a Python reference to + #: it. Without this attribute it would be collected while still installed, and + #: the process would die inside a keystroke somewhere unrelated. + self._proc = _HOOKPROC(self._on_key) + self._thread = threading.Thread(target=self._run, daemon=True, name="chord") + + def start(self, timeout: float = 2.0) -> bool: + self._thread.start() + self._ready.wait(timeout) + return self.installed + + def _install(self): + """Ask for the hook, and treat every way of not getting one the same. + + `SetWindowsHookExW` answers NULL when the OS refuses — policy, another process, + a desktop this one cannot reach into. It is *also* the call that would raise if + this build ever ran somewhere the symbol is missing. Both are the same event to + everyone upstream: there is no chord, the registered toggle still works, and the + startup block says so on one line. + """ + try: + return user32.SetWindowsHookExW(WH_KEYBOARD_LL, self._proc, None, 0) + except (AttributeError, OSError, ValueError): + return None + + def stop(self) -> None: + if self._tid is not None: + user32.PostThreadMessageW(self._tid, WM_QUIT, 0, 0) + + def describe(self) -> str: + return describe_chord(self.mods) + + def _break(self) -> None: + """A third key landed mid-hold, so Windows meant something else. Stop capturing. + + Under the old toggle gesture this needed no code at all: nothing had started, so + refusing to fire was the whole behaviour. Push-to-talk opens the microphone on + the press-down, which means `ctrl+win+d` now has something to undo — and it must + be undone here, on the keystroke, rather than left for the release. Holding + `ctrl+win` through three desktop switches would otherwise record all of them. + + Once per hold. `_talking` is cleared first, so the arrow key that follows the + first arrow key does not put a second break on the queue. + """ + if self._talking: + self._talking = False + self.presses.put(self.break_action) + + def _on_key(self, code, wparam, lparam): + # Negative `code` means "pass it on without looking", and it is not advice. + if code >= 0: + vk = ctypes.cast(lparam, ctypes.POINTER(_KBDLLHOOKSTRUCT)).contents.vkCode + name = _CHORD_VKS.get(vk) + if wparam == WM_KEYDOWN or wparam == WM_SYSKEYDOWN: + if name is not None and name in self._down: + self._down[name] = True + if not self._armed and all(self._down.values()): + # A fresh hold starts a fresh verdict: whatever was *pressed* + # before the chord formed is not this chord's business. What is + # still *held* is — see `_extra`. + self._armed = True + self._other = any(self._extra.values()) + if not self._other and self.gesture == "hold": + # The hold has begun. Two puts and no other work — the rule + # about what this callback may do on the input path of every + # keystroke on the machine is unchanged. + # + # Nothing at all in the toggle gesture: it has no press-down + # half, and warming on one would load the models every time + # somebody reached for `ctrl+win+arrow`. + self._talking = True + self.presses.put(self.warm_action) + self.presses.put(self.action) + elif name is not None: + # A modifier this chord does not want. Held state, not history. + self._extra[name] = True + self._other = True + self._break() + else: + # Every other key on the keyboard. One boolean, and nothing else + # about it is read, kept or compared. + self._other = True + self._break() + elif wparam == WM_KEYUP or wparam == WM_SYSKEYUP: + if name is not None and name in self._down: + self._down[name] = False + if self._armed: + self._armed = False + if self._talking: + self._talking = False + self.presses.put(self.end_action) + elif self.gesture == "toggle" and not self._other: + # The original gesture, unchanged: a clean release — both + # held, nothing else touched — flips hands-free listening. + # `_other` is the same rule doing the same job it always + # did, which is why `ctrl+win+d` still makes a desktop and + # starts nothing. + self.presses.put(self.toggle_action) + elif name is not None: + self._extra[name] = False + return user32.CallNextHookEx(self._hook, code, wparam, lparam) + + def _run(self) -> None: + self._tid = kernel_thread_id() + try: + # `hMod` NULL with a thread id of 0 is the documented shape for a low-level + # hook: unlike the other WH_ hooks it is not injected into other processes, + # so it needs no module handle and the callback stays on this thread. + self._hook = self._install() + self.installed = bool(self._hook) + finally: + # Set in `finally` and not after, because `start()` is *waiting* on it. An + # exception on the line above would otherwise leave the launch blocked for + # the full timeout and then report "unavailable" for a reason nobody could + # find — the traceback goes to a daemon thread's stderr, which on a windowed + # build is nowhere at all. + self._ready.set() + if not self._hook: + return + msg = wintypes.MSG() + while user32.GetMessageW(ctypes.byref(msg), None, 0, 0) > 0: + pass + user32.UnhookWindowsHookEx(self._hook) + self._hook = None diff --git a/flow/inject_mac.py b/flow/inject_mac.py new file mode 100644 index 0000000..87c97d5 --- /dev/null +++ b/flow/inject_mac.py @@ -0,0 +1,206 @@ +"""Paste into the app you were looking at, on macOS. + +`inject.py` is Win32 to the bone — `SendInput`, `OpenClipboard`, window handles — and +none of it exists here. This is its opposite number, satisfying the same two-function +contract `__main__.on_send` is written against: `paste(text, submit=…) -> bool`, and +`take_warnings()` for the reasons behind a False. + +**Why it had to exist.** Off Windows Flow ran in Lite, where Send copies the draft to the +clipboard and stops. That is a fine fallback and a poor product: the whole point is to +speak into the window you are already working in, and "now press Cmd-V yourself" is the +step Flow was built to remove. Everything else about the Mac was made to work first — the +window stays up, takes clicks, sits above the Dock — and the thing it all leads to still +handed you a clipboard. + +**System Events rather than CGEvent, for now.** Posting a synthetic Cmd-V through +`CGEventPost` is faster and needs no `osascript` process, and it is where this should +eventually go — `native/flow_stt.swift` is already a compiled binary with a build step. +It is also an entitlement question and a second thing to be right about before anybody +could try this once. `osascript` is on every Mac, needs no dependency (R16 holds at +three) and asks the OS for the same permission the compiled route would. + +**The permission is the thing that will actually bite.** Synthesising keystrokes needs +Accessibility, and macOS grants it to the *responsible* process — the terminal Flow was +started from, not Python and not Flow. Denied, System Events fails with `-1719` or `1002` +and does nothing at all, which from the user's side is indistinguishable from the Send +button being broken. So that case is detected by its error number and answered with the +exact path through System Settings, rather than left as a silent no-op (invariant 4). +""" + +import subprocess +import threading +import time + +#: How long the target app gets to read the clipboard before Flow puts the old contents +#: back. `inject.py`'s reasoning applies unchanged: the keystroke is queued, not waited +#: on, so restoring immediately would hand back the old text before the new text had been +#: read. The number is its number, for the same reason. +RESTORE_DELAY_SEC = 8.0 + +#: How long any one `osascript` call may take. Generous for a process that normally +#: answers in well under a second, and finite so a wedged System Events cannot hang the +#: send — the property `refine.sane_timeout` exists to protect, applied here too. +SCRIPT_TIMEOUT_SEC = 10.0 + +#: What System Events says when the terminal has not been granted Accessibility. `-1719` +#: is "not allowed assistive access" and `1002` is the keystroke-specific refusal; both +#: appear on stderr with the number in them. Matched on the numbers rather than the +#: prose, which is localised. +DENIED_CODES = ("-1719", "1002", "-25211") + +#: Names a frontmost process can have when the thing in front is Flow itself. Flow +#: pasting into its own draft box is the one outcome that would destroy the text being +#: sent, and `ui._bare_window` is what should make it impossible — this is the belt to +#: that pair of braces. +FLOW_NAMES = ("python", "python3", "flow", "tk", "wish") + +#: Warnings from the last paste, drained by the UI. A module-level queue for `inject.py`'s +#: reason: `paste` already returns success, and a caller that ignores the warning must +#: still see it. Locked because the restore runs on its own thread. +_warnings: list[str] = [] +_lock = threading.Lock() + + +def _warn(line: str) -> None: + with _lock: + _warnings.append(line) + + +def take_warnings() -> list[str]: + with _lock: + out, _warnings[:] = list(_warnings), [] + return out + + +def _run(argv: list[str], stdin: str | None = None) -> tuple[bool, str]: + """One child process. Returns `(ok, output-or-reason)` and never raises. + + A send that dies on an OSError from a helper nobody has heard of is worse than one + that reports what it could not do, so everything below treats failure as a sentence + to show the user rather than an exception to propagate. + """ + try: + done = subprocess.run( + argv, input=stdin, capture_output=True, text=True, + timeout=SCRIPT_TIMEOUT_SEC, encoding="utf-8", errors="replace", + ) + except OSError as exc: + return False, f"{argv[0]} would not start: {exc}" + except subprocess.TimeoutExpired: + return False, f"{argv[0]} did not answer in {SCRIPT_TIMEOUT_SEC:.0f}s" + if done.returncode != 0: + return False, (done.stderr or done.stdout or "").strip() or f"{argv[0]} failed" + return True, done.stdout or "" + + +def denied(reason: str) -> bool: + return any(code in reason for code in DENIED_CODES) + + +def permission_note() -> str: + """The one message worth getting right, because it is the only one most people see. + + Names the terminal rather than Flow or Python, because that is what the permission is + actually attached to — macOS assigns it to the responsible process — and looking for + "Flow" in that list is a dead end. + """ + return ("not pasted: macOS has not granted permission to send keystrokes. Open " + "System Settings > Privacy & Security > Accessibility and switch on the " + "terminal you started Flow from, then try again. The text is on the " + "clipboard, so Cmd-V works in the meantime.") + + +def frontmost() -> str: + """The name of the app that will receive the paste, or "" if it cannot be asked.""" + ok, out = _run(["osascript", "-e", + 'tell application "System Events" to get name of first process ' + "whose frontmost is true"]) + return out.strip() if ok else "" + + +def get_clipboard_text() -> str | None: + ok, out = _run(["pbpaste"]) + return out if ok else None + + +def set_clipboard_text(text: str) -> bool: + """`pbcopy` rather than Tk's clipboard, and the difference matters. + + Tk owns its selection for as long as the interpreter lives and serves it on request, + so a draft copied that way disappears the moment Flow exits. `pbcopy` hands the text + to the pasteboard server, where it behaves like anything else you have ever copied. + """ + ok, reason = _run(["pbcopy"], stdin=text) + if not ok: + _warn(f"could not reach the clipboard: {reason}") + return ok + + +def _restore_later(previous: str | None) -> None: + """Put the old clipboard back, once the paste has had time to happen. + + A daemon thread, so quitting Flow between the paste and the restore does not hold the + process open. The cost of losing the restore is the draft staying on the clipboard, + which is the state Lite mode leaves it in anyway. + """ + if previous is None: + return + + def worker() -> None: + time.sleep(RESTORE_DELAY_SEC) + _run(["pbcopy"], stdin=previous) + + threading.Thread(target=worker, name="flow-clipboard-restore", daemon=True).start() + + +def paste( + text: str, + *, + hwnd: int | None = None, + restore_clipboard: bool = True, + submit: bool = False, +) -> bool: + """Put `text` on the clipboard and send Cmd-V to whatever is frontmost. + + `hwnd` exists for the signature `__main__.on_send` is written against and is ignored: + macOS has no window handle to aim at, and it does not need one. Flow's own windows are + built without the `titled` bit and never take focus (`ui._bare_window`), so the app + that was frontmost when you started speaking is still frontmost now — which is why the + window work had to come first. + + **The clipboard is written before the keystroke is attempted**, deliberately, and it + is the same decision `inject.py` made: if the keystroke is refused, the text is still + somewhere the user can reach with their own Cmd-V. + """ + if not text: + return False + + # Asked before anything is touched, so the refusal costs no clipboard. + front = frontmost() + if front.lower() in FLOW_NAMES: + _warn(f"not pasted: {front} had the focus, not the window you were aiming at") + return False + + previous = get_clipboard_text() if restore_clipboard else None + if not set_clipboard_text(text): + return False + + # One script rather than two calls: each `osascript` is a process launch, and the gap + # between a Cmd-V and a Return is exactly where another window could come forward and + # take the Return. + keys = ['keystroke "v" using command down'] + if submit: + # `key code 36` is Return. Spelled as a code because `keystroke return` sends the + # character, and an app that tells them apart gets a newline instead of a send. + # The delay lets the paste land first — without it both events arrive in the same + # turn of the target's run loop and the Return can win. + keys += ["delay 0.05", "key code 36"] + script = chr(10).join(['tell application "System Events"', *keys, "end tell"]) + ok, reason = _run(["osascript", "-e", script]) + if not ok: + _warn(permission_note() if denied(reason) else f"not pasted: {reason}") + return False + + if restore_clipboard: + _restore_later(previous) + return True diff --git a/flow/native.py b/flow/native.py new file mode 100644 index 0000000..8402014 --- /dev/null +++ b/flow/native.py @@ -0,0 +1,249 @@ +"""Apple's on-device recogniser, as a `Transcriber` Flow can use instead of Whisper. + +**Why this exists.** faster-whisper's CT2 weights are published on HuggingFace and +nowhere else official — SYSTRAN's GitHub ships the library, not the models. On a network +that blocks `huggingface.co` that leaves copying 138 MB by hand onto every machine. +macOS has a speech engine already installed, with models the OS downloads through +System Settings, and using it ends the transport problem rather than routing around it. + +**Why a subprocess.** The dependency budget is three (R16) and PyObjC is not one of them. +Flow already shells out to `codex` and `claude`, so a process that reads audio and writes +text is a shape this app has. It also puts every Objective-C API behind a pipe, where a +crash is an exit code instead of a dead interpreter. + +**What it costs, stated rather than discovered.** Apple's recogniser reports no +`no_speech_prob`, so `clean.py` falls to the narrow whole-utterance filler check it +documents for exactly this case — hallucination filtering is weaker here than with +Whisper. `hotwords` has no equivalent on this path and is accepted and ignored, which +makes the constrained re-decode of a suspected mis-heard command a no-op rather than an +error. And the permission prompt is attributed to whatever launched Flow, usually a +terminal, because this is not a signed bundle. +""" + +from __future__ import annotations + +import struct +import subprocess +import sys +import threading +from pathlib import Path + +import numpy as np + +#: The helper's source, and where its build lands. Built into the user's cache rather +#: than into the checkout: a repo on a read-only share still has to work, and a binary +#: in a source tree is a thing that gets committed by accident. +SOURCE = Path(__file__).resolve().parent.parent / "native" / "flow_stt.swift" +BUILD_DIR = Path.home() / ".flow" / "bin" +BINARY = BUILD_DIR / "flow-stt" + +#: How long a build may take before it is called a failure. `swiftc` on a cold toolchain +#: is slow; a launch is not the place to find out how slow. +BUILD_TIMEOUT_SEC = 120.0 +#: How long one utterance may take. The helper has its own 30 s bound inside; this is the +#: outer one for a process that has stopped answering at all. +DECODE_TIMEOUT_SEC = 40.0 + + +class NotAvailable(RuntimeError): + """This machine cannot run the native engine, with the reason in the message. + + One exception for every way of not having it — wrong platform, no toolchain, a build + that failed, a permission the user declined — because every one of them means the + same thing to the caller: use Whisper, and say this sentence to explain why. + """ + + +def _run(argv: list[str], timeout: float) -> subprocess.CompletedProcess: + return subprocess.run(argv, capture_output=True, text=True, timeout=timeout, + check=False) + + +def build(force: bool = False) -> Path: + """Compile the helper if it is not already built. Returns the binary's path. + + Built on first use rather than shipped, because a prebuilt binary would have to be + signed and notarised to be worth anything on a Mac, and an unsigned one is a + Gatekeeper prompt with a worse story than the source it came from. `swiftc` is + present wherever Xcode Command Line Tools are, which is most machines that have + ever run `git`. + """ + if sys.platform != "darwin": + raise NotAvailable("the native engine is macOS only") + if BINARY.exists() and not force: + return BINARY + if not SOURCE.exists(): + raise NotAvailable(f"helper source missing: {SOURCE}") + try: + which = _run(["xcrun", "--find", "swiftc"], 20.0) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"no Swift toolchain: {exc}") from exc + if which.returncode != 0: + raise NotAvailable("no Swift toolchain - run: xcode-select --install") + BUILD_DIR.mkdir(parents=True, exist_ok=True) + try: + # `-parse-as-library` is required, not tuning: a single-file executable is + # parsed as a script, and `@main` cannot coexist with script mode. + built = _run(["xcrun", "swiftc", "-O", "-parse-as-library", + "-o", str(BINARY), str(SOURCE)], BUILD_TIMEOUT_SEC) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"build failed: {exc}") from exc + if built.returncode != 0 or not BINARY.exists(): + # The compiler's own words, trimmed. A build error the user cannot see is a + # feature that is missing for no stated reason. + detail = (built.stderr or built.stdout or "").strip().splitlines() + raise NotAvailable("build failed: " + (detail[-1] if detail else "no output")) + return BINARY + + +def available(compile_if_missing: bool = True, + timeout: float = 60.0) -> tuple[bool, str]: + """`(usable, why not)` for this machine, without starting a session on it. + + Runs the helper's own `--probe`, which is the only honest check: the engine exists + when the OS says it does, the locale resolves, on-device recognition is supported — + which needs Dictation enabled so macOS has downloaded the offline model — and the + user has granted the permission. Anything less is a guess that fails later, in the + middle of somebody's first sentence. + + **`compile_if_missing=False` is what `--engine auto` uses, and it is not a + micro-optimisation.** Measured in CI: with this asked unconditionally at startup, the + macOS suite went from 35 seconds to 643. Every launch on a Mac without Whisper models + was compiling Swift and then sitting on `--probe` until the timeout, because the + probe waits for an authorization dialog that a headless machine never answers — and a + user's first launch would have done exactly the same thing, for a full minute, before + showing a pill. + + So the rule is: **`auto` uses what is ready, and asking for the engine by name is + what builds it.** Once `--engine native` has been run once the binary is there, and + `auto` finds it from then on. + """ + if not compile_if_missing and sys.platform == "darwin" and not BINARY.exists(): + return False, ("not built yet - run once with --engine native to compile it") + try: + binary = build() + except NotAvailable as exc: + return False, str(exc) + try: + probe = _run([str(binary), "--probe"], timeout) + except subprocess.TimeoutExpired: + # Almost always the permission dialog, unanswered. Named as itself rather than + # as a generic failure, because the fix is a click and the user should hear so. + return False, ("probe timed out - grant Speech Recognition under " + "System Settings > Privacy & Security") + except (OSError, subprocess.SubprocessError) as exc: + return False, f"probe failed: {exc}" + if probe.returncode != 0: + return False, (probe.stderr or "probe refused").strip().splitlines()[-1] + return True, "" + + +class NativeTranscriber: + """Satisfies `asr.Transcriber` by talking to one long-lived helper process. + + Long-lived rather than one process per utterance, because starting a process and + authorising a recogniser costs more than the decode does. The process is started + lazily on the first decode — `load()` is what a caller uses to pay that cost at a + moment of its choosing, which is exactly what `Session._warm` already does for + Whisper's tiers. + """ + + #: Set so `Session._pump_health`'s idle-unload check reads something true. There is + #: no 605 MB to give back here — the models belong to the OS — but the session asks. + loading = False + + def __init__(self, binary: Path | None = None) -> None: + self._binary = Path(binary) if binary else None + self._proc: subprocess.Popen | None = None + #: One decode at a time over one pipe. `DecodeWorker` is single-threaded today, + #: and this makes that a guarantee of this class rather than a fact about its + #: caller. + self._lock = threading.Lock() + + # -- lifecycle --------------------------------------------------------- + + @property + def loaded(self) -> bool: + return self._proc is not None and self._proc.poll() is None + + def load(self, final=None) -> None: + """Start the helper. Idempotent, and the signature matches Whisper's tier load.""" + with self._lock: + self._ensure() + + def unload(self) -> None: + """Close the helper down. The idle path calls this; there is little to reclaim. + + Closing stdin is the documented way out — the helper returns from its read loop + and exits — so this asks before it insists. + """ + with self._lock: + proc, self._proc = self._proc, None + if proc is None: + return + try: + if proc.stdin: + proc.stdin.close() + proc.wait(timeout=2.0) + except (OSError, subprocess.SubprocessError): + pass + if proc.poll() is None: + proc.kill() + + def _ensure(self) -> None: + if self.loaded: + return + binary = self._binary or build() + try: + self._proc = subprocess.Popen( + [str(binary)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"could not start {binary}: {exc}") from exc + + # -- the one method the protocol asks for ------------------------------ + + def text(self, audio: np.ndarray, *, final: bool = False, + hotwords: str = "") -> str: + """Transcribe mono float32 at 16 kHz. + + `final` is accepted and ignored: Apple's recogniser has one quality, where + Whisper has a fast tier and a strong one. That is a real difference and not a + stub — there is no second model to reach for, so a partial and a final of the + same audio return the same words. + + `hotwords` is accepted and ignored too. Biasing has no equivalent here, so the + constrained re-decode that `Session.submit_rescue` performs comes back + unbiased rather than failing. + """ + block = np.ascontiguousarray(audio, dtype=np.float32) + if block.size == 0: + return "" + with self._lock: + self._ensure() + proc = self._proc + if proc is None or proc.stdin is None or proc.stdout is None: + raise NotAvailable("helper is not running") + try: + proc.stdin.write(struct.pack(" dict: return dict(value) if isinstance(value, dict) else {} +def _apps(value) -> dict: + """The per-app instructions as written: a table, or nothing. + + Shape only, exactly like `_hotkeys`, and the line it stops at is the same one. What + counts as an application name is `flow/inject.py`'s knowledge — it is whatever + `Target.process` reports — and that module binds `user32` at import, so it cannot be + read on a Mac while this one is loaded on every launch including Lite's. + + But the second half of `_hotkeys`' argument does not apply here, and the difference + is worth being clear about: an action name has five right answers and a typo is + knowable, whereas an executable name has as many right answers as there are programs + on the machine. There is no table to check against and no such thing as a name Flow + can prove wrong — an entry for an app that is not installed is not a mistake, it is + somebody who has not opened it yet. So nothing here refuses an entry by name the way + registration does, and a key that never matches simply never fires. + + Values are carried through untouched, including ones that are not strings. The + instruction is judged where it is used, which is the only place that knows whether it + has anything to say. + """ + return dict(value) if isinstance(value, dict) else {} + + def _counter(value) -> Counter: """`{phrase: positive count}`. Entries that are not that shape are dropped. @@ -170,6 +194,41 @@ def _counter(value) -> Counter: MAX_WORKSPACES = 5 +#: The shipped modifier-only chord (see `flow/hotkey.py`, `Chord`). +#: +#: Canonical *here* rather than in `flow/hotkey.py`, and the direction is deliberate: +#: this module is imported on every launch including Lite and on a Mac, while +#: `flow.hotkey` binds `user32` at import and cannot be imported at all off Windows. A +#: default the profile could not read without Win32 would be a default Lite could not +#: write back, and a field that vanishes from the file on one platform is worse than one +#: nobody can edit. `flow.hotkey` owns what the string *means*; this owns what it is. +CHORD_DEFAULT = "ctrl+win" + +#: The shipped panel width, by name. Spelled here and not imported from `flow/ui.py` +#: for the reason `CHORD_DEFAULT` is not imported from `flow/hotkey.py`: this module is +#: loaded on every launch including Lite's, and a default it could not read without the +#: UI would be a default Lite could not write back. `ui.panel_width` maps the name to +#: pixels and treats an unknown one as this, so the two cannot disagree about anything +#: except the spelling of a word — which `tests/test_overlay.py` checks. +PANEL_DEFAULT = "regular" + +#: Where the stack sits, by name — bottom-centre of the monitor under the pointer, or +#: the bottom-right corner Flow shipped. Spelled here rather than imported from +#: `flow/ui.py` for `PANEL_DEFAULT`'s reason, and judged there for the same one. +PLACE_DEFAULT = "bottom" + +#: Which gesture the chord is: `"hold"` for push-to-talk, `"toggle"` for the original +#: press-and-release. Spelled here rather than imported from `flow/hotkey.py` for +#: `CHORD_DEFAULT`'s reason — that module binds user32 at import and cannot load on a +#: Mac, and this one is read on every launch including Lite's. +GESTURE_DEFAULT = "hold" + +#: How many model names the settings menu will remember. A ceiling rather than a +#: judgement: this list is only ever appended to, by hand, one name at a time, and a menu +#: is not a place for an unbounded list. +CLI_MODEL_CAP = 12 + + def path_key(path: str | None) -> str | None: """One identity for one folder, however it was spelled. @@ -328,6 +387,37 @@ def __init__(self, path: Path | str | None = None) -> None: #: less than that: `_hotkeys` carries an entry whose value is not a string rather #: than dropping it, so that registration can refuse it by name. self.hotkeys: dict[str, str] = {} + #: The modifier-only chord, as written. `hotkey.parse_chord` judges what it + #: means, for the same reason `hotkeys` is judged there: this module is imported + #: on every launch including Lite, and `flow.hotkey` binds `user32` at import. + self.chord: str = CHORD_DEFAULT + #: exe name -> an extra instruction for rewrites made while that app is in front. + self.apps: dict[str, str] = {} + #: Which of `ui.PANEL_WIDTHS` the draft panel is drawn at. A name and not a + #: number: three widths that have each been drawn are worth more than an integer + #: nobody has rendered at, and the meaning is judged in `flow/ui.py` for the same + #: reason the hotkey table's is judged in `flow/hotkey.py` — this module is read + #: on every launch and must not need the module that knows what it means. + self.panel: str = PANEL_DEFAULT + #: Where the panels open. `"bottom"` is the shipped placement now — bottom-centre + #: of whichever monitor the pointer is on — and `"corner"` is the bottom-right + #: Flow used to use, kept for anybody who wants it back rather than removed. + self.place: str = PLACE_DEFAULT + #: How the chord behaves. Both gestures ship because neither replaces the other: + #: a hold is better for a sentence, a toggle is the only one that survives a + #: paragraph or a pair of hands that cannot hold two keys down. Judged in + #: `flow/hotkey.py`, which knows what the words mean. + self.gesture: str = GESTURE_DEFAULT + #: Which model to ask the agent CLI for, "" meaning whatever it defaults to, and + #: how hard to let it think. Both apply to whichever CLI answers - `refine.tuned` + #: drops either for a CLI not measured to accept it. + self.cli_model: str = "" + self.cli_effort: str = EFFORT_DEFAULT + #: Every model name that has been set, in the order they were first used. The + #: settings menu has no text field to type one into and is not getting one, so + #: this list *is* the menu: a name arrives once through `--cli-model` and is a + #: click from then on. + self.cli_models: tuple[str, ...] = () #: Field names that were present in the file and unusable, so a caller can say so #: rather than leaving the user to notice their setting reverted. Empty on a first #: run and on any valid file. @@ -398,6 +488,23 @@ def take(key, validate, default=None, stored=None): # names live — see `_hotkeys`, and `hotkey.overridden` for what it says about # each entry it refuses. self.hotkeys = take("hotkeys", lambda v, _d: _hotkeys(v), {}) + # Absent means the shipped chord, and the empty string means "off" — somebody + # who does not want a global keyboard hook needs a way to say so that is not + # deleting the key, because the next save would write it straight back. + self.chord = take("chord", _text, CHORD_DEFAULT) + # Same bargain as `hotkeys`: a value that is not a table degrades to none and is + # named, and what is *in* the table is judged where it is used. + self.apps = take("apps", lambda v, _d: _apps(v), {}) + self.panel = take("panel", _text, PANEL_DEFAULT) + self.place = take("place", _text, PLACE_DEFAULT) + self.gesture = take("gesture", _text, GESTURE_DEFAULT) + self.cli_model = take("cli_model", _text, "") + self.cli_effort = take("cli_effort", _text, EFFORT_DEFAULT) + if self.cli_effort not in EFFORTS: + self.cli_effort = EFFORT_DEFAULT + self.cli_models = tuple( + take("cli_models", lambda v, _d=None: _text_list(v, CLI_MODEL_CAP), []) + ) self.pairs = take("pairs", lambda v, _d: _counter(v), Counter()) self.misroutes = take("misroutes", lambda v, _d: _counter(v), Counter()) # `stored=[]` because JSON has no set: `save` writes this one as a sorted list, @@ -428,6 +535,17 @@ def save(self) -> bool: # the only advertisement this feature gets in a project with no settings # dialog to put it in. "hotkeys": dict(self.hotkeys), + "chord": self.chord, + # Written back as it was read, so a hand-edit survives every save Flow makes + # on its own — and an empty table lands in every profile, which is the only + # advertisement this feature gets in a project with no settings dialog. + "apps": dict(self.apps), + "panel": self.panel, + "place": self.place, + "gesture": self.gesture, + "cli_model": self.cli_model, + "cli_effort": self.cli_effort, + "cli_models": list(self.cli_models), "pairs": dict(self.pairs.most_common(MAX_PAIRS)), "misroutes": dict(self.misroutes.most_common(MAX_MISROUTES)), # Sorted so two saves of the same state produce the same file — a set's diff --git a/flow/refine.py b/flow/refine.py index 81bf8e1..39015ac 100644 --- a/flow/refine.py +++ b/flow/refine.py @@ -21,7 +21,7 @@ import subprocess import threading import time -from dataclasses import dataclass +from dataclasses import dataclass, replace #: R11: never hand the CLI an unbounded draft. Past this, only the tail is sent. MAX_CHARS = 2000 @@ -136,6 +136,40 @@ def sane_timeout(value) -> float: _POLISH_SLACK = 600 +#: The per-app instruction, wrapped so the model can tell it from the user's own words. +#: +#: **Named as the destination rather than as a rule**, because that is what makes it +#: obey without over-obeying. "This text is going into Slack" is a fact the model can +#: weigh against the request; "always be informal" is a competing order, and a competing +#: order beats the instruction the user just spoke — which is the failure that would make +#: this feature worse than not having it. +#: +#: **Placed before the request, not after.** Both prompts end with the text, and the +#: sentence nearest the text is the one that wins ties. The user's actual instruction +#: has to be that sentence: a per-app note is a standing preference, and the whole point +#: of speaking is to override a standing preference when this one is different. +_APP_PROMPT = ( + "This text is going into {app}. Bear this in mind, without letting it override " + "anything asked for below: {note}" +) + + +def app_note(app: str, note) -> str: + """The `_APP_PROMPT` block for `app`, or "" when there is nothing to say. + + Everything that could be missing is treated as nothing to say, and deliberately so: + an unreadable entry in a hand-written table must cost that entry and not the rewrite. + The profile carries values through untouched (`profile._apps`), which is what leaves + the judging here — this is the only place that knows whether an instruction has any + content, and a non-string is simply an entry with none. + """ + if not isinstance(note, str) or not note.strip() or not app: + return "" + # `chr(10)` rather than an escape, the idiom this module already uses for every + # prompt it assembles — these strings are read far more often than they are edited. + return _APP_PROMPT.format(app=app, note=note.strip()) + chr(10) + chr(10) + + @dataclass(frozen=True) class Cli: name: str @@ -177,6 +211,24 @@ class Cli: #: for everything that already fits, which is most things — see `ui.Pill.MARKER_MAX` #: for the wall and `CANDIDATES` for the one entry that has hit it. marker: str = "" + #: The flag that names a model, or empty where this CLI has not been measured to take + #: one. Read out of each CLI's own `--help` on a machine that has it, on 2026-08-31: + #: `-m, --model ` for codex, `--model ` for claude, `--model ` + #: for kiro-cli. The same discipline `verified` carries — a flag nobody has seen a + #: CLI print is a guess. + model_flag: str = "" + #: The flag that sets reasoning effort, or empty where this CLI does not offer one. + #: claude and kiro-cli both print `(low, medium, high, xhigh, max)` beside it. codex + #: has none: its only route is `-c model_reasoning_effort=…`, a config key that does + #: not appear in its help, and writing one down from memory is the thing `verified` + #: exists to forbid. + effort_flag: str = "" + #: Where in `argv` a tuning flag may be inserted. Not the end: codex's argv finishes + #: with `-`, the positional that tells it the prompt is on stdin, and a flag after it + #: would be read as its value. Not the front either, because `exec` and `chat` are + #: subcommands and a flag before them belongs to a different parser. So each entry + #: says where its own flags start, which is immediately after its subcommand. + tune_at: int = 1 #: Order is the preference order — codex first, per R10. @@ -256,11 +308,14 @@ class Cli: #: shapes, not worked around here. CANDIDATES: tuple[Cli, ...] = ( Cli("codex", ("codex", "exec", "--skip-git-repo-check", "-s", "read-only", - "-c", "project_doc_max_bytes=0", "-"), stdin_ok=True), - Cli("claude", ("claude", "--safe-mode", "-p"), stdin_ok=True), + "-c", "project_doc_max_bytes=0", "-"), stdin_ok=True, + model_flag="-m", tune_at=2), + Cli("claude", ("claude", "--safe-mode", "-p"), stdin_ok=True, + model_flag="--model", effort_flag="--effort"), Cli("kiro-cli", ("kiro-cli", "chat", "--no-interactive", "--trust-tools="), probe=(r"%LOCALAPPDATA%\Kiro-Cli\kiro-cli.exe",), - timeout_sec=60.0, marker="kiro"), + timeout_sec=60.0, marker="kiro", + model_flag="--model", effort_flag="--effort", tune_at=2), Cli("opencode", ("opencode",), verified=False), Cli("copilot", ("copilot",), verified=False), Cli("gemini", ("gemini",), verified=False), @@ -354,6 +409,52 @@ def trusted(path: str | None) -> str | None: return path +#: The effort levels claude and kiro-cli both enumerate beside `--effort`, in the order +#: they print them. Read out of their own `--help` rather than written from memory, and +#: identical across the two, which is the only reason one list can serve both. +EFFORTS: tuple[str, ...] = ("low", "medium", "high", "xhigh", "max") + +#: The level Flow asks for when nobody has said otherwise, and deliberately the cheapest. +#: +#: These calls are a *rewrite* — take what was dictated and make it read like a written +#: prompt — not a reasoning problem. Effort buys deliberation this task has no use for, +#: and pays for it in the one currency that matters here: the user is watching a spinner +#: between finishing a sentence and having it. The measured slowest verified call was +#: 35.8 s at whatever the CLI's own default was. +#: +#: Anyone who disagrees for their own model can say so — `--cli-effort`, or the settings +#: menu — and every level the CLIs offer is reachable from both. +EFFORT_DEFAULT = "low" + + +def tuned(cli: Cli, model: str = "", effort: str = EFFORT_DEFAULT) -> Cli: + """`cli` with a model and an effort level asked for, where it accepts them. + + A **new `Cli`** rather than an argument threaded through four call layers. `Cli` is + frozen and its argv is a tuple, so a tuned copy is the same kind of thing as the + original and every caller below — `resolve`, `_invoke`, `_clean`, the marker the pill + draws — goes on working without knowing this happened. + + Silently ignores what a CLI has not been measured to take. That is the point of + `model_flag` and `effort_flag` being per-entry: a user who picks a model while codex + is answering should get codex answering, not a crash and not an invented flag. What + they asked for is still recoverable — the name is on the profile and applies the + moment a CLI that takes one is chosen. + + Inserted at `tune_at`, which is after the subcommand and before codex's trailing `-`. + See that field for why neither end of the argv would do. + """ + extra: list[str] = [] + if effort and effort != "default" and cli.effort_flag: + extra += [cli.effort_flag, effort] + if model and cli.model_flag: + extra += [cli.model_flag, model] + if not extra: + return cli + at = max(1, min(cli.tune_at, len(cli.argv))) + return replace(cli, argv=cli.argv[:at] + tuple(extra) + cli.argv[at:]) + + def resolve(cli: Cli) -> str | None: """Where this CLI actually is, or None. The one answer detection and launch share. @@ -413,6 +514,8 @@ def _invoke_any( cwd: str | None = None, cancel: threading.Event | None = None, skipped: list[str] | None = None, + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str, Cli | None]: """Run `prompt`, falling through the preference order until one CLI answers. @@ -471,7 +574,11 @@ def _invoke_any( """ timeout = sane_timeout(timeout) if cli is not None: - out, reason = _invoke(cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel) + out, reason = _invoke(tuned(cli, model, effort), prompt, timeout=timeout, + cwd=cwd, cancel=cancel) + # The *untuned* entry goes back to the caller. What answered is still codex, and + # the pill's marker, the trace and `_clean`'s per-CLI stripping all key off that + # identity — a tuned copy is the same CLI with a flag on it, not another one. return out, reason, cli candidates = available() @@ -491,8 +598,12 @@ def _invoke_any( # fallback was built to end. reasons.append(f"no time left to try {candidate.name}") break - out, reason = _invoke(candidate, prompt, timeout=timeout, cwd=cwd, - cancel=cancel, cap=left) + # Tuned here rather than at the pin, so a fallback is asked for the same model + # and effort as the first choice. A walk that quietly reverted to a CLI's own + # defaults the moment the first candidate failed would be slowest exactly when + # the user is already waiting longest. + out, reason = _invoke(tuned(candidate, model, effort), prompt, timeout=timeout, + cwd=cwd, cancel=cancel, cap=left) if out is not None: if skipped is not None: skipped.extend(reasons) @@ -916,6 +1027,9 @@ def refine( context: list[str] | None = None, cancel: threading.Event | None = None, skipped: list[str] | None = None, + app: str = "", + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str]: """Apply a semantic instruction to `text`. @@ -930,6 +1044,10 @@ def refine( `cancel` abandons the call — the session sets it on close, so quitting does not wait out a rewrite nobody is going to read. + `app` is the per-app block from `app_note()` — already formatted, because the + profile table it comes from is the session's to read and this module has no business + knowing that `~/.flow/profile.json` exists. + `skipped` is an out-parameter rather than a third return value, and that is a judgement about blast radius: the two-tuple is unpacked at fifteen call sites across the tests and two scripts, none of which care which CLI was passed over on the way to @@ -947,6 +1065,12 @@ def refine( if polish else _PROMPT.format(instruction=instruction, text=tail) ) + # Applied to both routes on purpose. A polish ignores the spoken instruction, which + # makes it the pass with the *most* to gain from knowing where the words are headed — + # a prompt bound for a terminal and one bound for a chat window differ in exactly the + # way this note is for. + if app: + prompt = app + prompt if context: prior = chr(10).join(f"- {turn}" for turn in context) prompt = ( @@ -955,7 +1079,8 @@ def refine( ) out, reason, chosen = _invoke_any( - cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped + cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped, + model=model, effort=effort, ) if out is None: return None, reason @@ -984,6 +1109,8 @@ def ask( cancel: threading.Event | None = None, artifact: bool = False, skipped: list[str] | None = None, + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str]: """P9: put a question to the agent CLI and return its answer. @@ -1022,7 +1149,8 @@ def ask( ) out, reason, chosen = _invoke_any( - cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped + cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped, + model=model, effort=effort, ) if out is None: return None, reason diff --git a/flow/session.py b/flow/session.py index 8109eac..c15ce81 100644 --- a/flow/session.py +++ b/flow/session.py @@ -52,7 +52,8 @@ from .profile import path_key from .refine import TIMEOUT_SEC as REFINE_TIMEOUT_SEC from .refine import MAX_CHARS as REFINE_MAX_CHARS -from .refine import ask, available, refine, tail_sent +from .refine import EFFORT_DEFAULT, EFFORTS +from .refine import app_note, ask, available, refine, tail_sent from .thread import ASK_CONTEXT_CHARS, Thread # -- P4/P8: what a repair typed by hand teaches ------------------------------- @@ -295,11 +296,39 @@ def ask_framing(cwd: str | None) -> str: #: worker-idle check below, this is what bounds partial latency. PARTIAL_MIN_GROWTH_SEC = 0.7 -#: R8: drop the 141 MB model after a long quiet spell. The mic stays open — it is -#: cheap, and keeping it means speech still wakes the session with no keypress. This -#: is a deliberate narrowing of what docs/analysis.md §4 proposed (which released the -#: mic too): releasing it would make the app unable to hear its own wake-up. -IDLE_UNLOAD_SEC = 300.0 +#: R8: drop the models after a long quiet spell. The mic stays open — it is cheap, and +#: keeping it means speech still wakes the session with no keypress. This is a +#: deliberate narrowing of what docs/analysis.md §4 proposed (which released the mic +#: too): releasing it would make the app unable to hear its own wake-up. +#: +#: **"the 141 MB model" is what this comment used to say, and it was one tier out of +#: date.** Two are resident — `base.en` for partials at 141 MB and `small.en` for finals +#: at 464 MB — so what the idle path gives back is ~605 MB, not 141. The number is worth +#: correcting rather than rounding past: it is the entire case for unloading at all, and +#: it is four times better than the sentence defending it claimed. +#: +#: **Thirty minutes, not five.** Five was measuring the wrong thing — it asked how long +#: the session had been quiet, and answered as if quiet meant gone. It does not: the gaps +#: inside an ordinary working session run well past five minutes, so the common case was +#: not "reclaim memory from somebody who left", it was "pay a reload in the middle of +#: somebody's first sentence back". Half an hour rides out the gaps in a day and still +#: hands the memory back overnight, which is the case the unload was written for. +IDLE_UNLOAD_SEC = 1800.0 + +#: How long a warm request holds the models against the idle unload. +#: +#: Exists because the two clocks disagreed about what "idle" means. `_last_activity` is +#: only moved by Flow's own milestones, so a person who has just *reached for the chord* +#: is still idle by that measure — and the health pump, which runs every tick, could +#: unload the models between the press and the release that starts capture. That is the +#: one moment the warm exists to cover, and it is exactly the moment it would have lost. +#: +#: A grace window rather than a touch of `_last_activity`, because they answer different +#: questions and conflating them costs the unload its meaning: a chord press-down also +#: arrives from Windows' own `ctrl+win+arrow`, and letting that reset the idle clock +#: would mean anybody who switches virtual desktops through the day never unloads at +#: all. Sixty seconds covers press-hold-release-speak and then stops mattering. +WARM_GRACE_SEC = 60.0 #: How long to wait between attempts to reopen a microphone that went away. #: @@ -729,6 +758,11 @@ def __init__( #: the same brain — so this is read in exactly one place, the note that would #: otherwise name a focused window Lite cannot see. self.lite = lite + #: Whether Send puts the words in the other window or on the clipboard. Set by + #: `__main__` from whether it imported an injector, because `lite` stopped being + #: the same question the day a Mac got a paste path: it is Lite there — no global + #: hotkeys, no window handles — and it pastes. + self.pastes = not lite self.mic = mic or Mic(device=device) self.gate = SpeechGate() self.worker = DecodeWorker(self.asr) @@ -831,12 +865,28 @@ def __init__( #: launch. `getattr` because a profile is optional and the fakes predate the #: field. self.auto_ask = bool(getattr(profile, "auto_ask", True)) + #: Which model to ask the agent CLI for, and how hard to make it think. Taken + #: from the profile so a choice made once survives a restart, and defaulted to + #: the CLI's own model and the cheapest effort — see `refine.EFFORT_DEFAULT`. + self.cli_model: str = str(getattr(profile, "cli_model", "") or "") + self.cli_effort: str = str(getattr(profile, "cli_effort", "") + or EFFORT_DEFAULT) #: When the draft last stopped changing. None means nothing is pending. self._settled_at: float | None = None #: P8. What Flow has measured and learned about this person, on this machine. #: None disables learning entirely — the tests and the benchmarks pass None so #: a harness run never writes to the user's real profile. self.profile = profile + #: The executable in front, as `inject.Target.process` spells it, or "". + #: + #: Written by the pill rather than read here, and that is the split that keeps + #: `OpenProcess` off a 30 fps path: `Pill._track_target` already asks who has the + #: foreground every frame, so it is the one place that knows when the answer + #: *changed* and a name only has to be resolved then. Lite leaves it empty — it + #: has no target-window awareness at all (product.md) — which reads here as an + #: app with nothing configured, and that is the correct behaviour rather than a + #: gap: a rewrite with no per-app note is what every launch did until now. + self.target_app = "" #: A content-free shadow of the event stream (see flow/diag.py). Off unless #: the caller passes one, for the same reason `profile=None` disables learning: #: the tests build sessions in their hundreds, and a default that wrote to @@ -923,6 +973,13 @@ def __init__( #: The one preload this session owns, or None before the first arm. Single, not #: one per arm: see `_warm`. self._preload_thread: threading.Thread | None = None + #: Deadline until which the idle unload stands down, set by `warm`. Zero means + #: nobody has asked, which is the state every session starts and mostly stays in. + self._warm_until = 0.0 + #: Whether the current push-to-talk hold is the thing that opened the mic. False + #: when the hold began against a session already capturing, which is what stops + #: a chord from closing a microphone the toggle hotkey opened. + self._ptt_opened = False # -- lifecycle --------------------------------------------------------- @@ -968,6 +1025,30 @@ def start(self) -> None: self._warm() self._set_state(State.IDLE) + def warm(self) -> None: + """Start loading the models now, before anybody asks them to decode anything. + + The chord's press-down calls this, and the release that follows is what actually + arms. That gap — a person holding two modifiers, about to let go and speak — is + free time the load used to spend nowhere: `start()` deliberately does not await + the preload (see the docstring there, and the first-run download it exists to + keep off the UI thread), so on a cold arm the reload landed *inside* the first + utterance instead of in front of it. Measured on the run that prompted this: + first partial 1 230 ms, the four behind it ~570 ms. + + Safe to call at any moment and as often as anybody likes. `_warm` is + single-flight and finds an already-loaded model instantly, so the cost of a + spurious call is a thread that starts and exits — which matters, because + `ctrl+win` is also the prefix Windows uses for `ctrl+win+d` and `ctrl+win+arrow` + and the hook cannot tell those from a real chord until the third key lands. + + The grace window is the half that is not just a preload: without it the health + pump, which runs every tick, is free to unload between this call and the arm it + is preparing for. See `WARM_GRACE_SEC`. + """ + self._warm_until = time.perf_counter() + WARM_GRACE_SEC + self._warm() + def _warm(self) -> None: """One preload at a time, however often this session is armed. @@ -1064,6 +1145,85 @@ def stop_speaking(self) -> bool: self._emit("note", "stopped reading the answer") return True + # -- push to talk ------------------------------------------------------ + # + # The chord's two halves, as session verbs. `Pill` owns the state machine between + # them — it is the thread that may touch Tk and the one that already knows what + # `armed` means — and these are the two things it cannot do from outside: opening + # capture for the length of a hold, and closing it *without* discarding what was + # said into it. + + @property + def busy(self) -> bool: + """True while a decode this session submitted is still in flight. + + Exists for the push-to-talk paste, which must wait for the *final* rather than + paste the partial that preceded it. Narrow on purpose: it answers "is the + decoder still working", not "is Flow doing anything" — a CLI refine has its own + state, and `send()` already refuses while one is running. + """ + return self.worker.busy + + def talk_start(self) -> bool: + """Open the microphone for a hold. True if capture is running when this returns. + + Idempotent against a session that is already capturing, and the return value is + the reason: somebody who armed with the toggle hotkey and then reaches for the + chord has a live microphone already, and re-opening it would cut the utterance + they are in the middle of. `talk_end` reads the same fact from `_ptt_opened` and + gives back only what this took. + + Raises what `start()` raises — no microphone, a device held exclusively by + something else. The caller renders that; a swallowed failure here would be a + hold that records nothing and says so nowhere. + """ + with self._lifecycle: + if self._closed: + return False + if self._mic_started: + self._ptt_opened = False + self.warm() + return True + self.start() + self._ptt_opened = True + return True + + def talk_end(self) -> bool: + """Close a hold. True if there is now a decode in flight worth waiting for. + + **`mic.stop()` and not `pause()`, and the difference is the whole gesture.** + `pause()` bumps the capture generation, which is precisely how a deliberate stop + refuses results decoded from before it — and under push-to-talk the words the + user just said are in flight *at this moment*. Pausing here would throw away the + utterance the release exists to send. `_give_up_on_device` reached the same + conclusion from the other direction and the comment there is the longer version. + + **What was said is always committed, including on the break path**, and there is + deliberately no minimum-length rule deciding otherwise. `_utter` is what the gate + let through, so a hold with nothing spoken into it is already empty and + `_finalise` already returns early — a `ctrl+win+d` costs nothing without a + threshold, and a threshold would be a number that eventually eats somebody's + one-word answer. Whether the words get *pasted* is the caller's decision and a + different question; whether they are kept is not up for debate (P2). + """ + pending = bool(self._utter) + self._finalise() + if self._ptt_opened: + try: + self.mic.stop() + except Exception: + pass # already gone; there is nothing left to close + self._mic_started = False + # The hygiene `pause()` does either side of the generation bump, minus the + # bump: a gate left open would resume the next hold mid-utterance with no + # onset, and blocks captured after the stop belong to nobody. + self.gate.reset() + self.mic.drain() + self._ptt_opened = False + self._settle_state() + self._emit("disarm", "push-to-talk") + return pending + def close(self) -> None: """Give back everything `start()` and the constructor took, in that order. @@ -1381,6 +1541,7 @@ def _pump_health(self) -> None: idle = now - self._last_activity if ( idle >= IDLE_UNLOAD_SEC + and now >= self._warm_until and not self.draft.text and not self.gate.speaking and not self.worker.busy @@ -2777,6 +2938,40 @@ def set_cli(self, cli) -> None: self._emit("note", f"agent CLI: {cli.name}" if cli is not None else "agent CLI: automatic, in preference order") + def set_cli_model(self, model: str) -> None: + """Which model the agent CLI should use, or "" for whatever it defaults to. + + Applies to whichever CLI answers, including a fallback — `refine.tuned` drops it + for any CLI not measured to take a `--model`, so a name set for one is simply + ignored by another rather than breaking it. + + Remembered as well as applied. There is no way to type a model name into the + settings menu — Flow has no text field anywhere and the settings docstring + refuses to grow a dialog — so the list of names somebody has used is the menu, + and it is built from what has been set here. + """ + model = model.strip() + self.cli_model = model + if self.profile is not None: + self.profile.cli_model = model + if model and model not in self.profile.cli_models: + self.profile.cli_models = (*self.profile.cli_models, model) + self.profile.save() + self._emit("note", f"model: {model}" if model else "model: the CLI's own default") + + def set_cli_effort(self, effort: str) -> None: + """How hard the CLI should think, lowest by default. + + These calls are a rewrite rather than a reasoning problem, and the user is + watching a spinner while they run — see `refine.EFFORT_DEFAULT`. Anyone who + wants deliberation from their own model can have it, per level, from here. + """ + self.cli_effort = effort + if self.profile is not None: + self.profile.cli_effort = effort + self.profile.save() + self._emit("note", f"effort: {effort}") + def toggle_auto_ask(self) -> bool: self.auto_ask = not self.auto_ask # Saved now rather than at the next Send, for the reason `set_voice` gives: this @@ -2804,6 +2999,28 @@ def _pump_auto_ask(self) -> None: # -- semantic refine (off-thread: ~7 s measured) ------------------------ + def _app_note(self) -> str: + """The per-app block for whatever is in front, or "". + + Matched case-insensitively on the executable name, because `"Code.exe"` and + `"code.exe"` are the same program and a table that cared would be a table whose + entries silently stop matching after a vendor changes the capitalisation of a + shipped binary. + + Every way of having nothing to say lands on "" — no profile, no table, an app + with no entry, an entry that is blank or is not a string. A rewrite without a + note is exactly what Flow did before this existed, so the degraded path is the + old behaviour rather than an error. + """ + table = getattr(self.profile, "apps", None) if self.profile else None + if not table or not self.target_app: + return "" + wanted = self.target_app.lower() + for name, note in table.items(): + if isinstance(name, str) and name.strip().lower() == wanted: + return app_note(self.target_app, note) + return "" + def _start_refine(self, instruction: str, *, polish: bool = False) -> None: if self._refine_op is not None: # The refusal `send()` already makes, for the same reason. Two rewrites of @@ -2839,6 +3056,13 @@ def _start_refine(self, instruction: str, *, polish: bool = False) -> None: ) context = self.thread.tail() if self.following_up else [] + # Resolved here and not on the worker, because `target_app` is written by the UI + # thread every frame and the worker runs for the ~7 s the CLI takes. Reading it + # there would let the app the user tabbed to *during* the rewrite decide how the + # words they already spoke come out. + app = self._app_note() + if app: + self._emit("note", f"using your {self.target_app} note") def work() -> None: passed_over: list[str] = [] @@ -2846,7 +3070,8 @@ def work() -> None: before, instruction, cwd=self._refine_cwd, polish=polish, context=context, cancel=self._cancel, cli=self._cli, timeout=self._cli_timeout, - skipped=passed_over, + model=self.cli_model, effort=self.cli_effort, + skipped=passed_over, app=app, ) with self._refine_lock: self._refine_result = (op, revision, result, tuple(passed_over)) @@ -3004,9 +3229,9 @@ def toggle_mode(self) -> str: "converse mode - no agent CLI on PATH, so Ask has nothing to send") self._first_converse_notice() else: - self._emit("note", "dictate mode - Send copies the draft, and you paste it" - if self.lite - else "dictate mode - Send pastes into the focused window") + self._emit("note", "dictate mode - Send pastes into the focused window" + if self.pastes + else "dictate mode - Send copies the draft, and you paste it") return self.mode def send(self) -> str: @@ -3128,6 +3353,7 @@ def work() -> None: result = ask(framed, cwd=self._refine_cwd, context=context, cancel=self._cancel, artifact=artifact, cli=self._cli, timeout=self._cli_timeout, + model=self.cli_model, effort=self.cli_effort, skipped=passed_over) with self._ask_lock: # Written after `ask` returns and read under this lock, which is what diff --git a/flow/tray.py b/flow/tray.py new file mode 100644 index 0000000..2906f97 --- /dev/null +++ b/flow/tray.py @@ -0,0 +1,321 @@ +"""A notification-area icon, so Flow can be out of the way without being lost. + +The need, in the owner's words: "there are times where i wanted to dictate but at the +same time i wanted to see but i don't want it to keep it on my screen". The overlay can +already be parked off the desktop — `ui.park` does it — and parking alone is a trap. A +Flow with no window and no icon is a process you cannot reach, cannot configure and +cannot quit except through Task Manager, which is invariant 4's problem wearing a +different hat: hidden must not mean gone. + +So hiding gets an icon, and the icon is the way back. + +**Win32 through ctypes, no new dependency** (R16 holds at three). `Shell_NotifyIconW` +needs a window to send its click messages to, and Tk will not give us one — its window +procedure is Tcl's, and subclassing it to intercept a custom message would put our code +on the path of every event Tk handles. So this creates its own **message-only window** +(`HWND_MESSAGE`), which has no pixels, never draws and exists purely to receive. + +**It runs its own message loop on its own thread**, for the same reason. `GetMessageW` +blocks, and Tk's loop is not ours to block. The two never touch: the window procedure +runs on this thread, and everything it learns is put on a `queue.Queue` that the UI +drains from its own frame pump. Nothing here calls into Tk, which is the rule that makes +threading here safe rather than merely tested. + +**Stock icon, deliberately.** `IDI_APPLICATION` rather than an `.ico` shipped in the +package: an icon file is a binary asset in a repository that has none, and a tray icon +that is obviously a placeholder is more honest than one that took a build step to look +official. It is a line to change when Flow has artwork. +""" + +import ctypes +import queue +import sys +import threading +from ctypes import wintypes + +#: What the icon puts on the queue. Strings rather than callbacks, because the callback +#: would then run on this module's thread — and the one rule here is that nothing this +#: file owns ever touches Tk. +SHOW = "show" +QUIT = "quit" + +#: The message the shell sends us for every click on the icon. `WM_APP` and above are +#: reserved for an application's own use, which is exactly what this is. +_WM_APP = 0x8000 +_WM_TRAY = _WM_APP + 1 + +#: Menu command ids. Any positive int the popup can return; they mean nothing outside it. +_ID_SHOW = 1 +_ID_QUIT = 2 + +_WM_DESTROY = 0x0002 +_WM_RBUTTONUP = 0x0205 +_WM_LBUTTONUP = 0x0202 +_WM_LBUTTONDBLCLK = 0x0203 +_WM_COMMAND = 0x0111 + +_NIM_ADD, _NIM_DELETE = 0x0, 0x2 +_NIF_MESSAGE, _NIF_ICON, _NIF_TIP = 0x1, 0x2, 0x4 +_IDI_APPLICATION = 32512 +_IMAGE_ICON = 1 +_LR_SHARED = 0x8000 +_HWND_MESSAGE = -3 +_MF_STRING = 0x0 +_TPM_RETURNCMD = 0x0100 +_TPM_RIGHTBUTTON = 0x0002 + +_LRESULT = ctypes.c_ssize_t +_WNDPROC = ctypes.WINFUNCTYPE( + _LRESULT, wintypes.HWND, wintypes.UINT, wintypes.WPARAM, wintypes.LPARAM +) + + +class _WNDCLASSEXW(ctypes.Structure): + _fields_ = [ + ("cbSize", wintypes.UINT), + ("style", wintypes.UINT), + ("lpfnWndProc", _WNDPROC), + ("cbClsExtra", ctypes.c_int), + ("cbWndExtra", ctypes.c_int), + ("hInstance", wintypes.HINSTANCE), + ("hIcon", wintypes.HICON), + ("hCursor", wintypes.HANDLE), + ("hbrBackground", wintypes.HBRUSH), + ("lpszMenuName", wintypes.LPCWSTR), + ("lpszClassName", wintypes.LPCWSTR), + ("hIconSm", wintypes.HICON), + ] + + +class _NOTIFYICONDATAW(ctypes.Structure): + """The shell's icon record. + + `szTip` is 128 wide characters in every version since Windows 2000 and `cbSize` is + how the shell knows which layout it is being handed — so it is `sizeof` this struct + and never a number typed in. + """ + + _fields_ = [ + ("cbSize", wintypes.DWORD), + ("hWnd", wintypes.HWND), + ("uID", wintypes.UINT), + ("uFlags", wintypes.UINT), + ("uCallbackMessage", wintypes.UINT), + ("hIcon", wintypes.HICON), + ("szTip", wintypes.WCHAR * 128), + ] + + +def _declare() -> None: + """Give ctypes the signatures, rather than letting it guess. + + Without this every call is assumed to return `c_int`, and a 64-bit `HWND` does not + fit in one: `CreateWindowExW` came back as `OverflowError: int too long to convert` + from inside the tray thread, where nothing was watching. The handle-returning calls + are the ones that matter, and the parent handle has to be a real `HWND` too — + `HWND_MESSAGE` is -3, which is only meaningful once ctypes knows it is a pointer. + """ + u = ctypes.windll.user32 + u.CreateWindowExW.restype = wintypes.HWND + u.CreateWindowExW.argtypes = [ + wintypes.DWORD, wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD, + ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, + wintypes.HWND, wintypes.HMENU, wintypes.HINSTANCE, wintypes.LPVOID, + ] + u.DefWindowProcW.restype = _LRESULT + u.DefWindowProcW.argtypes = [wintypes.HWND, wintypes.UINT, + wintypes.WPARAM, wintypes.LPARAM] + u.LoadImageW.restype = wintypes.HANDLE + u.LoadImageW.argtypes = [wintypes.HINSTANCE, wintypes.LPCWSTR, wintypes.UINT, + ctypes.c_int, ctypes.c_int, wintypes.UINT] + u.CreatePopupMenu.restype = wintypes.HMENU + u.TrackPopupMenu.restype = wintypes.BOOL + u.TrackPopupMenu.argtypes = [wintypes.HMENU, wintypes.UINT, ctypes.c_int, + ctypes.c_int, ctypes.c_int, wintypes.HWND, + wintypes.LPVOID] + u.DestroyWindow.argtypes = [wintypes.HWND] + u.PostMessageW.argtypes = [wintypes.HWND, wintypes.UINT, + wintypes.WPARAM, wintypes.LPARAM] + ctypes.windll.kernel32.GetModuleHandleW.restype = wintypes.HINSTANCE + ctypes.windll.kernel32.GetModuleHandleW.argtypes = [wintypes.LPCWSTR] + + +class Tray: + """One notification-area icon, or nothing at all if the shell refuses. + + Every failure is reported rather than raised. A tray icon is a convenience on top of + a working app, and an app that will not start because the notification area was busy + would be a worse trade than an app with no icon — `start()` answers False and the + caller keeps the pill on screen, which is the state everybody had before this file. + """ + + def __init__(self, title: str = "Flow", events: queue.Queue | None = None) -> None: + self.title = title + #: What the icon's clicks arrive on. Owned by the caller when it passes one, so + #: a UI can drain this and its own events from the same place. + self.events: queue.Queue = events if events is not None else queue.Queue() + self.hwnd = 0 + self._thread: threading.Thread | None = None + self._ready = threading.Event() + self._ok = False + #: The window procedure, kept alive here on purpose. ctypes callbacks are garbage + #: like anything else, and one collected while Windows still holds its address is + #: an access violation in a thread nobody is watching. + self._proc = _WNDPROC(self._on_message) + self._class = f"FlowTray{id(self):x}" + + # -- the thread that owns the window ------------------------------------ + + def start(self) -> bool: + """Put the icon in the notification area. Returns whether it is actually there. + + Blocks until the answer is known — at most a moment, and worth waiting for: a + caller that hid its window on the strength of an icon that never appeared would + have hidden it for good. + """ + if self._thread is not None: + return self._ok + self._thread = threading.Thread(target=self._serve, name="flow-tray", + daemon=True) + self._thread.start() + self._ready.wait(timeout=5.0) + return self._ok + + def stop(self) -> None: + """Take the icon away. Safe to call twice, and safe to call if it never worked.""" + if not self.hwnd: + return + try: + _shell().Shell_NotifyIconW(_NIM_DELETE, ctypes.byref(self._icon_data())) + ctypes.windll.user32.DestroyWindow(self.hwnd) + except OSError: + pass + self.hwnd = 0 + + def _icon_data(self, with_icon: bool = False) -> _NOTIFYICONDATAW: + data = _NOTIFYICONDATAW() + data.cbSize = ctypes.sizeof(_NOTIFYICONDATAW) + data.hWnd = self.hwnd + data.uID = 1 + data.uFlags = _NIF_MESSAGE | _NIF_ICON | _NIF_TIP + data.uCallbackMessage = _WM_TRAY + if with_icon: + # `MAKEINTRESOURCE`: a stock icon is identified by an integer squeezed + # into a string pointer, which is what `LPCWSTR(id)` builds here. + data.hIcon = ctypes.windll.user32.LoadImageW( + None, wintypes.LPCWSTR(_IDI_APPLICATION), _IMAGE_ICON, 0, 0, + _LR_SHARED) + data.szTip = self.title + return data + + def _serve(self) -> None: + """Register, create, add the icon, then pump messages until the window dies.""" + try: + self._ok = self._build() + except OSError: + self._ok = False + finally: + self._ready.set() + if not self._ok: + return + msg = wintypes.MSG() + user32 = ctypes.windll.user32 + while user32.GetMessageW(ctypes.byref(msg), None, 0, 0) > 0: + user32.TranslateMessage(ctypes.byref(msg)) + user32.DispatchMessageW(ctypes.byref(msg)) + + def _build(self) -> bool: + _declare() + user32 = ctypes.windll.user32 + cls = _WNDCLASSEXW() + cls.cbSize = ctypes.sizeof(_WNDCLASSEXW) + cls.lpfnWndProc = self._proc + cls.hInstance = ctypes.windll.kernel32.GetModuleHandleW(None) + cls.lpszClassName = self._class + if not user32.RegisterClassExW(ctypes.byref(cls)): + return False + # `HWND_MESSAGE` as the parent: a window with no pixels, no place on the desktop + # and no chance of being shown by accident. It exists to be sent to. + self.hwnd = user32.CreateWindowExW( + 0, self._class, self.title, 0, 0, 0, 0, 0, + wintypes.HWND(_HWND_MESSAGE), None, cls.hInstance, None) + if not self.hwnd: + return False + return bool(_shell().Shell_NotifyIconW( + _NIM_ADD, ctypes.byref(self._icon_data(with_icon=True)))) + + # -- what the shell tells us -------------------------------------------- + + def _on_message(self, hwnd, message, wparam, lparam) -> int: + """The window procedure. Runs on this module's thread and never touches Tk. + + Everything it decides goes on the queue; the UI acts on it from its own loop. + """ + if message == _WM_TRAY: + event = lparam & 0xFFFF + try: + if event in (_WM_LBUTTONUP, _WM_LBUTTONDBLCLK): + self.events.put(SHOW) + elif event == _WM_RBUTTONUP: + self._popup() + except Exception as exc: # pragma: no cover - belt for a callback + # Nothing may escape a ctypes callback: Windows called us, and an + # exception unwinding into its stack is undefined at best. Printed + # rather than swallowed, because a tray that silently stops answering + # is the failure this file exists to prevent. + print(f"flow: tray click failed: {exc}", file=sys.stderr, flush=True) + return 0 + if message == _WM_DESTROY: + ctypes.windll.user32.PostQuitMessage(0) + return 0 + return ctypes.windll.user32.DefWindowProcW( + wintypes.HWND(hwnd), wintypes.UINT(message), + wintypes.WPARAM(wparam), wintypes.LPARAM(lparam)) + + def _popup(self) -> None: + """The right-click menu: the two things a hidden app has to offer. + + `SetForegroundWindow` first, and the `PostMessage` after, are both from the + documented recipe: a popup owned by a window that is not foreground never + receives the click that dismisses it, and stays on screen until something else + is clicked. + """ + user32 = ctypes.windll.user32 + menu = user32.CreatePopupMenu() + if not menu: + return + try: + user32.AppendMenuW(menu, _MF_STRING, _ID_SHOW, "Show Flow") + user32.AppendMenuW(menu, _MF_STRING, _ID_QUIT, "Quit Flow") + point = wintypes.POINT() + user32.GetCursorPos(ctypes.byref(point)) + user32.SetForegroundWindow(self.hwnd) + chosen = user32.TrackPopupMenu( + menu, _TPM_RETURNCMD | _TPM_RIGHTBUTTON, + point.x, point.y, 0, self.hwnd, None) + # `PostMessageW`, with the W. There is no bare `PostMessage` export in + # user32 — the name is a macro in C that resolves to one of the two — so + # asking for it raised `AttributeError: function 'PostMessage' not found` + # *after* the menu had been chosen from and before anything acted on the + # choice. Right-clicking the icon showed the menu and then did nothing. + user32.PostMessageW(self.hwnd, 0, 0, 0) + finally: + user32.DestroyMenu(menu) + if chosen == _ID_SHOW: + self.events.put(SHOW) + elif chosen == _ID_QUIT: + self.events.put(QUIT) + + +def _shell(): + return ctypes.windll.shell32 + + +def available() -> bool: + """Whether this platform has a notification area at all. + + Windows-only by construction. macOS has a menu bar item and Linux has whatever the + desktop environment offers, and neither is `Shell_NotifyIcon` — so this says no + rather than pretending, and the caller keeps its window on screen. + """ + return sys.platform == "win32" diff --git a/flow/ui.py b/flow/ui.py index 3c37b2f..6fdffc4 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -13,7 +13,9 @@ from __future__ import annotations import ctypes +import math import os +import queue import sys import time import tkinter as tk @@ -39,9 +41,10 @@ ensure as ensure_lexicon, pairs, ) +from . import tray from .notes import Notes from .profile import path_key, resolve_workspace -from .refine import available +from .refine import EFFORT_DEFAULT, EFFORTS, available from .session import CONVERSE, DICTATE, Session, State from .stats import today_note from .thread import MAX_TURNS as THREAD_MAX_TURNS @@ -53,6 +56,10 @@ class _RECT(ctypes.Structure): ("right", ctypes.c_long), ("bottom", ctypes.c_long)] +class _POINT(ctypes.Structure): + _fields_ = [("x", ctypes.c_long), ("y", ctypes.c_long)] + + #: SystemParametersInfo(SPI_GETWORKAREA) _SPI_GETWORKAREA = 0x0030 @@ -72,7 +79,7 @@ def __getattr__(self, _name): if sys.platform == "win32": - from .inject import foreground_hwnd, owned_by_flow, take_warnings + from .inject import classify, foreground_hwnd, owned_by_flow, take_warnings #: Its own handle rather than `ctypes.windll.user32`, which is a process-wide cached #: object: declaring `restype` on it would change the signature under `inject.py` @@ -93,12 +100,21 @@ def __getattr__(self, _name): _set_style.restype = ctypes.c_ssize_t else: # `inject.py` is not made portable and is not imported: 470 lines of Win32 with no - # meaning in a body that never types into another window. Its three exports are the + # meaning in a body that never types into another window. Its four exports are the # only ones this module needs, and each has an honest answer with no hands — there is - # no foreground to find, nothing of Flow's to recognise, and no paste to warn about. + # no foreground to find, nothing of Flow's to recognise, no window to name, and no + # paste to warn about. def foreground_hwnd() -> int: return 0 + def classify(_hwnd): + # An unnamed app, which reads downstream as one with no per-app note — the same + # answer `_track_target` gives on Lite, and the behaviour every launch had before + # per-app notes existed. + from .inject import Target + + return Target() + def owned_by_flow(_hwnd) -> bool: return False @@ -147,7 +163,21 @@ def _no_activate(win) -> bool: so a call that did nothing and a call that worked hand back the same plausible number, and there is no other way to tell them apart. The one thing this window style has to be is true. + + **Off Windows this cannot take, and the app is built on that.** `_menu` borrows the + foreground on Windows precisely because a `WS_EX_NOACTIVATE` window would otherwise + get no input for its popup, and says in as many words that Lite needs none of it — + the window is in the activation chain like any other and the popup gets its input the + ordinary way. + + Aqua does offer an equivalent, `MacWindowStyle ... noActivates`, and asking for it + was a mistake: it took the windows out of that chain, and a window that never + activates does not take clicks either. Send stopped working on a Mac. The frame is + handled in `_shell_window` now, where `overrideredirect` already lives and where it + always belonged. """ + if sys.platform != "win32": + return False try: # The wrapper has to exist before it can be styled, and this is what creates it. win.update_idletasks() @@ -160,21 +190,75 @@ def _no_activate(win) -> bool: return False +def _bare_window(win) -> None: + """Take the frame off, by the means the platform will accept. + + `overrideredirect` is how this is done everywhere, and on Aqua it is the cause of + both faults reported from a Mac: click the app you want to dictate into and Flow's + window vanishes, and clicking Send does nothing. A probe of six variants split on + exactly this line — every window without it kept its place when another app came + forward and had its button reached by a click; every window with it was deaf and + gone. Which is a fair description of what it means: a window the window manager has + been told to stop managing. + + Tk 9 on Aqua has a frameless window that is still a window. A style mask is the set + of bits an NSWindow is built from, and the one that puts a title bar on it is + `titled` — so a mask with *no* bits is bare, and nothing else about the window has + been given away. Measured on a Mac at 0 px of decoration against the control's 28, + and its button was reached from the background. + + The obvious-looking alternative, an `NSPanel` with the `nonactivatingpanel` bit, is + not available: Tk answers `cannot change the class after the mac window is created` + even for a window that has never been mapped, and a `Toplevel` built with + `class_="NSPanel"` is not one either — that argument names a Tk class, not an + NSWindow one. It is also not needed. Nonactivating is about not stealing focus, and + these windows do not take focus in the first place. + + Falls back rather than fails: `-stylemask` arrived in Tk 9, and a Mac on 8.6 should + get the old behaviour rather than a window with a title bar on it. + """ + if sys.platform == "darwin": + try: + win.wm_attributes("-stylemask", "") + return + except tk.TclError: + pass # Tk 8.6: no style masks. `overrideredirect` is all there is. + win.overrideredirect(True) + + def _shell_window(win, lite: bool, alpha: float) -> str: """Apply the window attributes every Flow window shares, and return its background. Two of the five are Windows-only Tk attributes. `-transparentcolor` is what keys the magenta out, so without it the keyed colour is not invisible — it is a magenta rectangle where the app should be — and `-toolwindow` does not exist off Windows at - all. Asking for either is a `TclError` before anything is drawn. + all. Asking for either is a `TclError` before anything is drawn, which is why the + platform is part of the guard and not only `lite`. + + **It used to be only `lite`**, on the reasoning that `__main__` forces lite mode off + Windows so the two can never come apart. They came apart the first time something + other than `__main__` built a `Pill`: `scripts/mac_report.py` asked for full mode on + a Mac and got `bad attribute "-transparentcolor"` out of Tk before a window existed. + An invariant a caller has to know about is one a caller can miss, and this one is + cheap to enforce where it is true. + + **On Aqua it is `_bare_window` that does this**, and not with `overrideredirect`: + that line is what made Flow's windows there both deaf to clicks and gone the moment + another app came forward. Two earlier attempts to help it were both harm.** A Mac reported the pill wearing a title bar, and the cause was not this + line failing — it was `MacWindowStyle` being asked for *afterwards*, which put a + frame back on and took the window out of the activation chain, so Send stopped taking + clicks. Removing that call was the fix. A withdraw-and-remap cycle added alongside it, + to "force Aqua to rebuild the NSWindow", was solving a problem the style call had + created — and on a real machine it left the window hidden after the remap. Both are + gone. What is left is the line that was always doing the work. The background is returned rather than left to the caller so a window cannot be given one that contradicts what was applied to it. """ - win.overrideredirect(True) + _bare_window(win) win.attributes("-topmost", True) win.attributes("-alpha", alpha) - if lite: + if lite or sys.platform != "win32": return SHELL win.attributes("-transparentcolor", TRANSPARENT) win.attributes("-toolwindow", True) @@ -200,6 +284,356 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: return rect.left, rect.top, rect.right, rect.bottom +#: `_tk_work_area`'s answer, measured once. A module global because the measurement +#: costs a window and `_sync_monitor` asks every frame — measuring per frame would open +#: and destroy a Toplevel thirty times a second. +_TK_WORK: tuple | None = None + + +#: How far down an Aqua window asked for `+0+0` can plausibly be pushed by the menu bar: +#: 30 px on an ordinary display, more on a notched one. Past this, the window manager +#: honoured the request literally — as Windows does — and the number means nothing here. +_AQUA_MENU_MAX = 80 + +#: How tall an Aqua title bar can plausibly be. 28 px measured; the cap is loose because +#: it only has to separate "a title bar" from "the window went somewhere else entirely". +_AQUA_TITLE_MAX = 60 + +#: Where the probe is put to measure its own title bar — far enough down that no menu bar +#: can be clamping it, so the whole difference from what was asked for is decoration. +_AQUA_FREE_Y = 300 + + +def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: + """The visible frame on macOS, or None if this build cannot say. + + **The maximise probe does not work here.** `state("zoomed")` on Aqua neither raises + nor maximises. Asked to maximise a 200x120 window at +80+80, Tk 9.0.3 returned + (80, 108, 280, 228) — the same window, the same size, the position it was already + in, and no error. `_tk_work_area` rejected that and fell back to the whole screen, + `bottom_centre` stood the pill 24 px above 878, and the pill spent its life inside + the Dock. The close-up in the report was a picture of Dock icons. + + `wm maxsize` is the call that knows, and only on this platform: Tk's Aqua port + answers it from `[NSScreen visibleFrame]`, which is the screen less the menu bar and + the Dock. On Windows the same call answers with the whole screen even with a taskbar + present, which is why `_tk_work_area` measures instead of asking — this is the one + platform where the shortcut is the *better* instrument, not a lazier one. + + **Everything is measured from one probe, and that is the point.** `maxsize` gives a + size and no origin, so the origin has to come from somewhere else — and the first + version took it from a probe while taking the size from the caller's window. Those + are not the same window. `maxsize` is a maximum *content* size, so it is short by + whatever decoration its window wears: 735 from a titled probe against 763 from the + `overrideredirect` pill, on the same display, differing by exactly the 28 px title + bar. Adding one window's origin to another's size counted that title bar twice and + put the work area 28 px too low — the pill moved off the Dock and back onto it. + + So one probe answers all three, and its own decoration cancels out: + + **its title bar** — asked for a y far below any menu bar, so the whole difference + between what was asked and where the client area landed is decoration. + + **the menu bar** — asked for `+0+0`, where Aqua refuses to put a titled window; + where it lands, less the title bar just measured, is the top of the visible frame. + + **the visible frame's height** — `maxsize` plus that same title bar, which is what + turns a content size back into a frame size. + + Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: a 28 px title bar, a + `+0+0` client top of 58 giving a menu bar of 30, and `maxsize` 1352x735 giving a + frame height of 763. The Dock's top edge is 30 + 763 = 793 and the Dock is 85 px + tall. All three are Tk asking Tk, so they were checked against something that is not: + `defaults read com.apple.dock tilesize` on the same machine says 69, and 69 plus + Apple's padding is the 85 this leaves. + + **Nothing is trusted without a shape check.** A `maxsize` of the whole screen has + told us nothing; a title bar or a menu bar outside the range one can be is a window + manager that honoured a request literally rather than clamping it, as Windows does. + Any of those and this returns None and the caller falls through to the maximise + probe, so the worst case is exactly the old behaviour rather than a new way to be + wrong. + """ + probe = None + try: + probe = tk.Toplevel(win) + probe.attributes("-alpha", 0.0) + + probe.geometry(f"200x120+80+{_AQUA_FREE_Y}") + probe.update_idletasks() + title = probe.winfo_rooty() - _AQUA_FREE_Y + if not 0 <= title <= _AQUA_TITLE_MAX: + return None + + probe.geometry("200x120+0+0") + probe.update_idletasks() + top = probe.winfo_rooty() - title + if not 0 <= top <= _AQUA_MENU_MAX: + return None + + mw, mh = probe.maxsize() + except (tk.TclError, AttributeError, TypeError, ValueError): + return None + finally: + if probe is not None: + try: + probe.destroy() + except tk.TclError: + pass + + height = mh + title + if not (0 < mw <= sw and 0 < height < sh and top + height <= sh): + return None # it answered with the whole screen, or the parts disagree + return 0, top, mw, top + height + + +def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: + """The usable area, measured by asking the window manager to maximise something. + + Reported from a Mac: the pill sat under the Dock. `_work_area` degrades to the whole + screen off Windows, so bottom-centre placement stood the stack on the very bottom + edge — behind the Dock on a default macOS desktop, behind the panel on a + bottom-taskbar Linux. + + **Measured rather than asked, because asking does not work.** `wm_maxsize` is the + obvious call and it is useless: on Windows it answers with the whole screen even + with a taskbar present, so a fallback built on it would have been wrong in exactly + the way it was meant to fix. What *is* reliable is maximising a window and looking + at where the window manager put it — it has to honour its own panels to do that. + Checked against `SystemParametersInfoW` on Windows, where the two agree exactly on + left, right and bottom. + + **macOS is the exception and is handled before any of this**, in `_aqua_work_area`: + there the maximise is accepted and ignored, and `wm maxsize` — useless on Windows — + is the call that knows where the Dock is. That path returns None unless what it + measured has the shape of a real work area, so this measurement stays the fallback. + + The probe is transparent while it is measured, so nothing flashes on screen. + + **`top` is taken as reported and is a title bar too low.** `winfo_rooty` is the + client area, and the frame inset differs between a normal window and a maximised one + — measuring the inset first and subtracting it made the answer worse, not better + (−8 against a true 0). It is left alone because `top` feeds one thing, the ceiling + in `bottom_centre`, where being conservative by a title bar costs nothing. The three + edges that place the stack are exact. + """ + global _TK_WORK + if _TK_WORK is not None: + return _TK_WORK + if sys.platform == "darwin": + aqua = _aqua_work_area(win, sw, sh) + if aqua is not None: + _TK_WORK = aqua + return aqua + fallback = (0, 0, sw, sh) + asked_w, asked_h = 200, 120 + probe = None + try: + probe = tk.Toplevel(win) + probe.attributes("-alpha", 0.0) + probe.geometry(f"{asked_w}x{asked_h}+80+80") + probe.state("zoomed") + probe.update_idletasks() + x, y = probe.winfo_rootx(), probe.winfo_rooty() + w, h = probe.winfo_width(), probe.winfo_height() + found = (x, y, x + w, y + h) + except (tk.TclError, AttributeError, ValueError): + # `zoomed` is documented for Windows and X11 and may not exist on this build. + # The whole screen is the honest answer then: wrong by a Dock, rather than + # wrong by whatever a broken measurement returned. + found = fallback + finally: + if probe is not None: + try: + probe.destroy() + except tk.TclError: + pass + if not _plausible_work_area(found, sw, sh, asked_w, asked_h): + found = fallback + _TK_WORK = found + return found + + +def _plausible_work_area(rect, sw: int, sh: int, asked_w: int, asked_h: int) -> bool: + """Whether a maximised probe actually got maximised. + + **The check that was missing, and the bug it let through.** On Aqua, + `state("zoomed")` does not raise and does not maximise either — it is accepted and + ignored. The probe stayed the 200x120 it was asked for, that rectangle passed a + check which only asked "positive, and no bigger than the screen", and the pill was + placed against a work area 200 px wide. It landed in the top-left corner of a + 1512-wide display, which is exactly where a Mac reported finding it. + + So the test is not "is this a rectangle" but "did the window manager do the thing". + Two ways of asking, because either alone has a hole: a window that never grew is the + direct evidence, and a rectangle far smaller than the display is what catches a + window manager that grew it a little and stopped. A real work area is the screen + minus a Dock or a taskbar — nowhere near half of it. + """ + left, top, right, bottom = rect + w, h = right - left, bottom - top + if not (w > 0 and h > 0 and w <= sw and h <= sh): + return False + if w <= asked_w or h <= asked_h: + return False # it never grew: `zoomed` was accepted and ignored + return w >= sw * 0.6 and h >= sh * 0.6 + + +#: `MonitorFromPoint`'s "nearest monitor" flag, for a cursor that is briefly nowhere — +#: between two displays, or on a monitor that has just been unplugged. +_MONITOR_DEFAULTTONEAREST = 2 + + +class _MONITORINFO(ctypes.Structure): + _fields_ = [ + ("cbSize", ctypes.c_ulong), + ("rcMonitor", _RECT), + ("rcWork", _RECT), + ("dwFlags", ctypes.c_ulong), + ] + + +def _pointer_monitor(sw: int, sh: int, win=None) -> tuple[tuple, tuple]: + """`(full, work)` for the monitor under the mouse, each `(left, top, right, bottom)`. + + **Two rectangles, because FluidVoice places against two.** `positionWindow` centres + on `screen.frame` but sits the overlay on `screen.visibleFrame`, and the asymmetry is + deliberate: centred on the *physical* display, so it lands where the eye expects it, + but lifted clear of the Dock. Windows hands back exactly that pair — `rcMonitor` and + `rcWork` — from one call, so the rule ports without being reinterpreted. + + **The monitor under the pointer, not the primary one.** `_work_area` asks + `SystemParametersInfoW`, which only ever answers for the primary display, so on a + two-monitor desk everything Flow draws lands on the wrong one whenever the user is + working on the other. FluidVoice resolves this per presentation + (`OverlayScreenResolver.screenForCurrentPointer`, and `preferredPresentationScreen` + in the notch path does the same), and the pointer is the right proxy: it is where + the user's attention is, and it costs nothing to ask. + + Falls back to the primary work area, then to the whole screen, so a machine where + the call is unavailable places exactly where it placed before. + """ + pt = _POINT() + try: + user32 = ctypes.windll.user32 + if user32.GetCursorPos(ctypes.byref(pt)): + handle = user32.MonitorFromPoint(pt, _MONITOR_DEFAULTTONEAREST) + info = _MONITORINFO() + info.cbSize = ctypes.sizeof(_MONITORINFO) + if handle and user32.GetMonitorInfoW(handle, ctypes.byref(info)): + full = (info.rcMonitor.left, info.rcMonitor.top, + info.rcMonitor.right, info.rcMonitor.bottom) + work = (info.rcWork.left, info.rcWork.top, + info.rcWork.right, info.rcWork.bottom) + if full[2] > full[0] and work[2] > work[0]: + return full, work + except (AttributeError, OSError): + pass + # Off Windows there is no `MonitorFromPoint` and no `rcWork`, so the two rectangles + # collapse into the one Tk can answer for. `win` is optional because the fallback + # has to keep working for the callers that have no window yet. + work = _tk_work_area(win, sw, sh) if win is not None else _work_area(sw, sh) + return work, work + + +#: `GetSystemMetrics` indices for the bounding box of every monitor together. +_SM_XVIRTUALSCREEN, _SM_YVIRTUALSCREEN = 76, 77 +_SM_CXVIRTUALSCREEN, _SM_CYVIRTUALSCREEN = 78, 79 + +#: How far past the desktop a hidden panel is parked. FluidVoice's number +#: (`parkWindowOffscreen`), and generous on purpose: it has to clear whatever monitor +#: someone plugs in next, not merely the ones present when the window was hidden. +PARK_MARGIN = 1024 + + +def _virtual_desktop(sw: int, sh: int) -> tuple[int, int, int, int]: + """Every monitor's bounding box, which is what a parked window has to clear. + + The union rather than the current monitor, for the same reason FluidVoice unions + `NSScreen.screens`: a window parked past the right edge of the *left* display in a + two-monitor desk is parked in the middle of the right one, in full view. + """ + try: + metric = ctypes.windll.user32.GetSystemMetrics + x, y = metric(_SM_XVIRTUALSCREEN), metric(_SM_YVIRTUALSCREEN) + w, h = metric(_SM_CXVIRTUALSCREEN), metric(_SM_CYVIRTUALSCREEN) + if w > 0 and h > 0: + return x, y, x + w, y + h + except (AttributeError, OSError): + pass + return 0, 0, sw, sh + + +def park_spot(w: int, h: int, desktop) -> tuple[int, int]: + """Where a hidden panel waits: past the far corner of every monitor there is. + + **Parked rather than unmapped, which is the point.** FluidVoice never destroys or + hides its overlay — `prepare()` builds it at launch and parks it, and `show` pulls + it back — with the reason written on the method: it is paying the window-server + surface cost once, so that appearing costs a move and nothing else. Under + push-to-talk that is the difference that matters, because the overlay now has to be + up between a key going down and somebody starting to talk, which is a tenth of a + second on a fast day. + + Its own function so the arithmetic is testable without a window, and so the one + thing that must never be true — a parked panel landing on a monitor somebody is + looking at — is a property a test can state. + """ + _left, _top, right, bottom = desktop + return right + w + PARK_MARGIN, bottom + h + PARK_MARGIN + + +def park(win) -> None: + """Hide `win` by moving it off every monitor, rather than by unmapping it. + + `withdraw()` was what the panels did, and it is the honest thing for a window nobody + will want again soon. Push-to-talk made that untrue: a panel now has to be up between + a key going down and somebody starting to speak, and a remap is work done in exactly + that gap. Parking is FluidVoice's answer (`parkWindowOffscreen`, with `prepare()` + paying the surface cost at launch), and it makes appearing a move and nothing else. + + A function rather than a method because `Bubble` and `ConversationCard` share no base + class — they share a *job*, and this is the third thing they have both needed. The + window is re-placed by `reposition` on the way back, which `_render` already calls, + so there is no unparking step to forget. + """ + w = max(1, win.width) + h = max(1, getattr(win, "_h", 1)) + x, y = park_spot(w, h, _virtual_desktop( + win.winfo_screenwidth(), win.winfo_screenheight())) + win.geometry(f"{w}x{h}+{x}+{y}") + + +def bottom_centre(w: int, h: int, full, work, offset: int = 0) -> tuple[int, int]: + """Where a panel of `w`×`h` goes, by FluidVoice's `positionWindow` arithmetic. + + Centred horizontally on `full` and stood `offset` above the bottom of `work`, then + clamped into `work` with the same two buffers FluidVoice uses — 10 px off the bottom + and 40 px off the top. The clamp is what makes the offset safe to expose as a + setting: a number typed into a profile cannot push the panel off the screen, and a + panel taller than the display lands against the bottom rather than above the top. + + The y arithmetic is flipped from the original and means the same thing. macOS + measures up from the bottom, so `visibleFrame.minY + offset` is the panel's bottom + edge; Windows measures down from the top, so the same edge is + `work.bottom - h - offset`. + """ + left, top, right, bottom = work + x = (full[0] + full[2]) // 2 - w // 2 + y = bottom - h - int(offset) + lowest = bottom - h - 10 + highest = top + 40 + if highest > lowest: + # A panel too tall for the display it is on. The bottom is the edge worth + # keeping: the top of a draft can run under the taskbar and still be read, and + # the chips that act on it live at the bottom. + y = lowest + else: + y = max(highest, min(y, lowest)) + x = max(left, min(x, right - w)) + return x, y + + def _dpi_aware() -> float: """Tell Windows this process draws its own pixels, and return the scale factor. @@ -409,6 +843,34 @@ def _unload_fonts() -> None: BAR_W, BAR_GAP = 4, 2 DB_FLOOR, DB_CEIL = -58.0, -12.0 # level range mapped onto bar height +#: The meter's shape, taken from FluidVoice's `BottomWaveformView` (`visualizerPeakHeight` +#: and `updateBars`) rather than invented here. +#: +#: **This changed what the meter *is*.** Flow's bars used to be a scrolling history — a +#: level per frame, pushed through a deque, so the shape travelled right to left like a +#: seismograph. FluidVoice's are a symmetric bloom: every bar is driven by the *same* +#: current level and shaped by a fixed envelope that makes the middle ones tallest, so +#: the meter breathes in place instead of scrolling. +#: +#: The bloom is the better answer to the question R13 says this widget exists to answer — +#: *am I being heard right now*. A history answers "was I heard, recently", and the eye +#: has to read left to right to get at it; a bloom answers it in one glance with no +#: direction to follow. It is also what makes the meter read as one object rather than +#: twelve, which is the whole visual difference between the two apps. +#: +#: `_ENVELOPE_FLOOR`/`_ENVELOPE_SPAN`: `factor = max(0.18, 0.96 - distance * 0.78)`, +#: where distance is 0 at the centre bar and 1 at the ends. `_LEVEL_EXPONENT`: normal +#: speech should push the bars high, so the response is deliberately not linear. +#: `_BAR_VARIATION`: a per-bar wobble of ±8%, so a held tone is not twelve identical +#: rectangles. +_ENVELOPE_FLOOR, _ENVELOPE_SPAN, _ENVELOPE_MIN = 0.96, 0.78, 0.18 +_LEVEL_EXPONENT = 0.55 +_BAR_VARIATION_BASE, _BAR_VARIATION_SWING, _BAR_VARIATION_RATE = 0.92, 0.08, 1.45 +#: Half-heights, in pixels, at the ends of the response. The minimum is what silence +#: draws — a flat line of stubs rather than an empty box, which is the one thing Flow's +#: old meter and FluidVoice's agree on. +BAR_MIN_H, BAR_MAX_H = 1.5, 12.0 + #: Where the meter starts and how wide it ends up — named because the bar label has to #: begin after it, and two places computing `BARS * (BAR_W + BAR_GAP)` is how the label #: would come to be drawn through the twelfth bar the day one of them changed. @@ -524,6 +986,50 @@ def _unload_fonts() -> None: #: How long the lift back to full opacity takes once the pointer arrives. HOVER_LIFT_SEC = 0.4 +#: The ceiling on one push-to-talk hold, after which Flow stops capturing on its own. +#: +#: Not a limit on how long anybody may speak — it is a limit on how long a *missing +#: keystroke* may hold the microphone open. A keyup can genuinely fail to arrive: the OS +#: drops a low-level hook that overran `LowLevelHooksTimeout`, a lock screen or a UAC +#: prompt takes the input desktop mid-hold, an RDP session grabs the keyboard. Without +#: this the failure is a session recording a room until somebody notices. +#: +#: Two minutes because it has to sit clear of the longest hold anybody would make on +#: purpose without being so far out that the recording is a surprise. Whatever was said +#: is committed and kept, and the note says where it went — the one thing this must not +#: do is end a real dictation by discarding it. +PTT_MAX_HOLD_SEC = 120.0 + +#: How long the pill must be held before a press becomes a hold-to-talk rather than a +#: click, and how far the pointer may travel before it is a drag instead of either. +#: +#: **This is push-to-talk for everybody who has no chord**, which on a Mac is everybody: +#: `Chord` is a `WH_KEYBOARD_LL` hook and there is no such thing off Windows. The +#: gesture is the same one — press, speak, release, and the words are yours — but the +#: button is a window Flow already draws, so it costs no Accessibility permission, no +#: Input Monitoring, and no signed bundle to ask for them from. That is the one thing +#: Flow Lite can do that a native app driving a system hotkey cannot. +#: +#: 300 ms because three gestures now share one button and the other two are older: a +#: deliberate click is well under 200 ms, and a drag declares itself by moving. Slop is +#: 4 px rather than 0 because a hand resting on a mouse is not perfectly still, and a +#: hold that lost its nerve on one pixel of tremor would be a gesture nobody could rely +#: on. +PILL_HOLD_SEC = 0.30 +PILL_DRAG_SLOP = 4 + +#: How long the release waits for the decode it is going to paste. +#: +#: The paste cannot be synchronous. A final decode measured 0.7-7 s on the machine this +#: was built for, and the release has to return to the frame loop long before that. So +#: the wait is a state, and this is its ceiling. +#: +#: Fifteen seconds is past the worst final in that trace by a wide margin, and the +#: behaviour at the ceiling is not a discard: the words are in the draft, on screen, +#: and the note points at the Send chip. A paste that lands a minute late would arrive +#: in whatever window the user has since moved to, which is worse than not pasting. +PTT_PASTE_WAIT_SEC = 15.0 + #: Unified with `CARD_W` (decisions.md 2026-08-09, Phase 6): the widest state either #: panel reaches is the draft's full rescue row — Refine, Continue, Edit, "Was a #: command", Send, 345 px of chip width — and at the old 380 that left `chip_row_gap` @@ -575,6 +1081,54 @@ def _unload_fonts() -> None: #: on, which leaves the pill its own room underneath. BODY_MAX_H = 340 +#: The tallest a panel band may be, and no longer the height it always is. +#: +#: **This was a fixed height, and the reference says it should not be.** A demo of +#: FluidVoice, read frame by frame, settles it: the overlay's bottom edge is at y=554 in +#: every frame from idle through three lines of growth, and the box is *snug* around the +#: text in each one — two lines at 0:05, two at 0:08, three at 0:11. It never holds empty +#: space. Pinning the height bought stability at the price of a hole in the middle of the +#: window, which is the same complaint the resizing caused, wearing different clothes. +#: +#: What that overlay does instead is size to its content and *debounce* the resize — +#: `scheduleSizeAndPositionUpdate`, 80 ms, cancel-and-reschedule, with +#: `animationBehavior = .none`. Streaming partials coalesce into one step instead of +#: thirty resizes a second. Flow gets the same result without a timer, by snapping to +#: whole body lines (`_settled_h`): a height that can only change when the text gains or +#: loses a *line* changes a handful of times an utterance, and a timer that has to be +#: cancelled correctly from a render loop is a thing to get wrong. +#: The settings strip's own height, including the air under it. +#: +#: **It appears with the panel and never at rest.** The owner asked for the settings that +#: matter to be reachable without a right-click — "Dictate and Converse for sure Then +#: workspace and voices" — and the choice of *when* was left to me. With a draft up is +#: the answer: those three only mean anything once there is something to send, and an +#: always-on strip would cost 22 px of the idle row, which is the one part of this +#: surface everybody has said they like small. +#: +#: FluidVoice does not pay this either — its `Dictate / AI Prompt / Actions` bar belongs +#: to the app being dictated into, not to the overlay. +SETTINGS_H = 22 + +#: How far apart the read-only values sit from each other. +SETTINGS_GAP = 14 + +#: 184 was the ceiling before the settings strip existed, and the strip is furniture +#: rather than content — so it goes *on top of* the ceiling rather than out of the +#: content's share. Taking it out of the share is what the tests caught: the live +#: partial, whose own `PARTIAL_MAX_H` is a flat 70 px, ran through the note and the chip +#: row on a panel pegged at 184 because everything above it had grown by 22 and it had +#: not been told. +PANEL_MAX_H = 184 + SETTINGS_H + +#: The shortest a band gets, so a one-word draft still has a panel rather than a sliver. +PANEL_MIN_H = 96 + +#: What the body font measures per line — the number `BODY_MAX_H` is already built from +#: ("340 px is 20 lines at the 17 px the body font measures"). Named here because the +#: band now steps by it. +BODY_LINE_H = 17 + #: The live partial's own ceiling, and it needs one for the same reason the draft does: #: it is wrapped to the full body column, so it is a multi-line block whose length nobody #: chose. Five lines at the 14 px `FONT_NOTE` measures. @@ -603,23 +1157,155 @@ def _unload_fonts() -> None: PARTIAL_GAP = 6 #: Characters a line of body text holds, measured on the real canvas at the body font and -#: `BUBBLE_W - 2 * PAD` = 352 px: 3 160 characters of ordinary prose wrapped to 56 lines, -#: so 56.4. Two things read it, and neither may cost a layout — how much draft is worth -#: handing the canvas, and how many lines are above what it shows. -BODY_CHARS_PER_LINE = 56 +#: the shipped 392 px column (`BUBBLE_W - 2 * PAD`): 3 160 characters of ordinary prose +#: wrapped to 51 lines, so 62.0. Two things read it, and neither may cost a layout — how +#: much draft is worth handing the canvas, and how many lines are above what it shows. +#: +#: **Rebound by `apply_panel_width` at launch**, because the column is a setting now. +#: This is the value at the shipped width, kept here so the module still reads straight +#: through and so a launch that never calls that function is the launch Flow always had. +#: The old 56 was measured at a 352 px column — `380 - 2 * PAD`, the bubble before +#: Phase 6 — and stayed one panel width behind until the measurement was re-taken; see +#: `_CHARS_PER_PX`. +BODY_CHARS_PER_LINE = 62 #: The window of draft actually laid out per event, and the reason render cost stops #: growing: `BODY_MAX_H` holds 20 lines, this is enough characters for about 28 of them, so #: the visible tail is always full even where the text wraps early — and a two-hour #: dictation is laid out at the same cost as a two-minute one (invariant 7, extended to -#: rendering). Measured before and after on the real canvas: 2.4 / 32.7 / 476.7 ms at -#: 1k / 10k / 50k characters, and flat afterwards. -BODY_TAIL_CHARS = 1600 +#: rendering). Lines, not characters, are what is held constant here: the re-measured +#: column holds more per line, so the character count moved with it and the 28 did not. +#: Measured before and after on the real canvas at the 392 px column: 0.8 / 14.1 / +#: 221.3 ms at 1k / 10k / 50k characters, and flat at ~1.4 ms afterwards. (The older +#: 2.4 / 32.7 / 476.7 in this comment were the same shape on a slower machine.) +BODY_TAIL_CHARS = 1750 #: How far past the cut to look for a space before giving up and cutting mid-word. #: Bounded, because a scan that can run the length of the draft is the cost being avoided. BODY_BOUNDARY_SCAN = 200 +#: The panel widths on offer, and why the list starts where it does rather than lower. +#: +#: 420 is a **floor**, not a default somebody liked. Two measured rows put it there: +#: the bubble's five-chip row runs to 345 px (`chip_row_gap`, which records that 380 +#: clipped Send by half a label), and the card's runs to 377 of the same 420. A "small" +#: option would have to either drop a chip or ship a row that clips, so there is not +#: one — this is a setting for people who want the draft easier to read, and every +#: direction that helps with that is up. +#: +#: Named rather than free-form for the reason `KEYS` is: three widths that have each +#: been drawn are worth more than an integer nobody has rendered at. +PANEL_WIDTHS: dict[str, int] = {"regular": 420, "large": 520, "larger": 640} +PANEL_DEFAULT = "regular" + +#: Where the stack sits. `"bottom"` is bottom-centre of the monitor under the pointer, +#: which is FluidVoice's placement (`BottomOverlayView.positionWindow`) and now Flow's +#: default; `"corner"` is the bottom-right Flow shipped. `Pill._placed` carries the +#: argument for the change. Named rather than free-form for `KEYS`' reason — a position +#: somebody has actually looked at beats a pair of coordinates nobody has rendered. +#: What each chord gesture is called in the menu. Phrased as what it *does* rather +#: than as its name, because "hold" and "toggle" are the words in the profile and this +#: is the row somebody reads once while deciding — the whole sentence is the label. +#: What each gesture is called wherever a user reads it: the Settings menu, the note +#: after a switch, and the startup line. +#: +#: "Push to talk" by name, and the owner asked for it by name — "I think I like the +#: wording push to talk the default so it's more clear". The label used to describe the +#: mechanics only ("Hold to talk, release to send"), which is accurate and makes somebody +#: work out what it is. Push-to-talk is a thing people already know from a decade of +#: voice chat, so naming it does the explaining, and the mechanics still follow it for +#: anyone who has not met the term. +GESTURE_LABELS = { + "hold": "Push to talk - hold to speak, release to send", + "toggle": "Toggle - press to start, press again to stop", +} + +PLACES = ("bottom", "corner") +PLACE_DEFAULT = "bottom" +PLACE = PLACE_DEFAULT + +#: How far above the work area's bottom edge the stack stands, in pixels. +#: +#: FluidVoice exposes this as `overlayBottomOffset` and so does Flow, for the reason +#: they do: the bottom of the screen is where a taskbar auto-hides, where a browser +#: puts its download shelf, and where some apps park a status bar, so the one number +#: that makes the overlay sit clear of all that is worth a setting. 24 rather than +#: their 0 because Flow's stack has a pill under the panel and the pill is the part +#: that would touch the edge. +PANEL_BOTTOM_OFFSET = 24 + + +def apply_place(name: str) -> None: + """Set where the stack sits. Call before the first draw, beside `apply_panel_width`. + + A module global for that function's reason and with the same concession behind it: + the alternative is threading a placement through every window that positions itself, + and rebinding one name before anything is drawn is the smaller change whose failure + mode is visible immediately. Unknown names fall back rather than raising — this + arrives from a hand-edited profile, and a typo should cost the setting, not the app. + """ + global PLACE + PLACE = name if name in PLACES else PLACE_DEFAULT + +#: Body characters per line, per pixel of column. Kept as a *ratio* because the column +#: is no longer a fixed number: a wider panel holds proportionally more, and a +#: `BODY_CHARS_PER_LINE` frozen at one width would under-feed the canvas at 640 px and +#: quietly put the bottom of the draft below the fold. +#: +#: **Anchored at 420, which is now also where the measurement was taken.** The ratio +#: behind the old 56 came from a 352 px column — `380 - 2 * PAD`, the bubble *before* +#: Phase 6 took it to 420 — so it was one panel width behind, and the setting deliberately +#: reproduced it rather than smuggle a behaviour change into a size option. The +#: measurement has since been re-taken by the same method at the shipped 392 px column +#: (3 160 characters of prose to 51 lines, 62.0 a line), and this is that figure. +#: +#: Still written as `62 / 392` rather than the raw 0.1581 so the shipped width comes back +#: exactly, and so the number a reader can check against the canvas is the one in the +#: source. +_CHARS_PER_PX = 62 / (420 - 2 * PAD) + +#: Lines of draft handed to the canvas per layout, which is what `BODY_TAIL_CHARS` +#: actually encodes: `BODY_MAX_H` shows 20, and about 28 are laid out so the visible +#: tail is full even where the text wraps early. Held constant across widths, so the +#: render-cost invariant (7) survives a wider panel instead of being re-measured. +_TAIL_LINES = 1750 / 62 + + +def apply_panel_width(width: int) -> None: + """Set the panel width and everything measured off it. Call before the first draw. + + Module globals rather than instance state, and that is a deliberate concession + rather than a preference. `BUBBLE_W` and `CARD_W` are read from roughly twenty + places across two window classes and the free functions that draw their chrome, and + threading a width through all of them would be a large refactor whose only new + behaviour is this one setting. Rebinding three derived numbers once, before anything + is drawn, is the smaller change and the one whose failure mode is visible + immediately. + + Clamped at the floor rather than trusted. The number can come from a hand-edited + profile, and a panel narrower than its own chip row is a window whose Send button is + half off the edge — the one control that must never be unreachable. + """ + global BUBBLE_W, CARD_W, BODY_CHARS_PER_LINE, BODY_TAIL_CHARS + BUBBLE_W = CARD_W = max(int(width), PANEL_WIDTHS[PANEL_DEFAULT]) + BODY_CHARS_PER_LINE = max(1, int((BUBBLE_W - 2 * PAD) * _CHARS_PER_PX)) + BODY_TAIL_CHARS = int(BODY_CHARS_PER_LINE * _TAIL_LINES) + + +def panel_width(name) -> int: + """A width for a profile value, falling back to the shipped one. + + Anything unknown is the default rather than a refusal, which is the opposite of how + `hotkey.parse` treats a bad combo — and the difference is what the two settings cost + when wrong. A hotkey that silently fell back would leave somebody pressing keys that + do nothing with no way to find out; a panel that falls back is a window that is + visibly not the size they asked for, and the evidence is on the screen. + """ + if isinstance(name, str): + return PANEL_WIDTHS.get(name.strip().lower(), PANEL_WIDTHS[PANEL_DEFAULT]) + return PANEL_WIDTHS[PANEL_DEFAULT] + + #: The line saying what is not in the window, at the note's font plus its gap. One number #: for both of them — `… N earlier lines` above a draft and `… N more lines` below an #: answer — because they are the same line in the same font pointing opposite ways. @@ -893,6 +1579,78 @@ def _mix(a: str, b: str, t: float) -> str: ) +def _settings_row(c: tk.Canvas, pill, w: int, y: int, tags="body") -> int: + """The controls worth reaching without a right-click. Returns the height it took. + + Three things, and the split between them is deliberate. **Mode is a control**: it + changes what Send does, it is the thing somebody switches mid-task, and it costs a + right-click and two menu levels today. **Workspace and voice are values**: their + worth is being *visible* — knowing which project Ask is running in without opening + anything — and they open the menu that already exists rather than growing a second + implementation of it. + + Everything here is a chip or a label with a hit region, drawn the way `_lay_out` + draws the row at the foot, so there is one shape language on the surface and one way + a thing on it is clicked. + """ + session = getattr(pill, "session", None) + if session is None: + return 0 + converse = getattr(session, "mode", DICTATE) != DICTATE + label = "Converse" if converse else "Dictate" + mid = y + SETTINGS_H // 2 - 4 + + # The mode, as a chip that acts. `v` rather than a real chevron: the strip is drawn + # in the same ASCII-safe font the rest of this surface uses, and a glyph that falls + # back to a box would be a control that looks broken. + text = f"{label} v" + width = chip_w("mode", text) + _round_rect(c, PAD, y, PAD + width, y + SETTINGS_H - 4, 9, + fill=CHIP, outline="", tags=("settings-mode", tags)) + c.create_text(PAD + width / 2, mid, text=text, fill=CODE, font=FONT_CHIP, + tags=("settings-mode", tags)) + c.tag_bind("settings-mode", "", + lambda _e: getattr(session, "toggle_mode", lambda: None)()) + + # The values. Truncated from the left for the workspace, because the tail of a path + # is the part that names the project and the head is the part everybody shares. + x = PAD + width + SETTINGS_GAP + for name, value, opener in _settings_values(pill, session): + if not value: + continue + shown = f"{name} {value}" + tag = f"settings-{name.strip(':')}" + item = c.create_text(x, mid, anchor="w", text=shown, fill=MUTED, + font=FONT_NOTE, tags=(tag, tags)) + bounds = c.bbox(item) + if bounds is not None: + if bounds[2] > w - PAD: + # Out of room. Dropped rather than clipped: half a path is a worse + # answer than no path, and the menu still has it. + c.delete(item) + break + x = bounds[2] + SETTINGS_GAP + if opener is not None: + c.tag_bind(tag, "", lambda _e, f=opener: f()) + return SETTINGS_H + + +def _settings_values(pill, session): + """`(name, value, opener)` for each read-only value on the strip.""" + workspace = getattr(session, "workspace", "") or "" + if workspace: + workspace = os.path.basename(str(workspace).rstrip("/" + chr(92))) or str(workspace) + speaker = getattr(session, "speaker", None) + voice = "" + if speaker is not None: + voice = "muted" if getattr(session, "muted", False) else ( + getattr(speaker, "name", "") or "on") + return ( + ("workshop:", workspace, getattr(pill, "_menu_workspace", None)), + ("voice:", voice, getattr(pill, "_menu_voice", None)), + ) + + def _panel_chrome(c: tk.Canvas, w: int, h: int, radius, ring_color: str, tags="body", seam: str | None = None) -> None: """The opaque three-hairline elevation every v2 surface shares (decisions.md @@ -929,7 +1687,12 @@ def _panel_chrome(c: tk.Canvas, w: int, h: int, radius, ring_color: str, # join is the one mark that makes two surfaces unmistakably two. c.create_line(4 + inner[0], 4, w - 4 - inner[1], 4, fill=RING_TOP, tags=tags) if seam == "top": - c.create_rectangle(0, 0, w, 4, fill=SHELL, outline="", tags=tags) + # Through 5, not 4. The inner ring is drawn *at* y=4, so a fill that stopped + # there left it standing — and a second hairline 3 px under the divider is + # exactly the "two surfaces that happen to touch" this is here to prevent. It + # went unseen while these were two windows, because a 1 px window gap was + # already drawing a darker line in the same place. + c.create_rectangle(0, 0, w, 5, fill=SHELL, outline="", tags=tags) elif seam == "bottom": c.create_rectangle(0, h - 4, w, h, fill=SHELL, outline="", tags=tags) c.create_line(0, h - 1, w, h - 1, fill=RING, tags=tags) @@ -968,7 +1731,15 @@ class Pill(tk.Tk): #: ran `__init__` — and so never built a `bubble`/`card` to dock to — draws exactly #: the idle pill this default describes, rather than recursing through `front`. _docked_w = PILL_W - _docked_above = True + _shell_h = PILL_H + #: Whether the window is parked with an icon standing in for it. Class-level for the + #: reason `lite` is: a fixture built with `__new__` must not recurse into `self.tk`. + _hidden = False + _tray = None + #: Where the window was when it was hidden, as (x, foot). Restored on the way back, + #: because somebody who dragged Flow to the left of their screen did not ask for it + #: to reappear in the middle. + _home = None #: Same reason again, for `_draw`'s motion state (§07): a bare fixture draws the #: resting frame these describe — not hovered, not mid-collapse, opacity untouched. _pointer_in = False @@ -981,6 +1752,25 @@ class Pill(tk.Tk): _dots_frame = 0 _tint = 0.0 _flash = 0 + #: Same reason a fourth time, for push-to-talk's two clocks. These are read by + #: `_toggle`, `_clear` and the mode switch — three paths a fixture drives directly + #: without ever having held the chord — so the default has to be "no hold, nothing + #: waiting" rather than a recursion. `None` is that in both cases. + _ptt_since: float | None = None + _ptt_wait: float | None = None + #: And for the three gestures now sharing the left button: when it went down, where, + #: whether it has travelled since, whether the press turned into an utterance, and + #: the `after` id that would turn it into one. All idle here, which is the state a + #: fixture that never touched a mouse should read as. + _press_at: float | None = None + _press_xy = (0, 0) + _press_moved = False + _press_talking = False + _press_timer = None + #: And once more for the settings menu's newest row, which asks the live chord what + #: gesture it is. `--no-hotkeys` leaves this None for real, so the default is not a + #: fixture convenience — it is the shipped value on one of the supported launches. + hotkeys = None def __init__( self, session: Session, on_send=None, hotkeys=None, arm=False, @@ -1005,7 +1795,12 @@ def __init__( Path(settings_path) if settings_path is not None else LEXICON_PATH ) self._arm_on_start = arm - self.levels: deque[float] = deque([0.0] * BARS, maxlen=BARS) + #: The level every bar is drawn from this frame, 0…1. One number, where this + #: used to be a `deque` of `BARS` of them: the meter blooms from its own centre + #: now rather than scrolling a history past, so there is no past to keep. See + #: `_bar_half_height`. The eased value still lives in `_eased_level`; this is + #: what the last frame actually drew, which `_flatten` needs to fade from. + self._meter_level = 0.0 #: The level actually drawn, eased toward `session.level_db` rather than #: jumping to it — rise 60 ms, fall 160 ms (§07), so peaks fall slower than #: they rise. Only the newest sample eases; everything already in `levels` is @@ -1024,6 +1819,16 @@ def __init__( self._hover_since: float | None = None self.armed = False self._disarmed_since = time.perf_counter() # starts disarmed, so the clock does too + #: When the current push-to-talk hold began, or None when no chord is held. The + #: clock exists for `PTT_MAX_HOLD_SEC`: a release can be missed — a hook the OS + #: drops for taking too long, a lock screen, an RDP session taking the keyboard — + #: and a hold whose end never arrives is a microphone left open indefinitely. + self._ptt_since: float | None = None + #: When the release happened and Flow started waiting for the decode to land, or + #: None when nothing is waiting. The paste cannot be synchronous: a final decode + #: measured 0.7-7 s in this user's own trace, so the release arms a wait and the + #: frame loop finishes the gesture. + self._ptt_wait: float | None = None self._flash = 0 # frames remaining of the error flash, out of `FLASH_FRAMES` #: Where the three waiting dots are in their 1.2 s loop, and how far the pill has #: travelled toward converse's violet (0 = dictate, 1 = converse). Both advance @@ -1046,26 +1851,33 @@ def __init__( #: `attributes("-alpha", …)` when the target has actually changed. self._drawn_alpha = 0.94 - self.work = _work_area(self.winfo_screenwidth(), self.winfo_screenheight()) - left, top, right, bottom = self.work - self.x = right - PILL_W - 28 - self.y = bottom - PILL_H - 24 + #: The monitor the stack is placed against, as the two rectangles FluidVoice + #: places against — `full` for centring, `work` for standing on. Refreshed from + #: the pointer's monitor in `_sync_monitor`; see `_pointer_monitor`. + self.full, self.work = _pointer_monitor( + self.winfo_screenwidth(), self.winfo_screenheight(), self) + self.x, self.y = self._placed(PILL_W) self.geometry(f"{PILL_W}x{PILL_H}+{self.x}+{self.y}") #: The width last drawn, so `_sync_dock` can tell whether a panel appeared or #: went away since the last frame — and, holding the right edge fixed, by how #: much the left edge has to move to match. Neither panel exists yet at this #: point in `__init__`, so this starts at the same idle width just drawn above. - self._docked_w = PILL_W - #: Which side of the pill a docked panel is actually on, so a squared corner - #: lands on the shared seam rather than on the free-standing side. Set by - #: `Bubble.reposition`/`ConversationCard.reposition`, the one place that - #: already decides above-vs-below; read back by `_draw`. - self._docked_above = True + self._docked_w = self.pill_w + #: How tall the one window is right now — the pill row, plus a panel band when a + #: panel is up. Compared in `_sync_shell` so a frame that changes nothing costs + #: no `geometry` call. + self._shell_h = PILL_H + #: What the notification-area icon puts its clicks on, drained in `_frame`. + #: Built here rather than with the icon, so `_drain_tray` has something to read + #: on every frame whether or not anybody has ever hidden the window. + self._tray_events: queue.Queue = queue.Queue() self.canvas = tk.Canvas( - self, width=PILL_W, height=PILL_H, bg=self.bg, highlightthickness=0 + self, width=BUBBLE_W, height=PILL_H, bg=self.bg, highlightthickness=0 ) - self.canvas.pack() + # `place`, not `pack`: this canvas is the *foot* of a window whose top edge moves + # when a panel opens, so it has to be positioned rather than filled. + self.canvas.place(x=0, y=0, width=BUBBLE_W, height=PILL_H) self.bubble = Bubble(self) #: P9's own surface (decisions.md 2026-08-03, "two surfaces, two jobs"). Built @@ -1090,7 +1902,13 @@ def __init__( # list and threw away the press handler that records where in the pill it was # grabbed. The pill dragged — it just snapped its top-left corner to the cursor # first, every time. - self.canvas.bind("", self._toggle, add="+") + # Press, motion and release rather than one `` handler, because the + # button now carries three gestures — see `_on_press`. `add="+"` for the reason + # the comment above gives, which has not changed: the drag handler is bound to + # the same event and binding without it would replace the whole list. + self.canvas.bind("", self._on_press, add="+") + self.canvas.bind("", self._on_release, add="+") + self.canvas.bind("", self._on_motion, add="+") self.canvas.bind("", self._menu) # Nothing animates under the hand (§07) — the same rule `Bubble`/ # `ConversationCard` already keep, extended to the pill's own motion. @@ -1134,7 +1952,82 @@ def drag(e): self.canvas.bind("", drag) self.canvas.bind("", press, add="+") + # -- one button, three gestures ---------------------------------------- + + def _on_press(self, e) -> None: + """Start the clock. Which gesture this is cannot be known yet. + + `_toggle` used to be bound straight to ``, which in Tk is the *press* — + so arming happened before the button came back up, and **every drag of the pill + also toggled listening**. That was there before hold-to-talk and is fixed by the + same change: a click is judged on release, like a button anywhere else. + """ + self._press_at = time.perf_counter() + self._press_xy = (e.x_root, e.y_root) + self._press_moved = False + self._press_talking = False + self._press_timer = self.after(int(PILL_HOLD_SEC * 1000), self._press_held) + + def _press_held(self) -> None: + """`PILL_HOLD_SEC` has passed with the button down and the pointer still. + + Fired by a timer rather than checked on release, and that is the whole + difference between this and a long-click: capture has to start *while the user + is still holding*, because the hold is the utterance. Waiting for the release to + notice would record nothing at all. + """ + self._press_timer = None + if self._press_at is None or self._press_moved: + return + self._press_talking = True + self._talk_start() + + def _on_release(self, _e=None) -> None: + """Decide what the press was, now that it is over. + + Three outcomes and one rule each: a hold ends the utterance and sends it, a + still click toggles, and a drag has already done its own job and must not do a + second one on the way out. + """ + if self._press_timer is not None: + self.after_cancel(self._press_timer) + self._press_timer = None + talking, moved = self._press_talking, self._press_moved + self._press_at = None + self._press_talking = False + if talking: + self._talk_end(send=True) + elif not moved: + self._toggle() + + def _on_motion(self, e) -> None: + """Past the slop, this press is a drag — unless it is already an utterance. + + The order matters. Once capture is open the pointer is irrelevant: somebody + talking into a held pill may well move the mouse, and cancelling their sentence + for it would be the gesture betraying them. Before that, motion is the thing + that tells a drag from a hold. + """ + if self._press_at is None or self._press_talking: + return + x, y = self._press_xy + if abs(e.x_root - x) > PILL_DRAG_SLOP or abs(e.y_root - y) > PILL_DRAG_SLOP: + self._press_moved = True + def _toggle(self, _e=None) -> None: + if self._ptt_since is not None: + # The toggle hotkey pressed with the chord still held. `pause()` below would + # bump the capture generation, which is how a deliberate stop refuses a + # decode from before it — and the utterance being spoken *right now* is + # exactly what that would refuse. Ending the hold first commits it. + # + # Not sent, because a toggle is not a release: the user reached for the + # other control mid-sentence, and pasting on their behalf is not what + # either gesture asked for. The words land in the draft, where Send takes + # them. The `return` is the rest of it — the hold has already stopped + # capture, so falling through to `pause()` would do the damage anyway. + self._talk_end(send=False) + return if self.armed: self.armed = False self._disarmed_since = time.perf_counter() # starts the 8 s idle dim @@ -1153,6 +2046,124 @@ def _toggle(self, _e=None) -> None: self._disarmed_since = None # an actively-capturing pill never dims self._draw() + # -- push to talk ------------------------------------------------------ + + def _talk_start(self) -> None: + """The chord's press-down: open the microphone for as long as it is held. + + Re-entrant on purpose. A hold that is already running must not restart capture — + the OS repeats a held key in some configurations, and a second `talk` arriving + mid-utterance would be indistinguishable from the user having spoken into a + microphone that had just been reopened under them. + """ + # Hidden Flow still hears the chord — the hook is global and does not care what + # is on screen — and a hold that showed nothing would be an open microphone with + # no way to tell it was open, which is what invariant 4 forbids. So the window + # comes back for the utterance, before anything else here decides not to run. + if self._hidden: + self.show_from_tray() + if self._ptt_since is not None: + return + if getattr(self.session, "editing", False): + # The hand editor is open, and `_pump_audio` throws away every block while it + # is. A hold here would open the microphone, capture nothing, and end with no + # paste and no explanation — the silent deafness invariant 4 forbids. Said + # instead, on the surface the editor is on. + self.front.note("editing — close the editor to dictate") + return + # A hold that begins while a previous release is still waiting for its decode + # supersedes it. Not dropped — `_ptt_wait` is only the *paste*, and abandoning it + # leaves the earlier words in the draft where the next Send will take them. + self._ptt_wait = None + try: + self.session.talk_start() + except Exception as exc: + # No microphone, a device held exclusively elsewhere, a session already + # closing. Same refusal `_toggle` makes, for the same reason: a green pill + # over a dead capture is the one lie this surface must not tell. + self._flash = FLASH_FRAMES + self.bubble.surface(f"could not start capture: {exc}") + self._draw() + return + self._ptt_since = time.perf_counter() + self.armed = True + self._disarmed_since = None + self._draw() + + def _talk_end(self, *, send: bool) -> None: + """The release: stop capturing, and hand what was said to `send` if it was clean. + + `send=False` is the `ctrl+win+d` path. The words are still committed and still + land in the draft — nothing spoken is ever dropped for the user's convenience + (P2) — they simply do not paste themselves into whatever window a desktop switch + just moved to. In the overwhelmingly common case there are no words at all: the + gate never opened in the 50 ms before the third key, so the draft stays empty and + the whole thing is invisible. + + Idempotent against a release with no hold behind it. The OS can deliver a keyup + whose keydown this process never saw — a chord begun before Flow launched, or + while a UAC prompt owned the input desktop — and that must not stop a capture the + toggle hotkey started. + """ + if self._ptt_since is None: + return + self._ptt_since = None + pending = self.session.talk_end() + # `talk_end` emits `disarm` when the hold was what opened the microphone, and the + # event handler clears `armed`. It deliberately does not when the hold began + # against an already-capturing session, and this must not either: the chord gives + # back exactly what it took. + if not pending: + # Nothing was said into the hold. A tap, or a shortcut. Say nothing, do + # nothing — a note here would fire on every `ctrl+win+arrow` on the machine. + self._draw() + return + self._ptt_wait = time.perf_counter() if send else None + self._draw() + + def _pump_talk(self) -> None: + """Finish the two halves of the gesture that cannot finish on a keystroke. + + Called every frame, and both branches are timeouts. The first is the hold whose + release never came; the second is the paste whose decode never landed. Neither is + hypothetical — the second is the exact state the app was in when it wedged with a + `state -> idle` and no `final` behind it, and a wait with no ceiling is how that + turned into a microphone open on a session nobody could reach. + """ + now = time.perf_counter() + + if self._ptt_since is not None and now - self._ptt_since >= PTT_MAX_HOLD_SEC: + # Treated as a release and not as a break: the user was dictating, and the + # thing that went missing is a keystroke, not their intent. What they said is + # committed and the draft holds it; it is not pasted, because a paste this + # far from the gesture would land somewhere they are no longer looking. + self._talk_end(send=False) + self.front.note( + f"stopped after {PTT_MAX_HOLD_SEC / 60:.0f} min — the chord was still " + "held. What you said is here; press Send when you want it") + return + + if self._ptt_wait is None: + return + + # The decode has to be *finished*, not merely started. `busy` covers the worker + # queue and the partial in flight, so this waits out a final that is still being + # transcribed rather than pasting the partial that preceded it. + if not self.session.busy: + text = self.session.draft.text + self._ptt_wait = None + if text: + self._send() + return + + if now - self._ptt_wait >= PTT_PASTE_WAIT_SEC: + self._ptt_wait = None + # Never a silent give-up, and never a discard: the words are on screen and + # the chip that sends them is the one the user already knows. + self.front.note( + f"still decoding after {PTT_PASTE_WAIT_SEC:.0f}s — not pasting on my " + "own this late. Press Send when it lands") + def _enter(self, _e=None) -> None: self._pointer_in = True self._hover_since = time.perf_counter() @@ -1300,6 +2311,155 @@ def _draft_menu(self, parent: tk.Menu) -> None: sub.add_command(label="Clear", command=self._clear) parent.add_cascade(label="Draft", menu=sub) + def _panel_menu(self, parent: tk.Menu) -> None: + """How wide the draft panel draws, chosen from the widths that have been drawn. + + **Applied immediately rather than at next launch.** Every part of both windows + reads `BUBBLE_W` and `CARD_W` while drawing a frame rather than caching them at + construction, so rebinding them and forcing a redraw is a complete change — and + a size you cannot see until you restart is one nobody can choose between. + + Saved on the way through, because this is the kind of setting somebody sets once + and would be annoyed to set again. A save that fails is said out loud rather than + swallowed: the width is still applied, so the session honours the choice and the + note explains why the next one will not. + """ + sub = _dark_menu(parent) + here = BUBBLE_W + + def choose(name: str) -> None: + apply_panel_width(panel_width(name)) + # Both windows, because they are one window at two moments and only one of + # them is on screen to notice the change. `_frame` re-reads the width and + # re-lays every item it draws, so re-anchoring is the whole of the update. + for window in (self.bubble, self.card): + window.reposition() + profile = getattr(self.session, "profile", None) + if profile is None: + # `--no-profile`. The size is applied and lasts the session, which is + # exactly what that flag asks for, so there is nothing to report. + return + profile.panel = name + # The same shape `_set_trigger` uses, and for the same reason: this is a + # setting somebody chooses once, so a save that failed has to be visible now + # rather than discovered at the next launch. + if profile.save(): + self.bubble.note(f"panel size: {name}") + else: + self._flash = FLASH_FRAMES + self.bubble.note(f"could not save {profile.path}") + + for name, width in PANEL_WIDTHS.items(): + sub.add_command( + label=name.capitalize() + (" (current)" if width == here else ""), + command=lambda n=name: choose(n), + ) + parent.add_cascade(label="Panel size", menu=sub) + + def _gesture_menu(self, parent: tk.Menu) -> None: + """What the chord does, switchable while Flow is running. + + This is here because shipping push-to-talk *instead of* the toggle took a + working gesture away from everybody who had it, with no way back short of + editing a file — and the two are not a preference between equals, they are good + at different things. A hold needs no decision about when you are finished and + cannot leave a microphone running; a toggle is the only one of the two that + survives a paragraph, a long thought with pauses in it, or hands that cannot + hold two keys down for a minute. + + **Applied to the live hook rather than at next launch**, and that is the whole + reason `Chord.gesture` is a plain attribute the callback reads. Switching by + rebuilding the chord would mean unhooking and re-installing a `WH_KEYBOARD_LL` + hook, which is the one call in that file the OS is entitled to refuse — and + being refused *while changing a setting* would leave somebody with no chord at + all and no obvious way back. One string assignment cannot fail. + + Absent when there is no chord to describe: `--no-chord`, `"chord": ""`, or a + hook the OS refused. Same rule the help sheet follows — the menu says what works + on this machine this launch. + """ + chord = getattr(self.hotkeys, "chord", None) if self.hotkeys else None + if chord is None: + return + sub = _dark_menu(parent) + + def choose(name: str) -> None: + chord.gesture = name + # A gesture change mid-hold would leave the release with nothing to end: + # `_talking` is latched inside the hook and the pill is holding a `_ptt_since` + # for a capture the new gesture has no word for. Ending it here is the same + # tidy-up `_toggle` does, and for the same reason — the words are kept. + if self._ptt_since is not None: + self._talk_end(send=False) + profile = getattr(self.session, "profile", None) + if profile is None: + return # `--no-profile`: applied for this session, which is what it asks + profile.gesture = name + if profile.save(): + self.front.note(f"chord: {GESTURE_LABELS[name]}") + else: + self._flash = FLASH_FRAMES + self.front.note(f"could not save {profile.path}") + + for name in GESTURE_LABELS: + sub.add_command( + label=(GESTURE_LABELS[name] + + (" (current)" if name == chord.gesture else "")), + command=lambda n=name: choose(n), + ) + parent.add_cascade(label=f"Chord ({chord.describe()})", menu=sub) + + def _effort_menu(self, parent: tk.Menu) -> None: + """How hard the agent CLI may think, where it offers the choice. + + Lowest by default, and that is a judgement rather than a saving: these calls are + a *rewrite* — take what was dictated and make it read like a written prompt — and + effort buys deliberation the task has no use for, paid for in the one currency + that counts here, which is the user watching a spinner between finishing a + sentence and having their words. + + Offered anyway, per level, because "make it think harder about my prompt" is a + reasonable thing to want from a model you know. + """ + current = getattr(self.session, "cli_effort", EFFORT_DEFAULT) + sub = _dark_menu(parent) + for level in EFFORTS: + sub.add_command( + label=level + (" (current)" if level == current else ""), + command=lambda v=level: self.session.set_cli_effort(v), + ) + parent.add_cascade(label=f"Effort ({current})", menu=sub) + + def _model_menu(self, parent: tk.Menu) -> None: + """Which model to ask the CLI for, from the names that have been used before. + + **The menu is a list of what somebody has already typed, and cannot be anything + else.** No CLI will enumerate its models — `codex exec --help` says `-m, --model + ` and stops — so the names cannot be discovered, and Flow has no text + field anywhere to type one into. Settings is a menu, not a dialog, and the + docstring above refuses a page for exactly this reason. + + So `--cli-model` is how a name arrives, once, and it is remembered; from then on + it is a click. The menu hides itself entirely until there is a second thing to + choose between, the same rule the CLI picker follows. + """ + known = tuple(getattr(getattr(self.session, "profile", None), "cli_models", ())) + current = getattr(self.session, "cli_model", "") + if not known: + return + sub = _dark_menu(parent) + + def choice(label: str, value: str) -> None: + sub.add_command( + label=label + (" (current)" if value == current else ""), + command=lambda v=value: self.session.set_cli_model(v), + ) + + choice("The CLI's own default", "") + for name in known: + choice(name, name) + parent.add_cascade(label=f"Model ({current or 'default'})", menu=sub) + def _settings_menu(self, parent: tk.Menu) -> None: """Everything somebody sets once, in one place they can find it twice. @@ -1309,7 +2469,9 @@ def _settings_menu(self, parent: tk.Menu) -> None: invites options to be added to it. """ sub = _dark_menu(parent) + self._gesture_menu(sub) self._trigger_menu(sub) + self._panel_menu(sub) # Also the CLI marker's refresh point: a CLI installed mid-session shows up here, # where a press is already paying for the PATH walk `_resolved` will not repeat. clis = self._clis = available() @@ -1335,6 +2497,13 @@ def choice(label: str, cli) -> None: for candidate in clis: choice(candidate.name, candidate) sub.add_cascade(label="Agent CLI", menu=picker) + self._effort_menu(sub) + self._model_menu(sub) + if tray.available(): + # Offered where the other once-and-forget settings are, and only where there + # is a notification area to hide into. `hide_to_tray` refuses rather than + # hides if the icon does not take, so this cannot strand anybody. + sub.add_command(label="Hide to tray", command=self.hide_to_tray) self._workspace_menu(sub) if getattr(self.session, "speaker", None) is not None: sub.add_command( @@ -1423,6 +2592,32 @@ def _set_trigger(self, word: str) -> None: #: path anybody's `--cwd` resolves to. WORKSPACE_NOT_SET = "(not set)" + def _popup_menu(self, build) -> None: + """Post one of the settings submenus on its own, under the pointer. + + The strip's values open the menu that already exists rather than growing a second + implementation of the same list — the workspace recents and the voice list are + both built with a tick showing the current choice, and two of anything is two + things to keep in step. + + `build` is one of the `_*_menu` methods, which add a cascade to a parent. A + throwaway parent is that cascade's home for the moment it is on screen. + """ + parent = _dark_menu(self) + build(parent) + if parent.index("end") is None: + return # nothing to choose between — see each builder's early return + try: + parent.tk_popup(self.winfo_pointerx(), self.winfo_pointery()) + finally: + parent.grab_release() + + def _menu_workspace(self) -> None: + self._popup_menu(self._workspace_menu) + + def _menu_voice(self) -> None: + self._popup_menu(self._voice_menu) + def _workspace_menu(self, parent: tk.Menu) -> None: """Where questions are asked from, as a list of places already chosen. @@ -1776,12 +2971,25 @@ def _send(self, submit: bool = False) -> None: boom" yields "boom", so a refusing enter-variant would make the degraded decode the working case and the fuller utterance the broken one — the exact inversion the word order exists to prevent. + + **Every send goes through here, which is what makes one line enough to stop + push-to-talk sending twice.** There are four ways to send — this chip, the + `send` hotkey, the spoken trigger routed as a `send` event, and converse's + auto-ask countdown — and a release that has armed a paste is a fifth thing + waiting to do the same job. The collision is not hypothetical: hold the chord, + say "…and that's the plan, boom", let go, and the trigger fires a send when the + decode routes while the release is still waiting to fire its own. + + A hold owns *one* send, and whoever gets there first has it. Cancelling the wait + here covers all four collisions at the one point they have in common, rather + than four guards that would have to be kept in step — and it is the right way + round, because a send that has already happened is the one thing that proves the + wait has nothing left to do. """ + self._ptt_wait = None text = self.session.send() problem = "" - if text and self.lite: - problem = self._copy(text) - elif text and self.on_send: + if text and self.on_send: # The window is chosen here, on the UI thread, from what was polled before # the click — not inside `paste()` after it. The handler reports back what # went wrong rather than printing it somewhere nobody is looking. @@ -1790,6 +2998,14 @@ def _send(self, submit: bool = False) -> None: # handler predating this — `send_check.py`'s fixture is one — still works. extra = {"submit": True} if submit else {} problem = self.on_send(text, self.paste_target, **extra) or "" + elif text and self.lite: + # The fallback, not the Lite behaviour. A handler is offered wherever Flow + # can actually put the words in the other window — Win32 injection, or + # System Events on a Mac — and the copy is what is left when it cannot. + # These two used to be the other way round, so a Mac that had grown a real + # paste path would still have copied: `lite` is about hotkeys and window + # handles, and it was standing in for "cannot send", which it is not. + problem = self._copy(text) if getattr(self.session, "mode", DICTATE) != DICTATE: # Converse: send() returns "" and the answer is still coming, so the bubble # stays up to render it and there is nothing to linger over. @@ -1802,7 +3018,7 @@ def _send(self, submit: bool = False) -> None: self.bubble.show_sent(text, problem) elif text: self.bubble.show_sent(text) - if self.lite: + if self.lite and self.on_send is None: # After the card, not instead of it: the words are the important half and # the note is what tells somebody the last step is theirs. self.bubble.note(COPIED_ENTER if submit else COPIED) @@ -1993,10 +3209,80 @@ def _clear(self) -> None: # Clear is the cheapest "stop" the user has, and with the microphone gated while # Flow talks it is one of the few ways left to cut a reply short. Doing that # first means one press does the obvious thing whichever is in progress. + # + # A pending push-to-talk paste is exactly such a thing in progress, and the + # nastiest one to leave running: the draft is cleared here, the decode lands a + # second later and refills it, and the wait pastes into the user's window the + # words they just pressed a key to stop. Clear means clear. + self._ptt_wait = None self.session.stop_speaking() self.session.draft.clear() self.bubble.hide() + def hide_to_tray(self) -> bool: + """Put Flow out of the way, with an icon to bring it back. + + The need, in the owner's words: "there are times where i wanted to dictate but at + the same time i wanted to see but i don't want it to keep it on my screen". The + chord still works while hidden — it is a global hook and does not care what is on + screen — so dictating is unchanged and only the window goes. + + **The icon comes first, and hiding is conditional on it.** A window parked off the + desktop with nothing in the notification area is a Flow that cannot be reached, + configured or quit except through Task Manager. So `Tray.start()` is asked first + and its answer is believed: no icon, no hiding, and a note saying so. Invariant 4 + in a new place — hidden must not mean gone. + """ + if not tray.available(): + self.front.note("hiding needs the Windows notification area") + return False + if self._tray is None: + self._tray = tray.Tray("Flow - press the chord to talk", self._tray_events) + if not self._tray.start(): + self._flash = FLASH_FRAMES + self.front.note("the notification area would not take an icon") + return False + # Where it was, as (x, foot) — the foot rather than the top, because that is + # the edge the shell is anchored by and the one a reopened panel measures from. + self._home = (self.x, self.y + self._shell_h) + self._hidden = True + park(self) + return True + + def show_from_tray(self) -> None: + """Bring the window back where the user left it. + + The icon stays. Somebody who hid Flow once will hide it again, and an icon that + vanished on the first click would make the second one a trip through the menus. + """ + if not self._hidden: + return + self._hidden = False + if self._home is not None: + x, foot = self._home + self.x, self.y = x, foot - self._shell_h + self._sync_shell() + self.deiconify() + self.lift() + + def _drain_tray(self) -> None: + """What the icon decided, acted on from Tk's own thread. + + `tray.Tray` runs its window procedure on a thread of its own and puts strings on + a queue rather than calling back, precisely so this is the only place Tk is + touched — see that module's docstring for why that rule is the whole of the + threading argument here. + """ + while True: + try: + event = self._tray_events.get_nowait() + except queue.Empty: + return + if event == tray.SHOW: + self.show_from_tray() + elif event == tray.QUIT: + self.quit_app() + def quit_app(self) -> None: # Idempotent, because ctrl+C reaches here down either of two paths and nothing # upstream can tell which one ran: caught in `_tick`, or escaping `mainloop` and @@ -2009,6 +3295,11 @@ def quit_app(self) -> None: # re-arm itself against a destroyed interpreter on its way out. self._alive = False try: + # Before the hotkeys and before the window: an icon outliving its process is + # a ghost in the notification area that only a hover clears, and the shell + # gives no second chance to remove one whose window has already gone. + if self._tray is not None: + self._tray.stop() if self.hotkeys is not None: self.hotkeys.stop() self.session.close() @@ -2063,6 +3354,81 @@ def _tick(self) -> None: if self._alive: self.after(30, self._tick) + @property + def width(self) -> int: + """This window's width, under the name `park` and the panels both use. + + The pill had no `width` while the panels did, so `park(self)` — the call that + hides this window — reached `tk.Misc.__getattr__` and went looking for a Tcl + command. "Hide to tray" did nothing at all, twice over: this, and a `_sync_shell` + that put the window straight back. + """ + return self.pill_w + + def band_h(self) -> int: + """How tall the panel band is: `PANEL_H`, unless the desktop is smaller. + + The row's own height comes off the top of what is available, which the panels' + `panel_h` did not do while they were windows of their own — they only had to fit + the work area, and the pill fitted it separately. Sharing one window makes them + one sum, and a 200 px-tall display was enough to put a 224 px shell 32 px past + the bottom of it. + """ + _left, top, _right, bottom = self.work + room = (bottom - top) - 2 * EDGE_AIR - PILL_H + return max(0, min(PANEL_MAX_H, room)) + + def _placed(self, w: int) -> tuple[int, int]: + """Where a stack `w` wide belongs on the current monitor, per `PLACE`. + + Two answers, and the setting picks between them rather than one being a + degraded version of the other: + + `"bottom"` is FluidVoice's (`positionWindow`) — centred on the physical display, + stood on the work area. It is the default because the corner has a problem the + centre does not: the bottom-right of the screen is where Windows puts the tray, + every toast notification, and most apps' own status chrome, so the one place + Flow reserved for itself is the busiest real estate on the desktop. The centre + is empty, it is where the eye already is, and it is the same place on every + machine regardless of what is docked to which edge. + + `"corner"` is what Flow shipped, kept because somebody who has spent months + with the pill in the bottom right should not have it moved by an upgrade. + """ + if PLACE == "corner": + _left, _top, right, bottom = self.work + return right - w - 28, bottom - PILL_H - 24 + return bottom_centre(w, PILL_H, self.full, self.work, PANEL_BOTTOM_OFFSET) + + def _sync_monitor(self) -> None: + """Follow the pointer's monitor, and re-place the stack when it changes. + + Flow read the work area **once, in `__init__`, from `SystemParametersInfoW`** — + which only ever answers for the primary display. So on a two-monitor desk every + window Flow drew was placed against a screen the user might not be looking at, + and the clamps that keep panels on-screen were clamping to the wrong rectangle. + That is the placement problem, and it was never a rounding error: it is the + whole width of a monitor. + + Checked every frame and acted on only when the rectangle actually moves, which + is the same shape `_track_target` uses for `classify` and for the same reason — + the question is cheap, the answer changes a few times an hour, and doing the + work unconditionally would be a `geometry` call per frame forever. + """ + full, work = _pointer_monitor( + self.winfo_screenwidth(), self.winfo_screenheight(), self) + if (full, work) == (self.full, self.work): + return + self.full, self.work = full, work + self.x, self.y = self._placed(self.pill_w) + # The panels are placed *from* the pill, so moving it is the whole move — but + # only for a panel that is up. `reposition` on a withdrawn window would place it + # and leave it withdrawn, which is work nobody can see. + self._sync_dock() + for panel in (self.bubble, self.card): + if getattr(panel, "_visible", False): + panel.reposition() + def _track_target(self) -> None: """Remember the last window that had the foreground and was not Flow's own. @@ -2078,6 +3444,12 @@ def _track_target(self) -> None: return hwnd = foreground_hwnd() if hwnd and not owned_by_flow(hwnd): + if hwnd != self.paste_target: + # Only when the window actually changed. `classify` opens a process + # handle, and this runs every frame — at 30 fps an `OpenProcess` per + # frame is a cost paid forever to answer a question whose answer moves a + # few times an hour. Resolved on the edge, remembered in between. + self.session.target_app = classify(hwnd).process self.paste_target = hwnd @property @@ -2113,17 +3485,40 @@ def _swap_surfaces(self) -> None: def _frame(self) -> None: self._track_target() + self._sync_monitor() + + # Same rule as the hotkeys below: another thread decided, this one acts. + self._drain_tray() # Hotkeys arrive on their own thread; Tk is only ever touched from this one. if self.hotkeys is not None: for name in self.hotkeys.drain(): if name == "toggle": self._toggle() + elif name == "warm": + # The chord's press-down, one put ahead of `talk`, so the models load + # during the hold instead of inside the first sentence. + self.session.warm() + elif name == "talk": + self._talk_start() + elif name == "talk-end": + self._talk_end(send=True) + elif name == "talk-break": + # Windows meant `ctrl+win+d`. Stop, keep whatever was said, paste + # nothing — see `_talk_end`. + self._talk_end(send=False) elif name == "send": self._send() elif name == "cancel": self._clear() elif name == "mode": + # A pending paste belongs to the mode it was spoken in. Dictate + # pastes into a window and converse asks a CLI, so a wait armed in + # one and fired in the other does something the user never asked + # for — and the switch is one keypress away at all times. Dropped + # rather than translated: the words stay in the draft, and Send in + # the new mode does whatever it now means, deliberately. + self._ptt_wait = None self.session.toggle_mode() elif name == "quit": self.quit_app() @@ -2133,7 +3528,7 @@ def _frame(self) -> None: self.session.tick() if getattr(self.session, "hearing", True): self._deaf_frame = 0 - self.levels.append(self._eased(self._norm(self.session.level_db))) + self._meter_level = self._eased(self._norm(self.session.level_db)) else: self._flatten() else: @@ -2142,10 +3537,15 @@ def _frame(self) -> None: # arrived, because the code that collects a reply sat behind this check. self.session.pump_results() self._deaf_frame = 0 - self.levels.append(self._eased(0.0)) + self._meter_level = self._eased(0.0) self._pump_warnings() self._pump_events() + # After `_pump_events`, so a draft the final decode just produced is on + # `session.draft` by the time the wait looks for it — otherwise every paste + # would cost one extra frame, and a decode that landed in the same frame as the + # timeout would be reported as never having arrived. + self._pump_talk() if self.converse: self.card.tick_countdown() @@ -2296,9 +3696,13 @@ def _flatten(self) -> None: return self._deaf_frame = min(self._deaf_frame + 1, DEAF_COLLAPSE_FRAMES) self._eased_level = 0.0 - done = round(BARS * self._deaf_frame / DEAF_COLLAPSE_FRAMES) - for i in range(BARS - 1, BARS - 1 - done, -1): - self.levels[i] = 0.0 + # Collapsed as one level rather than bar by bar from the right. Emptying the + # right-hand bars first was the correct picture of a *scrolling* meter going + # quiet — the silence arrived at one end and travelled. A bloom has no ends to + # arrive at, so going deaf is the whole shape settling at once, which is also + # what FluidVoice does while it is processing rather than listening. + fade = 1.0 - self._deaf_frame / DEAF_COLLAPSE_FRAMES + self._meter_level = max(0.0, self._meter_level * fade) def _advance_motion(self) -> None: """Step the two §07 animations that have to remember where they were. @@ -2339,6 +3743,34 @@ def _eased(self, target: float) -> float: def _norm(db: float) -> float: return max(0.0, min(1.0, (db - DB_FLOOR) / (DB_CEIL - DB_FLOOR))) + @staticmethod + def _bar_half_height(index: int, level: float) -> float: + """Half the height of bar `index` at `level`, as FluidVoice shapes it. + + Three terms, and each is doing something the other two cannot: + + 1. **The envelope** puts the peak in the middle. A bar's ceiling falls off with + its distance from the centre, floored at 18% so the outermost bars still move + rather than sitting dead at the ends. + 2. **The exponent** bends the response so ordinary speech reaches most of the + way up. Linear is what made Flow's old meter look timid at conversational + volume — the top half of the widget was reserved for shouting. + 3. **The variation** breaks the symmetry very slightly, so a sustained note draws + a shape rather than a comb. + + Returns a half-height because the bars are mirrored about the pill's centre + line: `PILL_H` is 40 and the meter has 8 px of air, so 12 px each way. + """ + centre_distance = abs(index - (BARS - 1) / 2) + normalised = min(centre_distance / max((BARS - 1) / 2, 1), 1.0) + factor = max(_ENVELOPE_MIN, _ENVELOPE_FLOOR - normalised * _ENVELOPE_SPAN) + peak = BAR_MIN_H + (BAR_MAX_H - BAR_MIN_H) * factor + amplified = max(0.0, min(1.0, level)) ** _LEVEL_EXPONENT + variation = (_BAR_VARIATION_BASE + + _BAR_VARIATION_SWING * math.cos(index * _BAR_VARIATION_RATE)) + height = BAR_MIN_H + (peak - BAR_MIN_H) * amplified * variation + return max(BAR_MIN_H, min(BAR_MAX_H, height)) + # -- painting ---------------------------------------------------------- @property @@ -2419,43 +3851,90 @@ def pill_w(self) -> int: empty draft shows nothing), and an idle pill must not claim a panel's width it is not actually sitting under. """ - front = self.front - return front.width if getattr(front, "_visible", False) else PILL_W - - def _sync_dock(self) -> None: - """Resize and reposition for whichever panel is up, right edge held fixed. - - Idempotent and cheap once nothing has changed, so it can run from two places - without either caring which ran first: this pill's own frame, *and* a panel's - `reposition`, which needs this pill's true — already docked — position before - it can put itself directly above or below it. Whichever runs first leaves the - other with nothing left to do. - - The right edge is what a bare pill has always anchored near (`right - PILL_W - - 28` at rest), so growing to dock keeps that edge still and moves the left edge - to meet it — the same edge a docked panel's own width now matches exactly. - - **The move is checked, not assumed** (2026-08-09). The width change used to be - the only trigger, so one `geometry` call carried the whole dock and there was - no second chance at it: `scripts/reel.py` caught the pill 420 px wide at its - *bare* x, hanging 215 px past the screen edge and visibly unjoined from the - panel above it, for five seconds at a stretch. The pill's own state was never - wrong — only ever (832, 420, 420) or (1047, 205, 205) — so the resize landed - and the move did not, and `w == self._docked_w` then answered "nothing to do" - on every frame after. Comparing against the window instead of against a - remembered width means the next frame fixes it, whatever dropped it. - - The position asked for is clamped into the work area, so a window manager has - no reason to refuse it; one that did would be re-asked every frame. + # The panel width, unconditionally, and that is the point. This used to answer + # `PILL_W` while nothing was docked, so the pill jumped 205 -> 420 the moment a + # draft appeared and back again when it went — the most visible motion on the + # screen, on every single utterance. One width, whatever is happening. + # + # Read from the constant rather than through `self.front`, which is what it did + # while the answer depended on which panel was up. It does not any more: the + # bubble and the card are the same width by construction (`apply_panel_width` + # sets both), and reaching through a window meant this could be asked before + # there was one to ask. + return BUBBLE_W + + def _sync_shell(self) -> None: + """One window, sized for whatever band is up, with its bottom edge held still. + + **This was `_sync_dock`, and the dock is gone.** The pill and its panel were two + windows the app kept adjacent by hand: a width they had to agree on, an + above-or-below decision, a `_docked_above` flag so the pill knew which corners to + square off, and an ordering rule saying whichever ran first left the other with + nothing to do. `scripts/reel.py` once caught them 215 px apart for five seconds, + because a resize had landed and the matching move had not — a failure only + possible when two windows have to be moved in two calls the compositor is free to + show a frame apart. + + There is one window now, so there is nothing to keep in step. What is left is a + height: the pill row, plus the panel band when a panel is up. + + **The bottom edge is the anchor**, and that is the whole of "the controls do not + move". A panel opening grows the window *upward* — the chip row, the meter and + the Send button stay at the pixel they were at, because they are measured from a + foot that never moves. Growing downward, or centring the growth, would move every + control on the surface every time a draft appeared. + + Idempotent and cheap once nothing has changed, so it can run from the frame pump + and from a panel's `reposition` without either caring which got there first. + + The geometry is compared against the *window* rather than against a remembered + value, for the reason the dock learned the hard way: state that says the move + happened is not evidence that it did, and comparing against the window means the + next frame fixes whatever dropped it. """ + # The band's *actual* height, not the ceiling it is allowed. Asking `band_h()` + # here made the shell 224 px tall around a 130 px band and left the row floating + # 54 px below it — which the shots caught immediately, because a detached row is + # exactly the two-boxes look this window was merged to end. + # Parked, with an icon standing in for it. Re-asserting geometry here is what + # dragged it straight back on screen the moment it was hidden — the frame pump + # runs thirty times a second and this used to win every one of them. + if self._hidden: + return + front = self.front + band = min(self.band_h(), max(0, int(getattr(front, "_h", 0) or 0))) if getattr(front, "_visible", False) else 0 + h = PILL_H + band w = self.pill_w - if w != self._docked_w: - left, _top, _right, _bottom = self.work - self.x = max(left, self.x + self._docked_w - w) - self._docked_w = w + left, top, right, bottom = self.work + # **Where it is, not where it belongs.** This asked `_placed` for both, on every + # frame, which meant a drag was undone before the hand had left the mouse: the + # pill snapped back to centre and could not be moved at all. `_sync_dock` never + # had the fault because it only recomputed x when the *width* changed, which was + # rare; recomputing unconditionally is what the merge introduced. + # + # `_placed` is still the answer at startup and whenever the pointer changes + # monitor — `_sync_monitor` asks it there, which is the one place a reposition is + # actually wanted. + foot = self.y + self._shell_h + x = max(left, min(self.x, right - w)) + y = max(top + EDGE_AIR, min(foot - h, bottom - h)) + # The width is compared too, and leaving it out was a defect. `apply_panel_width` + # rebinds `BUBBLE_W` while Flow is running — the panel-size setting — so `w` + # changes without x, y or the height changing with it. The row then kept the + # width it was built at while the band above it took the new one, which is two + # boxes of different widths stacked in one window, and exactly what a screenshot + # of "panel size: larger" showed. `_docked_w` is what `_draw` measures the row + # against, so it has to move in the same breath as the canvas. + if (self.x, self.y, self._shell_h, self._docked_w) != (x, y, h, w): + self.x, self.y, self._shell_h, self._docked_w = x, y, h, w self.canvas.configure(width=w) + self.canvas.place(x=0, y=h - PILL_H, width=w, height=PILL_H) if self.window_geometry() != (w, self.x, self.y): - self.geometry(f"{w}x{PILL_H}+{self.x}+{self.y}") + self.geometry(f"{w}x{h}+{self.x}+{self.y}") + + #: Kept under the old name because every caller in the app and the suite says it, and + #: the two never meant different things — the dock *was* the shell, badly. + _sync_dock = _sync_shell def window_geometry(self) -> tuple[int, int, int]: """Where this window actually is, as (width, x, y). @@ -2520,16 +3999,21 @@ def _draw(self) -> None: accent = self.accent w = self._docked_w seam = None - if w == PILL_W: - radius = PILL_H // 2 # idle: the capsule this pill has always been + if self._shell_h == PILL_H: + radius = PILL_H // 2 # alone in the window: the capsule this has always been else: - # Docked: squared on the seam it shares with the panel, rounded on the - # free-standing side, at the panel's own 8 px — one shape language, not a - # capsule with a corner cut off. - radius = (0, 0, 8, 8) if self._docked_above else (8, 8, 0, 0) - # The pill is the lower surface when the panel is above it, so it is the one - # that draws nothing on the join — the panel's bottom carries the single line. - seam = "top" if self._docked_above else "bottom" + # Sharing the window with a panel band directly above. Squared on the join, + # rounded on the free-standing foot, at the panel's own 8 px — one shape + # language, not a capsule with a corner cut off. + # + # `_docked_above` used to decide this, because the panel was a window that + # could end up on either side of the pill when there was no room above. There + # is one window now and the band is always the top of it, so the answer is a + # constant and the flag that carried it is gone. + radius = (0, 0, 8, 8) + # The row is the lower surface, so it draws nothing on the join — the band's + # bottom carries the single line. + seam = "top" _panel_chrome(c, w, PILL_H, radius, self.ring_color, seam=seam) # Mic glyph: capsule + stand, drawn rather than fonted so there is no @@ -2557,12 +4041,27 @@ def _draw(self) -> None: self._draw_dots(c, mid, accent) else: # Level bars (R13). Mirrored around the centre line so quiet reads as a - # flat line rather than an empty box. - for i, lvl in enumerate(self.levels): - h = max(1.5, lvl * (PILL_H - 16) / 2) + # flat line rather than an empty box, and blooming from the middle rather + # than scrolling — see `_bar_half_height` for what changed and why. + # + # `_meter_level` and not `self.levels[i]`: every bar reads the same level + # now, and the shape between them is the envelope rather than the past. + lvl = self._meter_level + shade = accent if lvl > 0.04 else MUTED + for i in range(BARS): + h = self._bar_half_height(i, lvl) x = METER_X + i * (BAR_W + BAR_GAP) - shade = accent if lvl > 0.04 else MUTED - c.create_rectangle(x, mid - h, x + BAR_W, mid + h, fill=shade, outline="") + # Rounded caps, radius half the bar width — FluidVoice draws its bars as + # `RoundedRectangle(cornerRadius: barWidth / 2)`, and at four pixels wide + # that is the difference between a meter and a bar chart. Squared off + # when the bar is shorter than its own cap, where a smoothed polygon + # would pinch into a lozenge. + if h * 2 > BAR_W: + _round_rect(c, x, mid - h, x + BAR_W, mid + h, BAR_W / 2, + fill=shade, outline="") + else: + c.create_rectangle(x, mid - h, x + BAR_W, mid + h, + fill=shade, outline="") self._draw_label(c, w, mid, accent) def _draw_dots(self, c: tk.Canvas, mid: int, accent: str) -> None: @@ -2873,7 +4372,7 @@ def _footer(self, drawn: int) -> None: fill=MUTED, font=("Segoe UI", 8)) -class ConversationCard(tk.Toplevel): +class ConversationCard(tk.Frame): """P9's surface: a question, the answer it produced, and the turns behind them. Converse mode used to share the draft bubble, and three outside users found every @@ -2916,7 +4415,11 @@ class ConversationCard(tk.Toplevel): def __init__(self, pill: Pill) -> None: super().__init__(pill) self.pill = pill - self.bg = _shell_window(self, pill.lite, 0.0) + # A `Frame` inside the pill's window, not a window of its own. Everything this + # class draws is canvas-local and did not change; what went is the *window* - + # its own shell, its own shadow, its own position, and all the arithmetic that + # kept it touching the pill. See `Pill._sync_shell`. + self.bg = pill.bg self.configure(bg=self.bg) self.canvas = tk.Canvas(self, bg=self.bg, highlightthickness=0) self.canvas.pack() @@ -2953,7 +4456,7 @@ def __init__(self, pill: Pill) -> None: self.canvas.bind("", self._drag) self.canvas.bind("", self._enter, add="+") self.canvas.bind("", self._leave, add="+") - self.withdraw() + self.place_forget() # -- content ----------------------------------------------------------- @@ -3021,11 +4524,18 @@ def show(self) -> None: def close(self) -> None: self._visible = False - self.withdraw() + # Give the band back rather than parking a window offscreen. `park` existed + # because hiding a Toplevel on Windows cost a taskbar flicker and a restack; + # there is no window here to hide, only a `place` to undo, and the pill's shell + # shrinks to the row on the next `_sync_shell`. + self.place_forget() + self.pill._sync_shell() @property def showing(self) -> bool: - return bool(self.winfo_exists() and self.state() != "withdrawn") + # `state()` was a window's state, and this is no longer a window. A band is + # showing when it has a place in the one it lives in. + return bool(self.winfo_exists() and self.winfo_ismapped()) def _push(self, row: tuple[str, str]) -> None: self._history.append(row) @@ -3039,8 +4549,7 @@ def _push(self, row: tuple[str, str]) -> None: def _show(self) -> None: if not self._visible: self._visible = True - self.deiconify() - self.attributes("-alpha", 0.97) + self.pill._sync_shell() self._render() # -- scrolling --------------------------------------------------------- @@ -3086,32 +4595,53 @@ def work_h(self) -> int: _left, top, _right, bottom = self.pill.work return bottom - top - 2 * EDGE_AIR + def panel_h(self) -> int: + """The tallest this band may be. Asked of the pill, which owns the window. + + The row shares that window, so the band's ceiling is what the desktop has left + after the row has taken its 40 px. + """ + return self.pill.band_h() + + def _settled_h(self, want: int) -> int: + """`want`, rounded up to a whole body line and clamped to the band's ceiling. + + **The snap is what replaces FluidVoice's 80 ms debounce.** Its overlay sizes to + its content and coalesces the resizes on a timer; sizing to content is right and + the timer is a thing to get wrong from inside a render loop that already runs + thirty times a second. A height that can only change when the text gains or loses + a *line* changes a handful of times an utterance by construction — no cancelling, + nothing to leak, and the same absence of thrash. + + The foot does not move whatever this returns: `Pill._sync_shell` grows the window + upward from a fixed bottom edge, so a step here moves the top edge and nothing + else. + """ + want = max(PANEL_MIN_H, want) + over = want - PANEL_MIN_H + want = PANEL_MIN_H + -(-over // BODY_LINE_H) * BODY_LINE_H + return max(PANEL_MIN_H, min(want, self.pill.band_h())) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" return CARD_W def reposition(self) -> None: - """Item 44's anchor, with this window's width. Above whenever above fits. - - No gap now: this window docks to the pill rather than floating near it - (Phase 5, decisions.md 2026-08-09) — the two meet at one hairline seam - instead of the 10 px of air a shadow used to go in. `_sync_dock` runs first - so the pill's own position already reflects the width it is about to share, - and `_docked_above` is set here because this is the one place that already - decides which side the seam is actually on. + """Take the top band of the pill's window, or give it back. + + **This used to place a window.** It chose above-or-below against the work area, + anchored to the pill's right edge, set `_docked_above` so the pill knew which of + its corners to square off, and called `_sync_dock` first so the pill had already + settled into the width the two were about to share. All of that existed to make + two windows look like one, and none of it survives one window: the panel is the + band above the pill row, at x=0, always. """ - self.pill._sync_dock() - left, top, right, bottom = self.pill.work - x = self.pill.x + self.pill.pill_w - CARD_W - above = self.pill.y - self._h - below = self.pill.y + PILL_H - fits_below = above < top + EDGE_AIR and below + self._h <= bottom - EDGE_AIR - self.pill._docked_above = not fits_below - y = below if fits_below else above - x = max(left + EDGE_AIR, min(x, right - CARD_W - EDGE_AIR)) - y = max(top + EDGE_AIR, min(y, bottom - self._h - EDGE_AIR)) - self.geometry(f"{CARD_W}x{self._h}+{x}+{y}") + self.pill._sync_shell() + if self._visible: + self.place(x=0, y=0, width=CARD_W, height=self._h) + else: + self.place_forget() # -- holding still under the hand -------------------------------------- @@ -3223,7 +4753,11 @@ def _render(self) -> None: # which is arithmetic rather than a constant — the same bargain `Bubble._render` # strikes, and the reason a 12 000-character artifact cannot size this window # past the bottom of the display. - spare = (self.work_h() - PAD - HELP_FOOT_BAND - q_h - CARD_GAP + # Against the panel, not against the desktop. This asked `work_h()` while the + # card was free to grow to it; with the card a fixed shape that let the answer be + # sized for a 672 px window and drawn into a 184 px one, and the top of the card + # — the "agent" label — was cut off by it. + spare = (self.panel_h() - PAD - HELP_FOOT_BAND - q_h - CARD_GAP - BODY_ELIDED_H - (note_h + 4 if self._note else 0)) shown, more, a_h = "", 0, 0 if self._answer: @@ -3239,20 +4773,21 @@ def _render(self) -> None: history_h = sum(h + CARD_GAP for h in self._heights) # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - self._h = min( - max(CARD_MIN_H, PAD + history_h + self._pinned_h + HELP_FOOT_BAND), - self.work_h(), - ) + # Snug around what is on the card, stepping a line at a time — and the pill + # row below it does not move when it steps, because the shell grows upward. + self._h = self._settled_h( + PAD + history_h + self._pinned_h + HELP_FOOT_BAND) c.configure(width=CARD_W, height=self._h) self.reposition() c.delete("body") # Squared on the seam it shares with the docked pill, rounded on the free # side — `reposition` is what decides above-vs-below, since it already has to. - above = getattr(self.pill, "_docked_above", True) - corners = (8, 8, 0, 0) if above else (0, 0, 8, 8) + # Always the top band of the one window now: rounded head, squared foot on the + # join it shares with the pill row below it. + corners = (8, 8, 0, 0) _panel_chrome(c, CARD_W, self._h, corners, self.ring_color, - seam="bottom" if above else "top") + seam="bottom") # -- the history, in what is left above the pinned block y, floor = PAD, PAD + self._view_h() @@ -3383,7 +4918,7 @@ def _new_conversation(self) -> None: self.pill.session.new_conversation() -class Bubble(tk.Toplevel): +class Bubble(tk.Frame): """The draft, floated above the pill (R14) with Refine / Continue / Send (R15).""" #: Copied from the pill at construction rather than read back off it, for the reason @@ -3408,7 +4943,11 @@ def __init__(self, pill: Pill) -> None: super().__init__(pill) self.pill = pill self.lite = pill.lite - self.bg = _shell_window(self, pill.lite, 0.0) + # A `Frame` inside the pill's window, not a window of its own. Everything this + # class draws is canvas-local and did not change; what went is the *window* - + # its own shell, its own shadow, its own position, and all the arithmetic that + # kept it touching the pill. See `Pill._sync_shell`. + self.bg = pill.bg self.configure(bg=self.bg) self.canvas = tk.Canvas(self, bg=self.bg, highlightthickness=0) self.canvas.pack() @@ -3455,7 +4994,7 @@ def __init__(self, pill: Pill) -> None: self.canvas.bind("", self._enter, add="+") self.canvas.bind("", self._leave, add="+") self.canvas.bind("", self._context_menu) - self.withdraw() + self.place_forget() # -- content ----------------------------------------------------------- @@ -3530,8 +5069,7 @@ def show_sent(self, text: str, problem: str = "") -> None: self._note = problem if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def show(self, text: str) -> None: @@ -3549,8 +5087,7 @@ def show(self, text: str) -> None: self._render() if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() def show_partial(self, text: str) -> None: # Partials are dimmed: they contain hallucinated fragments on mid-word @@ -3559,8 +5096,7 @@ def show_partial(self, text: str) -> None: self._for_activity = False if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def note(self, msg: str, undoable: bool = False) -> None: @@ -3577,8 +5113,7 @@ def surface(self, msg: str) -> None: self._for_activity = False if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def hide(self) -> None: @@ -3592,7 +5127,12 @@ def hide(self) -> None: self._text = self._partial = self._note = self._sent = "" self._note_undo = False self._for_activity = False - self.withdraw() + # Give the band back rather than parking a window offscreen. `park` existed + # because hiding a Toplevel on Windows cost a taskbar flicker and a restack; + # there is no window here to hide, only a `place` to undo, and the pill's shell + # shrinks to the row on the next `_sync_shell`. + self.place_forget() + self.pill._sync_shell() # -- geometry ---------------------------------------------------------- @@ -3607,80 +5147,56 @@ def work_h(self) -> int: _left, top, _right, bottom = self.pill.work return bottom - top - 2 * EDGE_AIR + def panel_h(self) -> int: + """The tallest this band may be. Asked of the pill, which owns the window. + + The row shares that window, so the band's ceiling is what the desktop has left + after the row has taken its 40 px. + """ + return self.pill.band_h() + + def _settled_h(self, want: int) -> int: + """`want`, rounded up to a whole body line and clamped to the band's ceiling. + + **The snap is what replaces FluidVoice's 80 ms debounce.** Its overlay sizes to + its content and coalesces the resizes on a timer; sizing to content is right and + the timer is a thing to get wrong from inside a render loop that already runs + thirty times a second. A height that can only change when the text gains or loses + a *line* changes a handful of times an utterance by construction — no cancelling, + nothing to leak, and the same absence of thrash. + + The foot does not move whatever this returns: `Pill._sync_shell` grows the window + upward from a fixed bottom edge, so a step here moves the top edge and nothing + else. + """ + want = max(PANEL_MIN_H, want) + over = want - PANEL_MIN_H + want = PANEL_MIN_H + -(-over // BODY_LINE_H) * BODY_LINE_H + return max(PANEL_MIN_H, min(want, self.pill.band_h())) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" return BUBBLE_W - def reposition(self, lift: int = 0) -> None: - """Anchor above the pill, clamped inside the work area. - - The clamp used to be `max(8, x)` alone, which pins the left edge and lets the - right edge run off the display — so on a screen whose coordinates the app had - got wrong, the bubble hung half outside it with its buttons unreachable. Both - edges are bounded now, and against the work area rather than the raw screen. - - It is only a *clamp*, and that was the gap: a window taller than the work area - cannot be placed inside it however carefully the position is computed, so the - reply path — the one item 37 deliberately did not touch — ran 795 px past the - bottom on an ordinary answer and 3 515 px on an artifact, at every pill corner. - The height is fitted in `_render` now (`work_h`), which is what makes the - arithmetic below a guarantee rather than a best effort. - - Fitting it exposed the next thing, at the desk: with the pill dragged to the top of - the work area there is no "above" left, so the bubble clamped to the top edge and - was drawn **over the pill it is anchored to**. Nothing clipped — that is item 42's - guarantee and it survives — but an anchor pointing at something it covers is not - an anchor. Below is the fallback, and only that. - """ - # No gap now: this window docks to the pill rather than floating near it - # (Phase 5, decisions.md 2026-08-09) — the two meet at one hairline seam - # instead of the 10 px of air a shadow used to go in. `_sync_dock` runs - # first so the pill's own position already reflects the width it is about - # to share, and `_docked_above` is set here because this is the one place - # that already decides which side the seam is actually on. - self.pill._sync_dock() - left, top, right, bottom = self.pill.work - x = self.pill.x + self.pill.pill_w - BUBBLE_W - above = self.pill.y - self._h - below = self.pill.y + PILL_H - # Above whenever above fits, which is every ordinary placement and is why this - # reads as one anchor rather than two. Below only when above has no room and below - # does — a fallback, not a mode. When *neither* fits, `above` goes through and the - # clamp below does what it has always done: a window as tall as the desktop cannot - # be placed clear of a pill on either side of it, and inventing a third rule for - # that would be pretending otherwise. - fits_below = above < top + EDGE_AIR and below + self._h <= bottom - EDGE_AIR - self.pill._docked_above = not fits_below - y = below if fits_below else above - x = max(left + EDGE_AIR, min(x, right - BUBBLE_W - EDGE_AIR)) - y = max(top + EDGE_AIR, min(y + lift, bottom - self._h - EDGE_AIR)) - self.geometry(f"{BUBBLE_W}x{self._h}+{x}+{y}") - - def _float_up(self) -> None: - """R14: rise into place rather than appearing, so the eye follows it. - - Generation-guarded. Each run schedules eight `after` callbacks that each move - the window, so two overlapping runs fight over the position and the bubble - visibly jitters between two places — which is what a fast show/hide/show cycle - produces. - """ - steps = 8 - self._anim += 1 - mine = self._anim - - def step(i: int) -> None: - if not self._visible or mine != self._anim: - return - t = i / steps - ease = 1 - (1 - t) ** 3 - self.attributes("-alpha", 0.96 * ease) - self.reposition(lift=int(18 * (1 - ease))) - if i < steps: - self.after(16, step, i + 1) + def reposition(self) -> None: + """Take the top band of the pill's window, or give it back. - step(0) + **This used to place a window**, and it took a `lift` argument so `_float_up` + could animate it in. Both are gone: the panel is the band above the pill row, at + x=0, and there is nothing left to move it relative to. + `_float_up` went with it. R14 asked for an appearance the eye could follow, and + 18 px of travel earned that when the bubble was a separate window arriving beside + another one. Inside a single shell there is nothing to arrive *at* — the window + itself grows upward from a bottom edge that never moves, which is the same cue + with no motion under it. + """ + self.pill._sync_shell() + if self._visible: + self.place(x=0, y=0, width=BUBBLE_W, height=self._h) + else: + self.place_forget() # -- holding still under the hand -------------------------------------- @@ -4013,16 +5529,26 @@ def _render(self) -> None: # `BODY_ELIDED_H` is counted in unconditionally here: a capped body always has # something above it to report, and guessing the other way is how a line lands # on a control. - body_cap = BODY_MAX_H - if self._frozen(): - around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) - if partial_h: - around += partial_h + PARTIAL_GAP - if self._sent: - around += 16 - if self._act is not None: - around += 20 - body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self._h - around)) + # Unconditional now, and it used to run only while `_frozen()`. That gate was + # right when the window sized itself to the body: the room left in it was a hard + # number only while something was stopping it from growing. The window is a fixed + # shape now (`PANEL_H`), so the room left in it is *always* a hard number, and a + # body still asking for `BODY_MAX_H` would draw 340 px of text through the note + # and the chip row of a 184 px panel. + # `SETTINGS_H` is in here for the reason everything else is: the body's budget is + # what is left after the fixed furniture, and a strip the height did not know + # about would be a strip drawn over the first line of the draft. + around = 74 + SETTINGS_H + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + if partial_h: + around += partial_h + PARTIAL_GAP + if self._sent: + around += 16 + if self._act is not None: + around += 20 + # Against the band's *ceiling*, not against the height it happens to be: the + # height is about to be computed from this, so reading it here would let a short + # frame pin the body short on the next one and never grow back. + body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self.panel_h() - around)) shown, earlier, text_h = self._body_slot(body, body_cap) if not body: # `_body_slot` probes `shown or " "` so `bbox` always has something to answer @@ -4065,18 +5591,24 @@ def _render(self) -> None: # character artifact to 4 179 px on a 672 px desktop. # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - self._h = min(max(96, text_h + extra + 74), self.work_h()) + # Snug around the draft again, stepping a line at a time rather than + # tracking every frame — see `_settled_h`. + self._h = self._settled_h(text_h + extra + 74 + SETTINGS_H) c.configure(width=BUBBLE_W, height=self._h) self.reposition() c.delete("body") # Squared on the seam it shares with the docked pill, rounded on the free # side — `reposition` is what decides above-vs-below, since it already has to. - above = getattr(self.pill, "_docked_above", True) - corners = (8, 8, 0, 0) if above else (0, 0, 8, 8) + # Always the top band of the one window now: rounded head, squared foot on the + # join it shares with the pill row below it. + corners = (8, 8, 0, 0) _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, - seam="bottom" if above else "top") + seam="bottom") y = PAD + # Above the words, because it describes what will happen to them. See + # `SETTINGS_H` for why it lives here and not on the idle row. + y += _settings_row(c, self.pill, BUBBLE_W, y) if self._sent: c.create_text( PAD, y, anchor="nw", text="sent", fill=MUTED, @@ -4344,7 +5876,7 @@ def tick_activity(self) -> None: if surfacing: self._for_activity = True self._visible = True - self.deiconify() + self.pill._sync_shell() elif act is None and self._for_activity and not ( self._text or self._partial or self._note ): @@ -4353,10 +5885,9 @@ def tick_activity(self) -> None: if self._visible: self._render() if surfacing: - # After the render, not before: `_float_up` repositions against `self._h`, - # which is what the render computes. Animating first moves the window to a - # height it does not have yet and the rise starts with a jump. - self._float_up() + # After the render, not before: `reposition` places the band against + # `self._h`, which is what the render computes. + self.reposition() def _indicator(self, y: int) -> None: """The one row that says what Flow is doing, and whether it can still hear. diff --git a/native/flow_stt.swift b/native/flow_stt.swift new file mode 100644 index 0000000..a8c5257 --- /dev/null +++ b/native/flow_stt.swift @@ -0,0 +1,258 @@ +// flow_stt.swift — Flow's macOS decoder, as a process Flow talks to over a pipe. +// +// Why a separate binary rather than a Python binding: Flow's dependency budget is +// three (R16) and PyObjC is not one of them. Flow already shells out to `codex` and +// `claude`, so a subprocess that reads audio and writes text is a shape the app +// already has — and it keeps every Objective-C API on the far side of a pipe, where a +// crash is an exit code rather than a dead interpreter. +// +// Why it matters: the CT2 weights faster-whisper needs exist on HuggingFace and +// nowhere else official — SYSTRAN's GitHub ships source only. On a network that blocks +// huggingface.co that leaves copying files by hand. Apple's recogniser needs no +// download at all: the models belong to the OS. +// +// Build: +// swiftc -O -parse-as-library -o flow-stt native/flow_stt.swift +// +// `-parse-as-library` is not optional and not decoration. A single-file executable is +// treated as a script, and `@main` and script mode are mutually exclusive — the +// compiler says so in as many words. The alternative was renaming this to `main.swift` +// and going back to top-level statements; the flag keeps the file named after what it +// is, and every caller that builds it passes the flag (`flow/native.py`, CI, the guide). +// +// Two ways to run it, and the first exists so the second is worth doing: +// +// flow-stt --file some.wav one transcript to stdout, then exit. +// Judge Apple's quality on your own voice before +// anybody wires this into a dictation loop. +// +// flow-stt serve: repeatedly read a length-prefixed block of +// float32 mono 16 kHz PCM from stdin and write one +// line of transcript to stdout. Stays warm, because +// a process per utterance would cost more than the +// decode. +// +// The framing is deliberately dull: 4 bytes little-endian sample count, then that many +// float32 samples. One line of UTF-8 back per block, newline-terminated, empty line for +// silence. Anything this cannot do goes to stderr and exits non-zero, so the Python +// side can report a reason rather than a hang. +// +// Three things here are load-bearing and easy to undo by tidying: +// +// * **`@main`, not top-level code.** Swift allows statements at file scope only in a +// file called `main.swift`. This one is not, so the entry point is a type. +// * **The recogniser gets its own queue.** Its callbacks default to the main queue, +// and `transcribe` blocks the calling thread waiting for one — on the main thread +// that is a deadlock, not a slow decode. +// * **Every read out of `Data` is unaligned.** `Data` gives no alignment guarantee +// and `load(as:)` requires one; the aligned form crashes on some buffers and not +// others, which is the worst way to find out. + +import AVFoundation +import Foundation +import Speech + +let SAMPLE_RATE = 16000.0 +let DECODE_TIMEOUT: TimeInterval = 30 + +func die(_ message: String, _ code: Int32 = 1) -> Never { + FileHandle.standardError.write(("flow-stt: " + message + "\n").data(using: .utf8)!) + exit(code) +} + +/// Ask once, block until the user has answered, and treat every non-authorized answer +/// the same. The prompt is attributed to whatever launched this — a terminal, usually — +/// which is the known wart of running unbundled and is documented on the Python side. +func authorize() { + let gate = DispatchSemaphore(value: 0) + var status: SFSpeechRecognizerAuthorizationStatus = .notDetermined + SFSpeechRecognizer.requestAuthorization { got in + status = got + gate.signal() + } + gate.wait() + guard status == .authorized else { + die("speech recognition not authorized (status \(status.rawValue)). " + + "System Settings > Privacy & Security > Speech Recognition.", 2) + } +} + +func makeRecognizer() -> SFSpeechRecognizer { + guard let rec = SFSpeechRecognizer(locale: Locale(identifier: "en-US")) else { + die("no recognizer for en-US on this machine", 3) + } + guard rec.isAvailable else { die("recognizer exists but is not available", 4) } + // The whole point. Without this the audio goes to Apple's servers, which is a + // different product from the one Flow is: local by construction. + guard rec.supportsOnDeviceRecognition else { + die("on-device recognition unavailable - enable Dictation in System Settings " + + "so macOS downloads the offline model", 5) + } + // Off the main queue, deliberately. `transcribe` waits on a semaphore for the + // result, and a recogniser delivering that result *to the thread doing the waiting* + // is a deadlock. This is the line that makes the blocking call safe. + rec.queue = OperationQueue() + return rec +} + +/// Transcribe one finished buffer. Synchronous on purpose: the caller has already +/// decided this audio is complete, and Flow's decode worker owns concurrency. +func transcribe(_ rec: SFSpeechRecognizer, _ samples: [Float]) -> String { + guard !samples.isEmpty else { return "" } + guard let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, + channels: 1, interleaved: false), + let buffer = AVAudioPCMBuffer(pcmFormat: format, + frameCapacity: AVAudioFrameCount(samples.count)), + let channel = buffer.floatChannelData + else { return "" } + buffer.frameLength = AVAudioFrameCount(samples.count) + samples.withUnsafeBufferPointer { src in + if let base = src.baseAddress { + memcpy(channel[0], base, samples.count * MemoryLayout.size) + } + } + + let request = SFSpeechAudioBufferRecognitionRequest() + request.requiresOnDeviceRecognition = true + request.shouldReportPartialResults = false + // Punctuation is the difference between dictation and a transcript; Flow's cleaner + // assumes sentences. + if #available(macOS 13.0, *) { request.addsPunctuation = true } + request.append(buffer) + request.endAudio() + + let gate = DispatchSemaphore(value: 0) + // Written from the recogniser's queue and read from this one, so the handoff is + // guarded rather than assumed. + let lock = NSLock() + var text = "" + let task = rec.recognitionTask(with: request) { result, error in + if let result = result, result.isFinal { + lock.lock(); text = result.bestTranscription.formattedString; lock.unlock() + gate.signal() + } else if error != nil { + // A recogniser that heard nothing reports an error rather than an empty + // result. Silence is not a failure here — Flow's gate already decided this + // block was speech, and an empty line is how "nothing said" is spelled. + gate.signal() + } + } + // Bounded, because a recogniser that never calls back would otherwise hang the + // decode worker and, through it, the draft the user is waiting for. + if gate.wait(timeout: .now() + DECODE_TIMEOUT) == .timedOut { task.cancel() } + lock.lock(); defer { lock.unlock() } + return text +} + +/// Read exactly `count` bytes, or nil if the pipe closed first. +func readExactly(_ handle: FileHandle, _ count: Int) -> Data? { + var out = Data() + while out.count < count { + let chunk = handle.readData(ofLength: count - out.count) + if chunk.isEmpty { return nil } + out.append(chunk) + } + return out +} + +func serve(_ rec: SFSpeechRecognizer) { + let input = FileHandle.standardInput + let output = FileHandle.standardOutput + while true { + guard let header = readExactly(input, 4) else { return } // Flow is gone + // Unaligned on purpose: `Data`'s backing store carries no alignment guarantee, + // and the aligned `load(as:)` traps on buffers that happen not to be. + let count = Int(header.withUnsafeBytes { + $0.loadUnaligned(fromByteOffset: 0, as: UInt32.self).littleEndian + }) + guard count > 0, count < 30 * Int(SAMPLE_RATE) * 60 else { return } + guard let payload = readExactly(input, count * 4) else { return } + var samples = [Float](repeating: 0, count: count) + // `_ =` because `withUnsafeMutableBytes` hands back whatever the closure + // returns, here `copyBytes`' byte count, and this leg builds with + // `-warnings-as-errors` so an ignored result is a failure rather than a note. + _ = samples.withUnsafeMutableBytes { dst in + payload.copyBytes(to: dst.bindMemory(to: UInt8.self)) + } + let line = transcribe(rec, samples).replacingOccurrences(of: "\n", with: " ") + if let data = (line + "\n").data(using: .utf8) { output.write(data) } + } +} + +func fromFile(_ path: String, _ rec: SFSpeechRecognizer) { + guard let file = try? AVAudioFile(forReading: URL(fileURLWithPath: path)) else { + die("cannot read \(path)") + } + guard let target = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, + channels: 1, interleaved: false), + let converter = AVAudioConverter(from: file.processingFormat, to: target), + let source = AVAudioPCMBuffer(pcmFormat: file.processingFormat, + frameCapacity: AVAudioFrameCount(file.length)) + else { die("cannot convert \(path) to 16 kHz mono") } + do { try file.read(into: source) } catch { die("read failed: \(error)") } + + let ratio = SAMPLE_RATE / file.processingFormat.sampleRate + let frames = AVAudioFrameCount(Double(source.frameLength) * ratio) + 4096 + guard let out = AVAudioPCMBuffer(pcmFormat: target, frameCapacity: frames) else { + die("cannot allocate output buffer") + } + var supplied = false + var error: NSError? + converter.convert(to: out, error: &error) { _, status in + if supplied { + status.pointee = .endOfStream + return nil + } + supplied = true + status.pointee = .haveData + return source + } + if let error = error { die("resample failed: \(error.localizedDescription)") } + guard let channel = out.floatChannelData else { die("no samples after resample") } + let samples = Array(UnsafeBufferPointer(start: channel[0], + count: Int(out.frameLength))) + print(transcribe(rec, samples)) +} + +enum Mode { + case probe + case file(String) + case serve +} + +@main +struct FlowSTT { + /// **Arguments are checked before anything is asked of the user.** A typo must not + /// raise a permission prompt, and CI has to be able to reach the usage line on a + /// runner where no permission could ever be granted — which is the only way the + /// compile leg can prove this binary links and runs at all. + static func main() { + let args = Array(CommandLine.arguments.dropFirst()) + let mode: Mode + if args == ["--probe"] { + mode = .probe + } else if args.count == 2 && args[0] == "--file" { + mode = .file(args[1]) + } else if args.isEmpty { + mode = .serve + } else { + die("usage: flow-stt [--probe | --file AUDIO]") + } + + // Everything past here needs a recogniser, and a recogniser needs consent. + authorize() + let rec = makeRecognizer() + switch mode { + case .probe: + // What the Python side calls to decide whether this engine exists at all, + // before it commits a session to it. Reaching this line is the answer. + print("ok") + case .file(let path): + fromFile(path, rec) + case .serve: + serve(rec) + } + } +} diff --git a/scripts/mac_area_probe.py b/scripts/mac_area_probe.py new file mode 100644 index 0000000..a6f1f63 --- /dev/null +++ b/scripts/mac_area_probe.py @@ -0,0 +1,78 @@ +"""Which call on Aqua knows where the Dock is. + +`_tk_work_area` measures the usable area by maximising a probe window and reading back +where the window manager put it. On Windows that agrees with `SystemParametersInfoW` +exactly. On macOS it came back as the whole screen — `state("zoomed")` neither raised +nor maximised — so the pill was placed 24 px above 878 and landed inside the Dock band. + +Three Aqua-specific candidates are asked here instead of guessed at: + + **wm maxsize** - Tk's Aqua port answers this from `[NSScreen visibleFrame]`, so it + should already be the screen minus the menu bar and the Dock. It gives a *size* and + no origin, which is why it is not enough on its own. (It is useless on Windows, where + it answers with the whole screen even with a taskbar present - which is why the + maximise probe exists at all.) + + **a window asked for +0+0** - Aqua will not put a titled window under the menu bar, so + where it actually lands is the top of the usable area. + + **wm attributes -fullscreen** - the whole screen including the menu bar, as a control: + if this and `zoomed` agree, `zoomed` is being treated as fullscreen. + +Every measurement comes from a window with the same decoration, which matters: +`maxsize` is a maximum *content* size, so it is short by whatever title bar its +window wears. Taking the origin from one window and the size from another counted +a 28 px title bar twice and put the answer 28 px too low. + + uv run python scripts/mac_area_probe.py +""" + +import sys +import tkinter as tk + +root = tk.Tk() +root.withdraw() +sw, sh = root.winfo_screenwidth(), root.winfo_screenheight() +print(f" platform {sys.platform} tk {tk.TkVersion} " + f"patch {root.tk.call('info', 'patchlevel')}") +print(f" screen {sw} x {sh}") + +mw, mh = root.maxsize() +print(f" wm maxsize {mw} x {mh} (lost: {sw - mw} w, {sh - mh} h)") + + +def probe(name, setup): + win = tk.Toplevel(root) + win.attributes("-alpha", 0.0) + win.geometry("200x120+80+80") + note = "ok" + try: + setup(win) + win.update_idletasks() + except tk.TclError as exc: + note = f"FAILED ({exc})" + x, y = win.winfo_rootx(), win.winfo_rooty() + w, h = win.winfo_width(), win.winfo_height() + print(f" {name:<24} ({x}, {y}, {x + w}, {y + h}) {note}") + win.destroy() + return x, y, w, h + + +probe("zoomed (today's probe)", lambda w: w.state("zoomed")) +_x, free, _w, _h = probe("asked for +80+300", lambda w: w.geometry("200x120+80+300")) +_x, clamped, _w, _h = probe("asked for +0+0", lambda w: w.geometry("200x120+0+0")) +probe("fullscreen (control)", lambda w: w.attributes("-fullscreen", True)) + +# `maxsize` is a maximum *content* size, so it is short by the decoration of the +# window that answered it - 735 from a titled window against 763 from an +# `overrideredirect` one on the same display. Every part of this comes from one +# probe so the title bar appears on both sides and cancels; mixing two windows +# counted it twice and put the answer 28 px too low. +title = free - 300 +top = clamped - title +bottom = top + mh + title +print(f"\n title bar {title} px (from +80+300, below any menu bar)") +print(f" menu bar {top} px (from +0+0, less that title bar)") +print(f" so the usable area is (0, {top}, {mw}, {bottom})") +print(f" and the Dock starts at y = {bottom} ({sh - bottom} px tall)") +root.destroy() diff --git a/scripts/mac_float_probe.py b/scripts/mac_float_probe.py new file mode 100644 index 0000000..fd733e7 --- /dev/null +++ b/scripts/mac_float_probe.py @@ -0,0 +1,144 @@ +"""Which window configuration on Aqua is bare, stays up, and still takes a click. + +Reported from a Mac: click the app you want to dictate into and Flow's window vanishes, +and clicking Send does nothing. + +**The first run of this probe found the cause, and it was not the one it was written to +test.** Four variants, two with `overrideredirect` and two without. The two without were +the two whose buttons a click reached, and the two without were the two still on screen +after clicking another app. `overrideredirect` on Aqua makes a window both deaf and +fugitive, and it is the one line Flow uses on every window it owns. + +The NSPanel class this was really written to test never applied at all - Tk answered +`cannot change the class after the mac window is created`, even on a withdrawn window - +so the panel was never the variable. + +That leaves the question `overrideredirect` was there to answer: it is what takes the +title bar off. Tk's own complaint listed the way out. Asked for a nonsense style bit it +enumerated the real ones - `titled, closable, miniaturizable, resizable, +fullsizecontentview, utility, nonactivatingpanel, docmodal` - and a style mask *without* +`titled` is a window with no title bar that was never made deaf to begin with. + +**A title bar is measured here, not looked at.** Ask for a window at a known y; how far +below that the client area lands is the decoration on it, and zero means bare. Same +arithmetic `_aqua_work_area` uses to find the menu bar. `winfo_viewable` is not trusted +for the disappearing - it reported all four windows healthy while two of them were gone +from the screen - so that one bit is the only thing left worth a human glance. + + uv run python scripts/mac_float_probe.py + +Click another application when it says to, note which numbers vanish, then click every +button. Paste the output. +""" + +import sys +import tkinter as tk + +AWAY_SEC = 8 +CLICK_SEC = 25 +ASK_Y = 220 +W, H = 420, 58 + +root = tk.Tk() +root.withdraw() +log: list[int] = [] +decor: dict[int, int] = {} + + +def vocabulary(name: str) -> None: + """Ask for a nonsense value; Tk's complaint enumerates the real ones.""" + try: + root.wm_attributes(name, "there-is-no-such-value") + print(f" {name:<12} accepted nonsense - it is not validated on this build") + except tk.TclError as exc: + print(f" {name:<12} {exc}") + + +print(f"platform {sys.platform}, tk {tk.TkVersion} " + f"patch {root.tk.call('info', 'patchlevel')}\n") +print("What this build will accept:") +for attribute in ("-class", "-stylemask"): + vocabulary(attribute) + + +def mask(win, *flags): + """A style mask with no `titled` bit is a window with no title bar.""" + win.wm_attributes("-stylemask", " ".join(flags)) + + +def window(n: int, title: str, setup, cls=None) -> tk.Toplevel: + """One variant, configured while unmapped, then measured for decoration. + + Whatever `setup` raises is printed and the window is still shown - a variant that + could not be configured is a *result*, and it belongs on screen wearing its reason so + that what is seen and what is printed cannot come apart. + """ + win = tk.Toplevel(root, class_=cls) if cls else tk.Toplevel(root) + win.withdraw() + note = "ok" + try: + setup(win) + except tk.TclError as exc: + note = f"FAILED ({exc})" + asked = ASK_Y + n * (H + 14) + win.geometry(f"{W}x{H}+40+{asked}") + win.configure(bg="#12141a") + tk.Label(win, text=f"{n} {title}", bg="#12141a", fg="#e6e8ee", + font=("Helvetica", 11)).pack(side="left", padx=10) + tk.Button(win, text=f"click {n}", highlightbackground="#12141a", + command=lambda: log.append(n)).pack(side="right", padx=10) + win.deiconify() + win.attributes("-topmost", True) + win.update_idletasks() + decor[n] = win.winfo_rooty() - asked + print(f" {n} {title:<44} title bar {decor[n]:>3} px {note}") + return win + + +print("\nVariants (title bar measured, not looked at):") +made = { + 1: window(1, "nothing asked for (the control)", lambda w: None), + 2: window(2, "overrideredirect (what Flow does today)", + lambda w: w.overrideredirect(True)), + 3: window(3, "stylemask {} - no bits at all", lambda w: mask(w)), + 4: window(4, "stylemask {fullsizecontentview}", + lambda w: mask(w, "fullsizecontentview")), + 5: window(5, "stylemask {} on a Toplevel made as NSPanel", + lambda w: mask(w), cls="NSPanel"), + 6: window(6, "stylemask {nonactivatingpanel} as NSPanel", + lambda w: mask(w, "nonactivatingpanel"), cls="NSPanel"), +} + + +def survey() -> None: + print(f"\nAfter {AWAY_SEC}s in the background, Tk claims:") + for n, win in made.items(): + try: + print(f" {n} viewable={bool(win.winfo_viewable())}") + except tk.TclError as exc: + print(f" {n} gone ({exc})") + print(" (Tk said all four were healthy last time while two were off the screen,\n" + " so please say which numbers you can actually still see.)") + print(f"\nNow click every button once, in any order. {CLICK_SEC}s.") + root.after(CLICK_SEC * 1000, finish) + + +def finish() -> None: + print("\nResults:") + for n in made: + print(f" {n} title bar {decor.get(n, -1):>3} px " + f"{'CLICK REACHED IT' if n in log else 'nothing got through'}") + print("\nWanted: title bar 0 px, still on screen, and the click reaching it.\n" + "Paste all of this back, with which numbers stayed visible.") + root.destroy() + + +print(f""" +Six windows are on screen down the left. + + **Click another application now** - Finder, a browser, anything - and leave Flow in + the background. Note which numbers disappear. + +Recording in {AWAY_SEC}s.""") +root.after(AWAY_SEC * 1000, survey) +root.mainloop() diff --git a/scripts/mac_frame_probe.py b/scripts/mac_frame_probe.py new file mode 100644 index 0000000..58afe43 --- /dev/null +++ b/scripts/mac_frame_probe.py @@ -0,0 +1,108 @@ +"""Which of five ways of asking Aqua for a frameless window actually works. + +Reported from a Mac: the pill sat there with a title bar and three traffic lights on it, +while the panels above it were correctly frameless. The difference between them is that +`Pill` **is** the root window — `class Pill(tk.Tk)` — and `Bubble`/`ConversationCard` are +`Toplevel`s. Aqua creates the root's `NSWindow` before Tk can restyle it, so what works +for a Toplevel does not necessarily work for `.`. + +Two wrong guesses have already been spent on this, so this asks the machine instead. +It opens five small windows in a row down the left of the screen, each labelled with the +technique that made it, and each *saying* whether it is a root window or a Toplevel. + + uv run python scripts/mac_frame_probe.py + +**Look at them and tell me which ones have no title bar.** That is the whole output; the +console text is only there to say what you are looking at. It closes itself after 20 +seconds. +""" + +import sys +import tkinter as tk +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +HOLD_SEC = 20 +W, H = 420, 64 + + +def style(win) -> str: + """`::tk::unsupported::MacWindowStyle`, reporting rather than assuming.""" + try: + win.tk.call("::tk::unsupported::MacWindowStyle", "style", + win._w, "plain", "noActivates") + return "ok" + except tk.TclError as exc: + return f"FAILED ({exc})" + + +def label(win, text: str, y: int) -> None: + win.geometry(f"{W}x{H}+40+{y}") + tk.Label(win, text=text, bg="#12141a", fg="#e6e8ee", + font=("Helvetica", 13)).pack(fill="both", expand=True) + win.configure(bg="#12141a") + + +def main() -> None: + notes = [] + + # 1. The root, styled the way Flow does it today. This is the one that came back + # with traffic lights on it. + root = tk.Tk() + root.overrideredirect(True) + root.attributes("-topmost", True) + notes.append(("1 root: overrideredirect then MacWindowStyle", style(root))) + label(root, "1 ROOT overrideredirect + MacWindowStyle", 60) + + # 2. The root again, but restyled while unmapped. Aqua builds the NSWindow when the + # window is first mapped, so a style asked for afterwards may never be applied to + # anything — withdrawing and remapping is the documented way to force a rebuild. + two = tk.Toplevel(root) # stands in for a root; see 3 for the real second root + two.withdraw() + two.overrideredirect(True) + notes.append(("2 toplevel: withdraw, overrideredirect, deiconify", style(two))) + label(two, "2 TOPLEVEL withdraw -> style -> deiconify", 150) + two.deiconify() + two.attributes("-topmost", True) + + # 3. A Toplevel with nothing but overrideredirect — what the bubble and card do, and + # what already works in the app. The control. + three = tk.Toplevel(root) + three.overrideredirect(True) + three.attributes("-topmost", True) + label(three, "3 TOPLEVEL overrideredirect only (the control)", 240) + + # 4. A Toplevel with the style and no overrideredirect, to find out which of the two + # is actually doing the work. + four = tk.Toplevel(root) + notes.append(("4 toplevel: MacWindowStyle only", style(four))) + label(four, "4 TOPLEVEL MacWindowStyle only", 330) + four.attributes("-topmost", True) + + # 5. The root, withdrawn and remapped. If this one is bare, the fix is three lines + # and `Pill` can stay a `tk.Tk`. If it is not, the pill has to become a Toplevel + # under a hidden root, which is a real change to how the app starts. + root.withdraw() + root.overrideredirect(True) + notes.append(("5 root: withdraw -> overrideredirect -> deiconify", style(root))) + root.deiconify() + root.attributes("-topmost", True) + root.geometry(f"{W}x{H}+40+60") + + print(f"platform {sys.platform}, tk {tk.TkVersion}\n") + for name, got in notes: + print(f" {name:<52} {got}") + print(f""" +Five windows are on screen now, down the left. Window 1 is the root as Flow builds +it today and window 5 is the same root after a withdraw/deiconify - if 5 is bare and +1 is not, the fix is three lines. If neither root is bare, `Pill` has to stop being +`tk.Tk`, which is a larger change. + +Say which numbers have no title bar. Closing in {HOLD_SEC}s.""") + root.after(HOLD_SEC * 1000, root.destroy) + root.mainloop() + + +if __name__ == "__main__": + main() diff --git a/scripts/mac_probe.py b/scripts/mac_probe.py new file mode 100644 index 0000000..4ee5ded --- /dev/null +++ b/scripts/mac_probe.py @@ -0,0 +1,118 @@ +"""Ask a Mac the two questions Windows answers with one API call, and print what it says. + +Flow places its windows against a *work area* — the desktop minus whatever the OS keeps +for itself. On Windows that is one call (`SystemParametersInfoW(SPI_GETWORKAREA)`) and +one more for the monitor under the pointer. Off Windows there is neither, so `ui.py` +falls back to what Tk will admit to, and the first Mac to run it put the pill under the +Dock. + +This exists so the fix is a measurement rather than a third guess. It prints every +number Tk can be asked for, next to what Flow currently computes from them, and it also +checks the two window attributes the pill depends on — the borderless frame and the +no-activate policy — because a Mac reported those wrong in the same breath. + + uv run python scripts/mac_probe.py + +Nothing is opened that the user has to close: the probe window is withdrawn before it +would be seen, except for the two seconds the visible check needs. +""" + +import sys +import tkinter as tk +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +import flow.ui as ui # noqa: E402 + + +def row(label: str, value) -> None: + print(f" {label:<34} {value}") + + +def main() -> None: + root = tk.Tk() + root.withdraw() + sw, sh = root.winfo_screenwidth(), root.winfo_screenheight() + + print(f"platform: {sys.platform} tk {tk.TkVersion}") + print("\nwhat Tk says about the screen") + row("winfo_screenwidth/height", f"{sw} x {sh}") + try: + row("wm_maxsize()", root.wm_maxsize()) + except Exception as exc: + row("wm_maxsize()", f"FAILED: {exc}") + row("winfo_vrootwidth/height", + f"{root.winfo_vrootwidth()} x {root.winfo_vrootheight()}") + row("winfo_vrootx/y", f"{root.winfo_vrootx()}, {root.winfo_vrooty()}") + row("winfo_fpixels('1i') (dpi)", root.winfo_fpixels("1i")) + + # The measurement that would settle it: zoom a window and read where it landed. + # Supported on Windows and, depending on the build, on Aqua. If it works, its + # numbers are the work area *including* the origin, which `wm_maxsize` cannot give. + print("\nthe measurement that would settle it") + probe = tk.Toplevel(root) + probe.geometry("200x120+80+80") + try: + probe.state("zoomed") + probe.update_idletasks() + x, y = probe.winfo_rootx(), probe.winfo_rooty() + w, h = probe.winfo_width(), probe.winfo_height() + row("zoomed geometry (l,t,r,b)", (x, y, x + w, y + h)) + except Exception as exc: + row("state('zoomed')", f"UNSUPPORTED: {exc}") + # Asking for a position past the bottom edge: a window manager that clamps reveals + # the bottom of the usable area by where it puts the window instead. + try: + probe.state("normal") + probe.geometry(f"200x120+40+{sh + 500}") + probe.update_idletasks() + row("asked y=%d, landed at" % (sh + 500), probe.winfo_rooty()) + except Exception as exc: + row("clamp test", f"FAILED: {exc}") + probe.destroy() + + print("\nwhat Flow computes from the above") + row("_work_area()", ui._work_area(sw, sh)) + row("_tk_work_area()", ui._tk_work_area(root, sw, sh)) + full, work = ui._pointer_monitor(sw, sh, root) + row("_pointer_monitor() full", full) + row("_pointer_monitor() work", work) + row("pill would be placed at", + ui.bottom_centre(ui.PILL_W, ui.PILL_H, full, work, ui.PANEL_BOTTOM_OFFSET)) + row("a 420x300 panel at", + ui.bottom_centre(ui.BUBBLE_W, 300, full, work, ui.PANEL_BOTTOM_OFFSET)) + row("hidden panels park at", ui.park_spot(ui.BUBBLE_W, 300, + ui._virtual_desktop(sw, sh))) + + print("\nthe two window attributes the pill depends on") + win = tk.Toplevel(root) + win.geometry("260x60+60+60") + ok = [] + for label, call in ( + ("overrideredirect(True)", lambda: win.overrideredirect(True)), + ("-topmost", lambda: win.attributes("-topmost", True)), + ("-alpha 0.94", lambda: win.attributes("-alpha", 0.94)), + ): + try: + call() + win.update() + row(label, "OK") + except tk.TclError as exc: + row(label, f"FAILED: {exc}") + row("MacWindowStyle plain/noActivates", + "OK" if ui._mac_window_style(win) else "FAILED (or not a Mac)") + + tk.Label(win, text="Any title bar on this? (2s)", bg="#101216", + fg="#e6e8ee").pack(fill="both", expand=True) + win.update() + row("visible for 2s at", (win.winfo_rootx(), win.winfo_rooty())) + win.after(2000, root.destroy) + root.mainloop() + print("\nPaste all of the above back. The line that matters most is whether the") + print("window above had a title bar, and what `zoomed geometry` says.") + print(f"(unused: {ok})" if ok else "") + + +if __name__ == "__main__": + main() diff --git a/scripts/mac_report.py b/scripts/mac_report.py new file mode 100644 index 0000000..d4886f2 --- /dev/null +++ b/scripts/mac_report.py @@ -0,0 +1,268 @@ +"""Everything I need to see about Flow on a Mac, in one PNG. + +Diagnosing this platform from a Windows machine has cost several rounds of "send me a +photo, now send me the log, now run the other probe". This collapses that: it drives the +real windows, screenshots them, renders the numbers beside them, and writes **one image** +carrying both. One file to send back, and the geometry in it is pixel-exact rather than +a phone photo of a screen at an angle. + + uv run --with pillow python scripts/mac_report.py # the image + uv run python scripts/mac_report.py --text # the numbers alone + +`--text` exists because the fastest way to answer a question about this platform has +turned out to be an agent running one command and pasting what it printed. It needs no +Pillow, no Screen Recording grant and no screenshot - just the numbers, which are most +of what the image was carrying anyway. + +Pillow is a `--with`, not a dependency, for `scripts/shots.py`'s reason: it is fetched +into the run and never enters the venv, so R16 still holds at three. + +**macOS will ask for Screen Recording** the first time, because a screenshot of other +windows is what that permission governs. Grant it to the terminal and run again — the +capture comes back black otherwise, which the report says out loud rather than leaving +you to wonder why the picture is empty. + +Three bands, top to bottom: + + **the numbers** - platform, Tk build, what each work-area method answers, where the + stack is therefore placed, and whether the native engine is ready. This is the half + a screenshot cannot carry and a log cannot show. + + **the whole screen** - scaled down, because the question "is it clear of the Dock and + centred" is about where the window sits *in* the display, and a crop of the window + cannot answer it. A neutral backdrop covers the desktop first, `scripts/shots.py`'s + trick and for a second reason here: this image gets sent to somebody, and a full-screen + capture of a working machine carries whatever happened to be open on it. + + **the stack, close up** - at full resolution, because "is there a title bar on it" is + about a 22 px band that a scaled-down screen loses. +""" + +import subprocess +import sys +import tkinter as tk +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) +sys.path.insert(0, str(ROOT / "scripts")) + +import flow.ui as ui # noqa: E402 +from ui_probe import FakeSession # noqa: E402 + +OUT = Path.home() / "flow-mac-report.png" +PAD = 24 +BG = (14, 16, 22) +FG = (230, 232, 238) +DIM = (150, 156, 170) +GOOD = (120, 210, 160) +BAD = (240, 140, 140) + + +def _pil(): + """Imported here, not at the top, so `--text` needs neither Pillow nor a permission. + + The text form is the one an agent on the machine can run and paste back, and asking + it for a screenshot dependency and a Screen Recording grant to print twelve numbers + would be the report getting in the way of being read. + """ + from PIL import Image, ImageDraw, ImageFont + + return Image, ImageDraw, ImageFont + + +def mono(size: int): + """A real monospace if this machine has one, so the numbers line up in columns.""" + _Image, _Draw, ImageFont = _pil() + for path in ("/System/Library/Fonts/Menlo.ttc", + "/System/Library/Fonts/SFNSMono.ttf", + "C:/Windows/Fonts/consola.ttf"): + try: + return ImageFont.truetype(path, size) + except OSError: + continue + return ImageFont.load_default() + + +def facts(pill) -> list[tuple[str, str, tuple]]: + """(label, value, colour). Everything a round trip has had to ask for separately.""" + sw, sh = pill.winfo_screenwidth(), pill.winfo_screenheight() + rows: list[tuple[str, str, tuple]] = [] + + def row(label, value, colour=FG): + rows.append((label, str(value), colour)) + + row("platform", f"{sys.platform} tk {tk.TkVersion}") + row("python", sys.version.split()[0]) + row("screen (as Tk sees it)", f"{sw} x {sh}") + row("", "") + + row("_work_area()", ui._work_area(sw, sh)) + measured = ui._tk_work_area(pill, sw, sh) + whole = (0, 0, sw, sh) + row("_tk_work_area()", measured, + BAD if measured == whole else GOOD) + if measured == whole: + row("", "the maximise probe was refused or ignored;", DIM) + row("", "falling back to the whole screen, so the Dock is not excluded", DIM) + full, work = ui._pointer_monitor(sw, sh, pill) + row("monitor full / work", f"{full} / {work}") + row("", "") + + row("place setting", ui.PLACE) + row("pill placed at", (pill.x, pill.y)) + row("pill size", f"{ui.PILL_W} x {ui.PILL_H}") + bottom_gap = work[3] - (pill.y + ui.PILL_H) + row("gap below the pill", f"{bottom_gap} px to the work-area bottom", + GOOD if 0 <= bottom_gap <= 60 else BAD) + centred = abs((pill.x + pill.pill_w / 2) - (full[0] + full[2]) / 2) + row("off centre by", f"{centred:.0f} px", + GOOD if centred < 4 else BAD) + row("", "") + + row("engine", "macOS on-device speech" if _native_ready()[0] else "whisper") + ok, why = _native_ready() + row("native available", "yes" if ok else f"no - {why}", GOOD if ok else DIM) + row("chord", "none off Windows; hold the pill instead", DIM) + return rows + + +def _native_ready() -> tuple[bool, str]: + try: + from flow.native import available + + return available(compile_if_missing=False, timeout=10.0) + except Exception as exc: # pragma: no cover - a report must not die reporting + return False, f"{type(exc).__name__}: {exc}" + + +def panel(rows, width: int): + """The numbers, as an image, so they travel in the same file as the picture.""" + Image, ImageDraw, _Font = _pil() + font = mono(20) + line = 30 + img = Image.new("RGB", (width, PAD * 2 + line * (len(rows) + 1)), BG) + d = ImageDraw.Draw(img) + d.text((PAD, PAD), "flow - macOS report", font=mono(24), fill=FG) + for i, (label, value, colour) in enumerate(rows, start=1): + y = PAD + line * i + d.text((PAD, y), label, font=font, fill=DIM) + d.text((PAD + 380, y), value, font=font, fill=colour) + return img + + +def grab(): + """The whole screen, and whether it came back black. + + A black capture is what macOS returns before Screen Recording is granted, and it + looks exactly like a bug in Flow. Said out loud instead. + """ + from PIL import ImageGrab + + img = ImageGrab.grab() + extremes = img.convert("L").getextrema() + return img, extremes[1] > 12 + + +def stack_box(pill, ratio: float) -> tuple[int, int, int, int]: + """The pill and whatever is docked to it, in capture pixels, and nothing else. + + Read off the windows rather than guessed at with a margin. The first version + reserved 320 px above the pill for "whatever might be docked" and 60 below it, which + on a real desktop cropped in the taskbar and cut the pill's own bottom edge off - and + the bottom edge is exactly where a title bar or a broken dock seam would show. + """ + top, bottom = pill.y, pill.y + ui.PILL_H + for panel in (pill.bubble, pill.card): + try: + if not panel.winfo_ismapped(): + continue + top = min(top, panel.winfo_rooty()) + bottom = max(bottom, panel.winfo_rooty() + panel.winfo_height()) + except tk.TclError: + continue + edge = 14 + box = (pill.x - edge, top - edge, pill.x + pill.pill_w + edge, bottom + edge) + return tuple(int(v * ratio) for v in box) + + +def backdrop(pill) -> None: + """Cover the desktop, so the report is a picture of Flow and not of the machine. + + `scripts/shots.py` does this to keep its captures clean. Here it is also a privacy + line: the whole point of this file is that the image gets sent to somebody, and a + full-screen grab of a working machine carries every window that happened to be open. + + Topmost and then lowered under the pill, which is the order that file found: a plain + `lift()` is refused and the desktop shows through, so it has to outrank everything + and then step back behind Flow's own windows. + """ + back = tk.Toplevel(pill) + back.overrideredirect(True) + back.configure(bg="#23262b") + back.geometry(f"{pill.winfo_screenwidth()}x{pill.winfo_screenheight()}+0+0") + back.attributes("-topmost", True) + back.update_idletasks() + back.lower(pill) + + +def main() -> None: + text_only = "--text" in sys.argv[1:] + session = FakeSession() + # Lite, which is what `__main__` runs off Windows - `lite = args.lite or + # sys.platform != "win32"`. A report that built a different window from the one the + # app builds would be measuring something nobody uses. + pill = ui.Pill(session, lite=True) + pill.armed = True + if not text_only: + backdrop(pill) + + def report() -> None: + rows = facts(pill) + if text_only: + for label, value, _c in rows: + if label or value: + print(f" {label:<24} {value}") + pill.quit_app() + return + screen, lit = grab() + ratio = screen.width / max(1, pill.winfo_screenwidth()) + + Image, _Draw, _Font = _pil() + head = panel(rows, 1400) + shot = screen.copy() + shot.thumbnail((1400, 900)) + close = screen.crop(stack_box(pill, ratio)) + if close.width > 1400: + close.thumbnail((1400, 10_000)) + + parts = [head, shot, close] + out = Image.new("RGB", (1400, sum(p.height + PAD for p in parts) + PAD), BG) + y = 0 + for part in parts: + out.paste(part, ((1400 - part.width) // 2, y)) + y += part.height + PAD + out.save(OUT) + + for label, value, _c in rows: + if label or value: + print(f" {label:<24} {value}") + print(f"\nwrote {OUT} ({out.width}x{out.height})") + if not lit: + print("\nThe screen capture came back black. macOS needs Screen Recording\n" + "for the terminal: System Settings > Privacy & Security > Screen\n" + "Recording. Grant it, then run this again.") + try: + subprocess.run(["open", "-R", str(OUT)], check=False) + except OSError: + pass + pill.quit_app() + + # Late enough that the bubble has been drawn and placed, which is most of what the + # picture is of. + pill.after(1200, report) + pill.mainloop() + + +if __name__ == "__main__": + main() diff --git a/tests/test_apps.py b/tests/test_apps.py new file mode 100644 index 0000000..d9e45a1 --- /dev/null +++ b/tests/test_apps.py @@ -0,0 +1,271 @@ +"""Per-app notes: a standing instruction that depends on which app is in front. + +The feature is one sentence — *"this text is going into slack.exe, bear it in mind"* — +appended to the rewrite prompt when the profile has something to say about the app that +holds the foreground. Almost none of it is new machinery. `Pill._track_target` already +polls the foreground every frame so Send can be aimed, `inject.classify` already names +the process behind a window because the terminal detection needed it, and `refine()` +already builds its call out of named prompt constants. What this adds is the table, the +lookup, and the two places that could get it wrong. + +Which is where the tests are aimed, because those two are the whole risk: + + **It must not out-shout what the user just said.** A per-app note is a *standing* + preference, and the entire point of speaking an instruction is to override a standing + preference on this one occasion. So the note is phrased as a destination rather than a + rule, and it is placed before the request rather than after it — the sentence nearest + the text is the one that wins ties, and that sentence has to be the user's. + + **It must cost nothing when there is nothing to say.** No profile, no table, an app + with no entry, a blank entry, an entry that is not a string, Lite, a window the OS + will not name: every one of those has to come out as a rewrite that behaves exactly + the way every rewrite behaved before this existed. + +The frame budget gets its own test in `tests/test_lite.py` (`classify` opens a process +handle, and this is polled at 30 fps), because that is where the polling lives. +""" + +import sys +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from flow.profile import Profile, _apps # noqa: E402 +from flow.refine import _POLISH_PROMPT, _PROMPT, Cli, app_note # noqa: E402 +from flow.session import Session # noqa: E402 + + +class Silent: + """The `asr`/`mic` surface `Session` needs and none of these tests exercise.""" + + def __getattr__(self, _name): + return lambda *_a, **_kw: None + + +class TestTheNoteIsBuiltOnlyWhenThereIsSomethingToSay(unittest.TestCase): + """`app_note`, on every shape a hand-written table can produce.""" + + def test_an_app_and_an_instruction_make_a_block(self): + block = app_note("slack.exe", "keep it informal") + self.assertIn("slack.exe", block) + self.assertIn("keep it informal", block) + + def test_it_names_the_destination_rather_than_giving_an_order(self): + # The phrasing is the safeguard, not decoration. "This text is going into Slack" + # is a fact the model weighs against the request; "always be informal" is a + # competing order, and a competing order beats the instruction the user just + # spoke — which would make the feature worse than not having it. + block = app_note("slack.exe", "keep it informal").lower() + self.assertIn("going into", block) + self.assertIn("without letting it override", block) + + def test_every_way_of_having_nothing_to_say_is_the_empty_string(self): + # Each of these is a rewrite that behaves exactly as it did before per-app notes + # existed, which is the only acceptable degraded path for a feature nobody asked + # to have switched on. + for app, note in ( + ("slack.exe", ""), + ("slack.exe", " "), + ("slack.exe", None), + ("slack.exe", 7), + ("slack.exe", ["informal"]), + ("", "keep it informal"), + ): + with self.subTest(app=app, note=note): + self.assertEqual(app_note(app, note), "") + + def test_the_instruction_is_trimmed_but_not_otherwise_touched(self): + # A `lexicon.txt`-shaped bargain: Flow does not reformat what somebody wrote in + # their own file. Whitespace goes because trailing space in JSON is invisible and + # never deliberate; nothing else does. + self.assertIn("Use British spelling.", + app_note("word.exe", " Use British spelling. ")) + + +class TestTheNoteGoesInFrontOfTheRequest(unittest.TestCase): + """Position is the second safeguard, and it is the one a refactor would lose.""" + + def _prompt(self, polish: bool) -> str: + from flow import refine as refine_mod + + seen = {} + + def capture(_cli, prompt, **_kw): + seen["prompt"] = prompt + # A real `Cli`, because `_clean` reads `.name` off whatever comes back to + # decide which CLI's furniture to strip. + return "REVISED", "", Cli("codex", ("codex", "exec")) + + with mock.patch.object(refine_mod, "_invoke_any", capture): + refine_mod.refine("shipping on friday", "make it formal", polish=polish, + app=app_note("slack.exe", "keep it informal")) + return seen["prompt"] + + def test_a_semantic_rewrite_carries_the_note(self): + self.assertIn("slack.exe", self._prompt(polish=False)) + + def test_a_polish_carries_it_too(self): + # The polish ignores the spoken instruction entirely, which makes it the pass + # with the *most* to gain from knowing where the words are headed: a prompt bound + # for a terminal and one bound for a chat window differ in exactly this way. + self.assertIn("slack.exe", self._prompt(polish=True)) + + def test_the_users_own_instruction_sits_nearer_the_text_than_the_note(self): + # The sentence nearest the text wins ties. A per-app note is a standing + # preference and speaking is how you override one on this occasion, so the + # spoken instruction has to be the closer of the two. + prompt = self._prompt(polish=False) + self.assertLess(prompt.index("slack.exe"), prompt.index("make it formal")) + + def test_neither_prompt_mentions_an_app_when_there_is_no_note(self): + # The default path for everybody who never writes an `apps` table, which is + # almost everybody: the prompt has to be byte-for-byte what it always was. + self.assertNotIn("going into", _PROMPT) + self.assertNotIn("going into", _POLISH_PROMPT) + + +class TestTheTableIsCarriedThroughUntouched(unittest.TestCase): + """`profile._apps`, which checks the shape and deliberately stops there.""" + + def test_a_table_survives(self): + self.assertEqual(_apps({"code.exe": "be terse"}), {"code.exe": "be terse"}) + + def test_anything_that_is_not_a_table_degrades_to_nothing(self): + for value in (None, "code.exe", 3, ["code.exe"], True): + with self.subTest(value=value): + self.assertEqual(_apps(value), {}) + + def test_an_entry_is_never_refused_by_name(self): + # The difference from the `hotkeys` table, and it is worth being explicit about. + # An action name has five right answers, so a typo is knowable. An executable + # name has as many right answers as there are programs in the world — an entry + # for an app that is not installed is not a mistake, it is somebody who has not + # opened it yet. So nothing is dropped and nothing is reported; a key that never + # matches simply never fires. + table = {"nothing-like-this.exe": "be terse", "": "x", "7": "y"} + self.assertEqual(_apps(table), table) + + def test_it_survives_a_save_and_a_load(self): + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + p = Profile(path) + p.apps = {"code.exe": "be terse"} + self.assertTrue(p.save()) + self.assertEqual(json.loads(path.read_text(encoding="utf-8"))["apps"], + {"code.exe": "be terse"}) + again = Profile(path) + self.assertTrue(again.load()) + self.assertEqual(again.apps, {"code.exe": "be terse"}) + self.assertNotIn("apps", again.faults) + + def test_an_empty_table_lands_in_every_saved_profile(self): + # The only advertisement this feature gets, in a project with no settings dialog + # to put it in — the same job `"hotkeys": {}` already does. + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + self.assertTrue(Profile(path).save()) + self.assertIn("apps", json.loads(path.read_text(encoding="utf-8"))) + + +class TestTheSessionLooksTheAppUp(unittest.TestCase): + """`Session._app_note`: the join between the window in front and the table.""" + + def session(self, apps=None, app="") -> Session: + s = Session(asr=Silent(), mic=Silent()) + self.addCleanup(s.close) + if apps is not None: + s.profile = type("P", (), {"apps": apps})() + s.target_app = app + return s + + def test_the_app_in_front_selects_its_entry(self): + s = self.session({"code.exe": "be terse", "slack.exe": "be informal"}, + app="slack.exe") + self.assertIn("be informal", s._app_note()) + self.assertNotIn("be terse", s._app_note()) + + def test_the_match_ignores_case(self): + # "Code.exe" and "code.exe" are the same program. A table that cared would be one + # whose entries silently stop matching the day a vendor changes the + # capitalisation of a shipped binary. + for written, running in (("Code.exe", "code.exe"), ("code.exe", "CODE.EXE"), + (" code.exe ", "Code.Exe")): + with self.subTest(written=written, running=running): + s = self.session({written: "be terse"}, app=running) + self.assertIn("be terse", s._app_note()) + + def test_an_app_with_no_entry_gets_nothing(self): + s = self.session({"code.exe": "be terse"}, app="notepad.exe") + self.assertEqual(s._app_note(), "") + + def test_every_way_of_having_no_table_gets_nothing(self): + for apps in ({}, None): + with self.subTest(apps=apps): + self.assertEqual(self.session(apps, app="code.exe")._app_note(), "") + + def test_no_profile_at_all_gets_nothing(self): + # `--no-profile`, and the launch before one has ever been saved. + s = self.session(app="code.exe") + s.profile = None + self.assertEqual(s._app_note(), "") + + def test_no_app_in_front_gets_nothing(self): + # Lite, which has no target-window awareness at all, and the moments when the OS + # declines to name the foreground. + s = self.session({"code.exe": "be terse"}, app="") + self.assertEqual(s._app_note(), "") + + def test_a_key_that_is_not_a_string_cannot_break_the_lookup(self): + # `_apps` carries entries through untouched by design, so this table is + # reachable — and a rewrite that raised because of one bad key would cost the + # user the words they had already spoken. + s = self.session({7: "be terse", "code.exe": "be brief"}, app="code.exe") + self.assertIn("be brief", s._app_note()) + + def test_a_blank_instruction_is_the_same_as_no_entry(self): + # How somebody switches one app off without deleting the line they wrote. + s = self.session({"code.exe": " "}, app="code.exe") + self.assertEqual(s._app_note(), "") + + +class TestItSaysWhenItUsedOne(unittest.TestCase): + """P2: a rewrite that quietly obeyed an invisible rule is one nobody can debug.""" + + def session(self, apps, app) -> Session: + s = Session(asr=Silent(), mic=Silent()) + self.addCleanup(s.close) + s.profile = type("P", (), {"apps": apps})() + s.target_app = app + return s + + def notes(self, s) -> str: + return " | ".join(e.text for e in s.events() if e.kind == "note") + + def test_the_note_is_named_when_it_applies(self): + s = self.session({"slack.exe": "keep it informal"}, "slack.exe") + s.draft.set("shipping on friday") + with mock.patch("flow.session.refine", return_value=("REVISED", "codex")): + s._start_refine("make it formal") + self.assertIn("slack.exe", self.notes(s)) + + def test_and_nothing_is_said_when_none_applies(self): + # The common case by far. A line about a note that did not exist would be noise + # in the one channel that has to stay believable. + s = self.session({"slack.exe": "keep it informal"}, "notepad.exe") + s.draft.set("shipping on friday") + with mock.patch("flow.session.refine", return_value=("REVISED", "codex")): + s._start_refine("make it formal") + self.assertNotIn("note", self.notes(s).replace("notes", "")) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 60942f9..7f20c4f 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -50,6 +50,9 @@ def bubble(text: str = "", **kw): #: `reposition` does arithmetic on it now that the pill's width can dock. b.pill.pill_w = ui.PILL_W b.pill.work = WORK + # The panel band's height comes off the pill now that they share a window, + # so a Mock pill would otherwise answer `panel_h()` with a Mock. + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock( mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None, @@ -97,7 +100,17 @@ def test_the_body_handed_to_the_canvas_is_bounded(self): def test_a_draft_that_fits_is_drawn_whole(self): # The other half, and the one that makes this a window rather than a truncation: # nothing changes for the drafts people actually dictate. - b = bubble(draft(400)) + # + # 400 characters until the panel became a fixed shape (`PANEL_H`). "Fits" is a + # smaller number now — the window no longer grows to whatever the draft asks for, + # so what fits is what fits in 184 px, and more drafts window. The window still + # says so, which is what the class next door asserts. + # + # 40 rather than the ~200 the same panel holds on a real desktop: this fixture + # has no Plex face installed, Tk substitutes, and the substitute measures several + # times taller per line. The number is the fixture's, not the product's — the + # shots in `scripts/shots.py` show three full lines in the same 184 px. + b = bubble(draft(40)) b._render() self.assertEqual(drawn_body(b), b._text) @@ -178,16 +191,28 @@ def test_the_body_cannot_outgrow_a_window_frozen_under_the_hand(self): b.canvas.band(b._text[-40:])[1], b._h - ui.PAD - ui.CHIP_H, f"the body runs past the chip row of a {b._h} px window") - def test_and_takes_the_room_back_when_the_hand_leaves(self): + def test_and_there_is_no_room_to_take_back_any_more(self): + """This asserted the window grew once the hand left. It cannot: `PANEL_H`. + + The freeze was built to stop the window resizing under a hand reaching for a + chip, and a fixed shape makes that unreachable rather than guarded — the stronger + version of the same guarantee. What still catches up is the *content*, which is + what the caller actually wanted to see. + """ b = bubble(draft(300)) b._render() b._pointer_in = True b._text = draft(30_000) b._render() - frozen_h = b._h + frozen_h, frozen_body = b._h, drawn_body(b) b._pointer_in = False b._render() - self.assertGreater(b._h, frozen_h) + self.assertEqual(b._h, frozen_h) + # The body does not move either, and that is not a weaker check than it looks: + # the draft is windowed to its *tail*, so a 300-character draft and a 30 000- + # character one lay out the same last lines. Freezing had one observable effect + # and it was the height. + self.assertEqual(drawn_body(b), frozen_body) def test_five_chips_at_once_stay_inside_the_bubble(self): # Draft held, `can_rescue` true, dictate mode: Refine, Continue, Edit, Was a @@ -220,7 +245,8 @@ def test_a_windowed_draft_says_how_much_is_above_it(self): self.assertRegex(self._elision(b), r"^… \d+ earlier lines$") def test_a_draft_that_fits_says_nothing(self): - b = bubble(draft(400)) + # See the sibling test for why 40 and not 400: a substituted font measures taller. + b = bubble(draft(40)) b._render() self.assertEqual(self._elision(b), "") @@ -291,276 +317,52 @@ def test_this_window_has_no_way_to_show_an_answer_at_all(self): #: The four corners of the work area a pill can be dragged to. The bubble anchors above and #: to the right of the pill, so these are the four directions the anchor can point off. -def corners(pill_w: int = None, pill_h: int = None): - left, top, right, bottom = WORK - pill_w = ui.PILL_W if pill_w is None else pill_w - pill_h = ui.PILL_H if pill_h is None else pill_h - return { - "top-left": (left, top), - "top-right": (right - pill_w, top), - "bottom-left": (left, bottom - pill_h), - "bottom-right": (right - pill_w, bottom - pill_h), - } - - -def geometry_of(b, x: int, y: int) -> str: - """Render with the pill at (x, y) and return the geometry string itself. - - The real `reposition` rather than the fixture's stub, and the string it built rather - than a recomputation of it: a check that re-derives the formula it is checking passes - whatever the formula says. - """ - b.pill.x, b.pill.y = x, y - b.reposition = ui.Bubble.reposition.__get__(b) - box: list[str] = [] - b.geometry = box.append - b._render() - return box[-1] - - -def placed(b, x: int, y: int) -> tuple[int, int, int, int]: - """The window rect `reposition` computes, as (x1, y1, x2, y2).""" - size, _, offset = geometry_of(b, x, y).partition("+") - w, _, h = size.partition("x") - px, _, py = offset.partition("+") - return int(px), int(py), int(px) + int(w), int(py) + int(h) - - -class TestTheWindowIsInsideTheWorkAreaWhereverThePillIs(unittest.TestCase): - """Item 37 bounded the draft's size; nothing bounded the window's placement. - - Measured on a real `tk.Tk` before the fix, with the pill put at each corner of the work - area and the rect read back from `GetWindowRect` as well as from Tk — 12 of 36 - placements left the desktop, all of them on the reply path and all off the **bottom**: - a 4 000-character answer sized the window **1 459 px** and a 12 000-character artifact - **4 179 px**, both pinned at `top + 8` on a 672 px work area, so the chip row landed at - screen y **1 427** and **4 147**. - - Worth saying plainly, because the decision reads the owner's screenshot the other way - round: the **top** edge was never the breach. `max(top + EDGE_AIR, …)` has held it at - every corner in every state. The finding stands exactly as the decision states it — the - bubble leaves the screen by position and takes the chips with it — and the edge it - leaves by is the bottom. +class TestTheBubbleIsABandInThePillsWindow(unittest.TestCase): + """What replaced 39 tests about where this window goes. + + Those tests were right for as long as this was a window: they checked that the + bubble opened above the pill, fell back to below when the pill was at the top of the + screen, stayed inside the work area at every corner, and did all of it byte-for-byte + identically to the frame before. Every one of them was asking the same question — + *are these two windows still touching?* + + There is one window now. The bubble is a `Frame` placed at (0, 0) inside it, and the + pill row is a canvas at its foot. Nothing can come apart, so there is nothing left to + check here — the shell's own geometry is `Pill._sync_shell`'s, and + `test_pill.TestTheShellIsOneWindow` is where it is checked. """ - #: The reply states left this table on 2026-08-03 with `show_reply`. They were the - #: only ones that ever sized this window past the desktop -- a draft is capped at - #: `BODY_MAX_H` -- so what is left is the two item 37 already bounded. The tall-window - #: guarantees they were pinning are asserted on `ConversationCard` now, which is the - #: window that can be that tall. - def states(self): - return [ - ("1k draft", {"_text": draft(1_000)}), - ("50k draft", {"_text": draft(50_000)}), - ] - - def test_every_edge_is_inside_the_work_area_at_every_corner(self): - left, top, right, bottom = WORK - for label, state in self.states(): - for corner, (px, py) in corners().items(): - with self.subTest(state=label, corner=corner): - x1, y1, x2, y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual(y1, top, "the top edge left the work area") - self.assertLessEqual(y2, bottom, "the bottom edge left the work area") - self.assertGreaterEqual(x1, left) - self.assertLessEqual(x2, right) - - def test_the_chip_row_is_inside_it_too(self): - # The property the height bound exists for, and the one a placed-only clamp would - # fake: the row is drawn from `self._h`, so a window bounded without bounding the - # height would put the chips below its own bottom edge and look fixed. - _left, top, _right, bottom = WORK - for label, state in self.states(): - for corner, (px, py) in corners().items(): - with self.subTest(state=label, corner=corner): - b = bubble(**state) - _x1, y1, _x2, _y2 = placed(b, px, py) - chip_top = y1 + b._h - ui.PAD - ui.CHIP_H - chip_bottom = y1 + b._h - ui.PAD - self.assertGreaterEqual(chip_top, top) - self.assertLessEqual(chip_bottom, bottom, "the chips are off screen") - - def test_the_longest_draft_still_does_not_size_the_window_past_the_desktop(self): - b = bubble(draft(50_000)) + def band(self, b): + put: list[dict] = [] + b.reposition = ui.Bubble.reposition.__get__(b) + b.place = lambda **kw: put.append(kw) + b.place_forget = lambda: put.append({}) b._render() - self.assertLessEqual(b._h, WORK[3] - WORK[1] - 2 * ui.EDGE_AIR) + return put[-1] - def test_and_a_short_one_still_sizes_the_window_to_itself(self): - # The other direction, so the bound cannot pass this by firing for everything. - b = bubble(draft(200)) - b._render() - self.assertLess(b._h, WORK[3] - WORK[1] - 2 * ui.EDGE_AIR) + def test_it_takes_the_full_width_at_the_top_of_the_window(self): + band = self.band(bubble(draft(1_000))) + self.assertEqual((band["x"], band["y"], band["width"]), (0, 0, ui.BUBBLE_W)) + self.assertLessEqual(band["height"], ui.PANEL_MAX_H) - def test_the_air_is_one_number_and_both_places_use_it(self): - # `EDGE_AIR` is what makes the clamp a proof rather than a best effort — the height - # is fitted to `work - 2 * air` and the position is clamped by `air`, and the two - # have to be the same number. A literal in either place is how they drift apart. - # - # Asserted against `reposition` directly rather than through a state that happens to - # fill the desktop: item 45 gave the reply a head window, so nothing renders to - # exactly `work_h` any more and a check that relied on one would have been pinning a - # coincidence. - _left, top, right, bottom = WORK - b = bubble() - b._h = bottom - top # taller than the fit allows, which is what a clamp is for - box: list[str] = [] - b.geometry = box.append - b.pill.x, b.pill.y = corners()["bottom-right"] - ui.Bubble.reposition(b) - self.assertEqual(box[-1].partition("+")[2], - f"{right - ui.BUBBLE_W - ui.EDGE_AIR}+{top + ui.EDGE_AIR}") - b._h = bottom - top - 2 * ui.EDGE_AIR # exactly the fit - ui.Bubble.reposition(b) - _size, _, offset = box[-1].partition("+") - self.assertEqual(int(offset.partition("+")[2]) + b._h, bottom - ui.EDGE_AIR) - - -#: Three x positions along the top edge of the work area — the pill dragged where there is -#: no "above" left. Left, middle and right, because the anchor is horizontal as well as -#: vertical and a fallback that only worked in one corner would pass a single-point check. -def along_the_top() -> dict[str, tuple[int, int]]: - left, top, right, _bottom = WORK - return { - "top-left": (left, top), - "top-middle": ((left + right - ui.PILL_W) // 2, top), - "top-right": (right - ui.PILL_W, top), - } - - -#: Every geometry string `reposition` produced **before** item 44, captured by running the -#: harness against the tree as it stood. This is the regression half and it is a table rather -#: than a formula on purpose: a check that recomputes what it is checking cannot fail. -#: -#: The rows absent from it are the ones the fallback is *for* — a draft-sized window with the -#: pill along the top, where "above" has no room. Everything else must come through byte for -#: byte, including the reply-sized windows at the top, which are taller than either side of -#: the pill and so keep today's clamp. -#: -#: **The reply rows are gone, and that is the second time they moved rather than the first -#: time they were rewritten.** Item 45 re-captured them at 643 px where the full-text probe -#: had sized them 656; item 63 removed the path, because this window no longer draws an -#: answer. The draft rows below are byte-identical to the day they were captured, which is -#: the whole point of a table: one that gets quietly re-baselined pins nothing. -#: Re-baselined 2026-08-09 for the IBM Plex Sans migration: `FONT_BODY` reports an -#: 18 px line to the real canvas against Segoe UI's 17, so the same capped draft lays -#: out one pixel taller (414 → 415) and the bottom-anchored placements ride up one -#: pixel to match (208 → 207, the same bottom edge). Traced to the font swap, not a -#: silent re-pin. -#: -#: Re-baselined again the same day for docking (Phase 5): the pill and this window -#: meet at one hairline seam now, not the 10 px of air a shadow used to go in, so -#: every placement that resolved to "above" moves ten pixels closer to the pill — -#: 207 → 217 here. "mid-left" is unaffected because that placement was already -#: resolving to a *different* branch of `reposition`'s clamp, one the gap never -#: reached. -#: -#: Re-baselined a third time the same day: `BUBBLE_W` moved to 420 (Phase 6, the -#: two panels unified at the draft's own widest state). Width and the right-anchored -#: x shift with it everywhere (380→420, 892→852). The 1 000-character draft's height -#: drops too (415→398) — wider text wraps to fewer lines for the same character -#: count — while the 50 000-character one holds at 415, because that row is capped -#: by `BODY_MAX_H` rather than by how the text wraps, and a cap does not move with -#: the column it bounds. -GEOMETRY_BEFORE = { - ("1k draft", "bottom-left"): "420x398+8+234", - ("1k draft", "bottom-right"): "420x398+852+234", - ("1k draft", "mid-left"): "420x398+8+8", - ("50k draft", "bottom-left"): "420x415+8+217", - ("50k draft", "bottom-right"): "420x415+852+217", - ("50k draft", "mid-left"): "420x415+8+8", -} - - -class TestTheBubbleOpensBelowWhenAboveHasNoRoom(unittest.TestCase): - """A fallback, not a mode — tooltip behaviour, and item 42's desk check found the need. - - With the pill dragged to the top of the work area there is no "above" left, so the - bubble clamped to the top edge and was drawn **over the pill it is anchored to**. - Nothing clipped and nothing was unreachable — item 42 guarantees that and this must not - take it away — but an anchor pointing at something it covers is not an anchor. - - Above is tried first and used whenever it fits. Below is used only when above does not - fit *and* below does. When **neither** fits — a window as tall as the desktop, which is - what a full reply is — the arithmetic is today's exactly and the bubble clamps to the top - over the pill. That case is not fixed here, deliberately: no anchor can place a window - taller than the space either side of it, and pretending otherwise would be a third rule. - """ + def test_a_draft_past_the_ceiling_stops_at_the_ceiling(self): + # The band is snug around its content again, so this is no longer "every draft + # gets the same band" — it is "no draft gets more than the desktop has left". + # A 1k draft and a 50k one both overflow, so both stop in the same place. + self.assertEqual(self.band(bubble(draft(1_000)))["height"], + self.band(bubble(draft(50_000)))["height"]) - #: The reply states left this table on 2026-08-03 with `show_reply`. They were the - #: only ones that ever sized this window past the desktop -- a draft is capped at - #: `BODY_MAX_H` -- so what is left is the two item 37 already bounded. The tall-window - #: guarantees they were pinning are asserted on `ConversationCard` now, which is the - #: window that can be that tall. - def states(self): - return [ - ("1k draft", {"_text": draft(1_000)}), - ("50k draft", {"_text": draft(50_000)}), - ] - - def test_a_pill_along_the_top_opens_the_bubble_below_it(self): - # The defect, stated as geometry: the bubble's top must not be above the pill's - # bottom. Red at all three positions before this item, where it sat at y=8 with the - # pill occupying y=0..40. - _left, top, _right, _bottom = WORK - for label, state in self.states(): - for name, (px, py) in along_the_top().items(): - with self.subTest(state=label, at=name): - _x1, y1, _x2, _y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual( - y1, py + ui.PILL_H, - "the bubble is drawn over the pill it is anchored to", - ) - - def test_every_other_placement_is_byte_identical(self): - for (label, name), before in GEOMETRY_BEFORE.items(): - state = dict(self.states())[label] - places = dict(along_the_top()) - places.update(corners()) - places["mid-left"] = (WORK[0], (WORK[1] + WORK[3]) // 2) - with self.subTest(state=label, at=name): - self.assertEqual(geometry_of(bubble(**state), *places[name]), before) - - def test_above_is_still_the_default_wherever_it_fits(self): - # The other direction. A pill in its usual place has room above it, and the bubble - # must still be there — a fallback that fired whenever it could would be a mode. - b = bubble(_text=draft(1_000)) - _x1, y1, _x2, y2 = placed(b, *corners()["bottom-right"]) - self.assertLessEqual(y2, WORK[3] - ui.PILL_H, - "the bubble should sit above the pill, not below it") - - def test_when_neither_side_fits_the_clamp_is_todays(self): - # A window as tall as the desktop has no room on either side of a pill. This is - # the case the fallback deliberately does not fix, and it is pinned so nobody - # reads its absence as an oversight. Driven against `reposition` directly now: - # the state that used to produce a desktop-tall bubble was a full reply, and this - # window has not drawn one since item 63. The card is where that height lives. - _left, top, _right, bottom = WORK - for name, (px, py) in along_the_top().items(): - with self.subTest(at=name): - b = bubble(draft(1_000)) - b._h = bottom - top - 2 * ui.EDGE_AIR - b.pill.x, b.pill.y = px, py - box: list[str] = [] - b.geometry = box.append - ui.Bubble.reposition(b) - self.assertEqual(int(box[-1].rpartition("+")[2]), top + ui.EDGE_AIR) - - def test_the_work_area_guarantee_survives_the_second_anchor(self): - # Item 42's property, re-asserted against the new placements: a second way to - # choose y is a second way to leave the desktop. - left, top, right, bottom = WORK - for label, state in self.states(): - for name, (px, py) in along_the_top().items(): - with self.subTest(state=label, at=name): - x1, y1, x2, y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual(y1, top) - self.assertLessEqual(y2, bottom) - self.assertGreaterEqual(x1, left) - self.assertLessEqual(x2, right) + def test_a_short_draft_takes_less_than_a_long_one(self): + # What the fixed height cost and this gets back: no empty space inside the + # window. FluidVoice's overlay is snug around two lines and then three; a demo + # of it, read frame by frame, is why this changed back. + self.assertLess(self.band(bubble("a note"))["height"], + self.band(bubble(draft(50_000)))["height"]) - -if __name__ == "__main__": - unittest.main() + def test_a_hidden_bubble_gives_its_band_back(self): + # `place_forget` rather than parking a window offscreen: there is no window to + # park, and the pill's shell shrinks to the row on the next `_sync_shell`. + self.assertEqual(self.band(bubble(draft(400), _visible=False)), {}) class TestTheChipsSurviveARedraw(unittest.TestCase): @@ -645,7 +447,14 @@ def test_leaving_catches_everything_up(self): b._render() frozen = b._h b._leave() - self.assertNotEqual(b._h, frozen, "the window never caught up") + # The window is a fixed shape now (`PANEL_H`), so "caught up" cannot mean "is a + # different size" any more. What it means is that the note held back while the + # hand was here is on the canvas once it has gone — which is the thing anybody + # cared about, and was only ever inferred from the height. + self.assertEqual(b._h, frozen) + self.assertTrue(any("microphone overflowed" in i["text"] + for i in b.canvas.items if "text" in i), + "the note never caught up") def test_a_countdown_does_not_resize_its_own_chip(self): # Chip width followed the label, so `Ask` -> `Ask 4s` -> `Ask` moved the hit @@ -696,3 +505,91 @@ def test_the_row_is_drawn_above_the_body_it_outlived(self): self.assertIn('tag_raise("chips")', inspect.getsource(ui.Bubble._render)) self.assertIn('tag_raise("chips")', inspect.getsource(ui.ConversationCard._render)) + +class TestTheSettingsStrip(unittest.TestCase): + """The controls worth reaching without a right-click. + + Asked for by name — "Dictate and Converse for sure Then workspace and voices" — and + the split between them is the design: mode is a *control*, because it changes what + Send does and is switched mid-task; workspace and voice are *values*, whose worth is + being visible, and which open the menu that already exists rather than growing a + second implementation of the same list. + """ + + def strip(self, **session): + canvas = MeasuringCanvas() + pill = mock.Mock() + pill.session = mock.Mock(**{"mode": ui.DICTATE, "workspace": "", + "speaker": None, "muted": False, **session}) + took = ui._settings_row(canvas, pill, ui.BUBBLE_W, ui.PAD) + texts = [i["text"] for i in canvas.items if i.get("text")] + return took, texts, canvas, pill + + def test_the_mode_is_named_and_is_a_chip(self): + _took, texts, _c, _p = self.strip() + self.assertTrue(any(t.startswith("Dictate") for t in texts), texts) + + def test_converse_says_converse(self): + _took, texts, _c, _p = self.strip(mode=ui.CONVERSE) + self.assertTrue(any(t.startswith("Converse") for t in texts), texts) + + def test_clicking_the_mode_switches_it(self): + _took, _texts, canvas, pill = self.strip() + fire = next(fn for tag, seq, fn in canvas.bindings + if tag == "settings-mode" and seq == "") + fire(None) + pill.session.toggle_mode.assert_called_once() + + def test_a_value_opens_the_menu_that_already_exists(self): + # Rather than a second implementation of the same list: the workspace recents + # and the voice list are both built with a tick showing the current choice, and + # two of anything is two things to keep in step. + _took, _texts, canvas, pill = self.strip(workspace=r"D:/dev/products/flow") + fire = next(fn for tag, seq, fn in canvas.bindings + if tag == "settings-workshop" and seq == "") + fire(None) + pill._menu_workspace.assert_called_once() + + def test_the_workspace_is_shown_by_its_last_component(self): + # The tail of a path names the project; the head is the part everybody shares. + for path in ("D:/dev/products/flow", "D:" + chr(92) + "dev" + chr(92) + "flow", + "/home/sam/flow"): + with self.subTest(path=path): + _took, texts, _c, _p = self.strip(workspace=path) + self.assertIn("workshop: flow", texts) + + def test_a_workspace_that_is_not_set_takes_no_room(self): + _took, texts, _c, _p = self.strip(workspace="") + self.assertFalse([t for t in texts if t.startswith("workshop")]) + + def test_the_voice_is_absent_when_nothing_speaks(self): + _took, texts, _c, _p = self.strip(speaker=None) + self.assertFalse([t for t in texts if t.startswith("voice")]) + + def test_and_says_muted_when_it_is(self): + _took, texts, _c, _p = self.strip(speaker=mock.Mock(name="x"), muted=True) + self.assertIn("voice: muted", texts) + + def test_it_takes_the_height_the_layout_reserved_for_it(self): + # `around` in `_render` counts `SETTINGS_H`; a strip that took more would be + # drawn over the first line of the draft. + took, _texts, _c, _p = self.strip() + self.assertEqual(took, ui.SETTINGS_H) + + def test_a_pill_with_no_session_draws_nothing(self): + # Fixtures build pills with `__new__`, and a strip that assumed a session would + # take the whole surface down with it. + canvas = MeasuringCanvas() + self.assertEqual(ui._settings_row(canvas, mock.Mock(session=None), + ui.BUBBLE_W, ui.PAD), 0) + + def test_the_ceiling_grew_by_the_strip_rather_than_the_draft_paying_for_it(self): + """The strip is furniture, not content. + + Taking it out of the content's share is what the editor tests caught: the live + partial's own cap is a flat 70 px, and on a panel pegged at the old ceiling it + ran straight through the note and the chip row because everything above it had + grown by 22 px and it had not been told. + """ + self.assertEqual(ui.PANEL_MAX_H, 184 + ui.SETTINGS_H) + diff --git a/tests/test_card.py b/tests/test_card.py index 4adb9f6..3e9df7b 100644 --- a/tests/test_card.py +++ b/tests/test_card.py @@ -44,6 +44,9 @@ def card(**kw): #: `reposition` does arithmetic on it now that the pill's width can dock. c.pill.pill_w = ui.PILL_W c.pill.work = WORK + # The panel band's height comes off the pill now that they share a window, + # so a Mock pill would otherwise answer `panel_h()` with a Mock. + c.pill.band_h = lambda: ui.PANEL_MAX_H c.pill.x, c.pill.y = 900, 560 c.pill.session = mock.Mock(can_take_reply=True, auto_ask_in=None) c.canvas = MeasuringCanvas() @@ -55,11 +58,12 @@ def card(**kw): c._h = ui.CARD_MIN_H c._pinned_h = 0 c._countdown = None + #: What `reposition` asked for. It used to be a `geometry` string, because this was + #: a window; it is a `place` call now, because the card is a band inside the pill's + #: one window. The record is kept because tests read the height back out of it. c.placed = [] - c.geometry = c.placed.append - c.deiconify = lambda: None - c.attributes = lambda *a, **kw: None - c.withdraw = lambda: None + c.place = lambda **kw: c.placed.append(f"{kw['width']}x{kw['height']}+0+0") + c.place_forget = lambda: None for name, value in kw.items(): setattr(c, name, value) return c @@ -452,7 +456,17 @@ def test_the_count_grows_with_the_answer(self): for n in (6_000, 12_000)] self.assertLess(counts[0], counts[1]) - def test_the_answer_still_sizes_the_card(self): + def test_the_answer_sizes_the_card_again(self): + """It did; then it did not for two commits; now it does, and that is right. + + Fixing the height stopped the card moving and left a hole in it instead. A + FluidVoice demo read frame by frame settled the argument: that overlay's bottom + edge is at y=554 in every frame while the box is snug around two lines, then + three. Snug is what a reader wants; what must not move is the *foot*, and + `Pill._sync_shell` grows the shell upward so the pill row never does. + + Stepping by a whole body line (`_settled_h`) is what keeps it from thrashing. + """ self.assertGreater(self.answered(prose(4_000))._h, self.answered(prose(200))._h) @@ -713,7 +727,9 @@ def test_switching_back_finds_the_answer_waiting(self): self.assertEqual(c._answer, "you add it with a migration") c.deiconify.assert_not_called() c.show() - c.deiconify.assert_called_once() + # It used to `deiconify` a window of its own. There is one window now, and a + # band that has been given a place in it is a band that is showing. + self.assertTrue(c.placed) @staticmethod def raised(call) -> bool: diff --git a/tests/test_chord.py b/tests/test_chord.py new file mode 100644 index 0000000..4eb6eb4 --- /dev/null +++ b/tests/test_chord.py @@ -0,0 +1,680 @@ +"""The modifier-only chord: hold ctrl+win to dictate, let go to send. + +Push-to-talk. The press-down warms the models and opens the microphone, the hold is the +utterance, and the release stops capture and sends what was said. It used to be a +toggle that fired one word at the release, and the three moments here are the change. + + +`RegisterHotKey` cannot express this, which is the whole reason the code under test +exists — it takes a virtual key and there is no VK for "nothing". So the chord runs on a +`WH_KEYBOARD_LL` hook, and that is a decision (R16 said no global hooks) narrowed rather +than reversed. Three properties are what the narrowing rests on, and each is asserted +here rather than left to the comment that claims it: + + **It never learns which key you pressed.** A key outside the chord sets a boolean. The + suite proves the shape of that by driving keys through and checking that what changes + is *whether* the chord fires, never a record of what was typed — see + `TestDItLearnsNothingAboutTheKeysItRejects`. + + **It never swallows anything.** Every event reaches `CallNextHookEx`, including the one + that fires the chord. Ctrl and Win have real jobs and Flow does not get to keep them. + + **It sends nothing when Windows meant something else.** ctrl+win is a *prefix* in + Windows itself — ctrl+win+d makes a virtual desktop, ctrl+win+left and +right switch + between them. Every one presses a third key, and under push-to-talk that third key + *stops* a capture rather than merely declining to start one, because the press-down + already opened the microphone. `TestCWindowsOwnsCtrlWinToo` states what this costs and + what survives it; the promise that survives is that no desktop switch ever pastes. + +The hook is never installed. `Chord._on_key` is the callback the OS would call, so the +suite calls it directly with the structure Windows would pass — which tests the state +machine that is actually hard, and asks nothing of the developer's keyboard. +""" + +import ctypes +import sys +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +# Windows-only, and the guard is `tests/test_hotkey.py`'s word for word: `flow.hotkey` +# calls `ctypes.WinDLL("user32", use_last_error=True)` at module scope, so the failure +# would be the `import` below rather than anything a decorator could reach. +if sys.platform != "win32": # pragma: no cover - the CI legs that are not Windows + raise unittest.SkipTest("Windows-only: flow.hotkey binds user32 at import") + +import queue # noqa: E402 + +import flow.hotkey as hotkey # noqa: E402 +from flow.hotkey import ( # noqa: E402 + CHORD_NAMES, + VK_LCONTROL, + VK_LMENU, + VK_LSHIFT, + VK_LWIN, + VK_RCONTROL, + VK_RWIN, + WM_KEYDOWN, + WM_KEYUP, + WM_SYSKEYDOWN, + WM_SYSKEYUP, + Chord, + describe_chord, + parse_chord, +) +from flow.profile import CHORD_DEFAULT # noqa: E402 + +VK_D, VK_LEFT, VK_A = 0x44, 0x25, 0x41 + + +class _Keyboard: + """Drives `Chord._on_key` the way the OS would, and remembers what was passed on. + + `CallNextHookEx` is stubbed rather than called for real, for one reason and one + convenience. The reason: what this suite is checking is that *every* event reaches + it, and a real call returns a number that says nothing about whether it happened. The + convenience: the callback runs with `self._hook` still None, because nothing here + installs a hook, and a NULL hook handle is exactly the argument a stub should not + have to care about. + """ + + def __init__(self, chord): + self.chord = chord + self.passed = [] + + def _event(self, message, vk): + # The real `KBDLLHOOKSTRUCT`, by address, because `_on_key` casts the LPARAM and + # a fake that skipped the cast would not exercise the line most likely to be + # wrong on a 64-bit build. + block = hotkey._KBDLLHOOKSTRUCT(vkCode=vk, scanCode=0, flags=0, time=0, + dwExtraInfo=None) + # Held for the duration of the call: a struct that went out of scope here would + # be freed under the pointer the callback is about to read. + self._block = block + with mock.patch.object(hotkey, "user32") as fake: + fake.CallNextHookEx.return_value = 0 + self.chord._on_key(0, message, ctypes.addressof(block)) + self.passed.append((message, vk, fake.CallNextHookEx.called)) + + def down(self, *vks): + for vk in vks: + self._event(WM_KEYDOWN, vk) + return self + + def up(self, *vks): + for vk in vks: + self._event(WM_KEYUP, vk) + return self + + def sys_down(self, *vks): + for vk in vks: + self._event(WM_SYSKEYDOWN, vk) + return self + + def sys_up(self, *vks): + for vk in vks: + self._event(WM_SYSKEYUP, vk) + return self + + +def _chord(mods=("ctrl", "win")): + presses = queue.Queue() + return Chord(presses, frozenset(mods)), presses + + +def _drained(presses): + out = [] + while not presses.empty(): + out.append(presses.get_nowait()) + return out + + +#: One clean hold, start to finish, as the queue sees it. +HOLD = ["warm", "talk", "talk-end"] + + +def _fired(presses): + """Everything one gesture put on the queue, in order. + + Named `_fired` still, but it no longer means "the one word a release emits" — under + push-to-talk a hold is three moments, and the order between them *is* the feature. + Filtering any of them out would hide the two things most worth asserting: that the + warm arrives before the capture rather than with it, and that a hold Windows took + over is stopped on the third key rather than at the release. + """ + return _drained(presses) + + +class TestAWhatAChordMeansIsReadOffWhatSomebodyTyped(unittest.TestCase): + """`parse_chord`, on the string the guide prints and the shapes a hand-edit makes.""" + + def test_the_shipped_chord_parses_to_the_two_modifiers_it_names(self): + self.assertEqual(parse_chord("ctrl+win"), (frozenset({"ctrl", "win"}), "")) + + def test_the_shipped_default_is_a_chord_this_parser_accepts(self): + # The one assertion that would catch the two files disagreeing. `CHORD_DEFAULT` + # lives in `flow/profile.py` because that module imports on a Mac and + # `flow.hotkey` cannot — a split that buys platform reach and costs exactly this + # risk, so the risk is bought back here. + mods, reason = parse_chord(CHORD_DEFAULT) + self.assertIsNotNone(mods, reason) + + def test_case_and_spacing_are_the_writers_business(self): + # Same rule as `parse`, and for the same reason: this is JSON somebody typed by + # hand, and a setting that depends on where the spaces went is a bug report + # waiting to be filed. + canonical, _reason = parse_chord("ctrl+win") + for text in ("CTRL+WIN", "Ctrl+Win", " ctrl + win ", "cTrL+wIn", + "ctrl+win\n", "win+ctrl"): + with self.subTest(text=text): + self.assertEqual(parse_chord(text)[0], canonical) + + def test_one_modifier_is_refused_because_a_bare_tap_is_a_thing_hands_do(self): + # The refusal worth explaining. A single-modifier "chord" fires every time that + # key is tapped and released cleanly, and a bare Ctrl tap is a thing hands do + # constantly while thinking — so it would be a dictation app that starts + # recording at random, from a setting nobody would connect to the symptom. + for text in ("ctrl", "win", "shift", "ctrl+ctrl", "ctrl + CTRL"): + with self.subTest(text=text): + mods, reason = parse_chord(text) + self.assertIsNone(mods) + self.assertIn("two modifiers", reason) + + def test_four_modifiers_are_refused_because_that_is_not_a_shape(self): + mods, reason = parse_chord("ctrl+alt+shift+win") + self.assertIsNone(mods) + self.assertIn("cannot be held", reason) + + def test_a_key_is_not_a_modifier_and_is_named_as_the_thing_that_was_wrong(self): + # The likeliest hand-edit by far: somebody reads "chord" and writes the combo + # they already know. The reason has to point at the word that broke it, because + # the fix is deleting that word and nothing else. + mods, reason = parse_chord("ctrl+win+space") + self.assertIsNone(mods) + self.assertIn("space", reason) + self.assertIn("not a modifier", reason) + + def test_the_shapes_that_are_not_strings_at_all(self): + for value in (None, 3, ["ctrl", "win"], {"ctrl": "win"}, True): + with self.subTest(value=value): + self.assertEqual(parse_chord(value), (None, "not a string")) + + def test_an_empty_string_is_refused_here_and_meant_off_at_the_call_site(self): + # `_chord` in `flow/__main__.py` never reaches the parser with a blank, because + # blank is how somebody turns the chord off in the file. Refused rather than + # accepted anyway, so that the two places cannot drift into disagreeing about + # what an empty setting means. + self.assertEqual(parse_chord("")[1], "empty") + self.assertEqual(parse_chord(" + ")[1], "empty") + + def test_every_chord_that_parses_can_be_written_back_out(self): + # The round trip that keeps the startup line honest: the block says `chord + # toggle ctrl+win`, and a chord that could be asked for but not named would put + # something else there. + for a in CHORD_NAMES: + for b in CHORD_NAMES: + if a == b: + continue + with self.subTest(chord=f"{a}+{b}"): + mods, _ = parse_chord(f"{a}+{b}") + self.assertEqual(parse_chord(describe_chord(mods))[0], mods) + + def test_the_report_spells_a_chord_the_same_way_round_every_time(self): + # "win+ctrl" and "ctrl+win" are the same chord, and the startup block has to + # call them the same thing or the line stops being something to compare against + # the guide. + self.assertEqual(describe_chord(parse_chord("win+ctrl")[0]), "ctrl+win") + + +class TestBTheHoldIsTheUtterance(unittest.TestCase): + """The whole feature: press both, speak while they are down, let go to send. + + Three moments, and the order between them is the point. The press-down warms the + models and opens the microphone; the release stops it and sends what was said. The + old gesture put one word on this queue at the release and nothing at the press, and + the difference is a reload that used to land inside the user's first sentence. + """ + + def test_a_clean_hold_warms_then_captures_then_sends(self): + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_the_warm_lands_before_the_capture_and_not_beside_it(self): + # The whole reason the warm exists. A warm that arrived with `talk` would be the + # preload `Session.start` already does, and the hold would still be spent + # waiting. Asserted at the press, before any release exists. + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), ["warm", "talk"]) + + def test_nothing_starts_until_the_chord_is_complete(self): + # One modifier is not a hold. A bare Ctrl tap is a thing hands do constantly + # while thinking, and opening a microphone on it would be the defect + # `parse_chord` refuses single-modifier chords to avoid. + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_it_ends_on_the_first_release_and_not_again_on_the_second(self): + # The reason `_armed` is latched rather than recomputed. Two modifiers go up as + # two events, and a chord that asked "are they all up now?" would either end + # twice — sending the same utterance into the window twice — or need the + # releases in a particular order. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.up(VK_LWIN) + self.assertEqual(_fired(presses), HOLD) + keys.up(VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_the_order_the_two_go_down_in_does_not_matter(self): + for first, second in ((VK_LCONTROL, VK_LWIN), (VK_LWIN, VK_LCONTROL)): + with self.subTest(first=first): + chord, presses = _chord() + _Keyboard(chord).down(first, second).up(first, second) + self.assertEqual(_fired(presses), HOLD) + + def test_a_repeated_keydown_does_not_reopen_the_microphone(self): + # The OS repeats a held key in some configurations, and `_armed` is what stops a + # second `talk` arriving mid-utterance — which the UI could not tell apart from + # the user having spoken into a microphone reopened under them. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LCONTROL, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk"]) + + def test_holding_it_three_times_is_three_utterances(self): + # Obvious, and the one that would catch a flag that latches on and never clears. + chord, presses = _chord() + keys = _Keyboard(chord) + for _ in range(3): + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD * 3) + + def test_either_side_of_the_keyboard_works(self): + chord, presses = _chord() + _Keyboard(chord).down(VK_RCONTROL, VK_RWIN).up(VK_RWIN, VK_RCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_a_generic_modifier_code_works_because_injected_keys_carry_one(self): + # A physical press arrives sided (`VK_LCONTROL`); a synthesised one may carry the + # generic `VK_CONTROL`. `flow/inject.py` synthesises keys itself, so a hook that + # listened for only one of the two would behave differently depending on whether + # a human or a program pressed the chord. + chord, presses = _chord() + _Keyboard(chord).down(hotkey.VK_CONTROL, VK_LWIN).up(VK_LWIN, hotkey.VK_CONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_the_alt_bearing_chords_arrive_as_sys_keys_and_still_work(self): + # Windows sends WM_SYSKEYDOWN rather than WM_KEYDOWN while Alt is held. A chord + # containing alt that only watched the plain messages would never fire at all. + chord, presses = _chord(("ctrl", "alt")) + _Keyboard(chord).sys_down(VK_LCONTROL, VK_LMENU).sys_up(VK_LMENU, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_it_puts_the_words_it_was_built_with(self): + # The chord is a second way in, not a second thing to handle: it writes into + # `Hotkeys.presses`, so everything downstream drains one stream and cannot tell a + # chord from a registered combo. + presses = queue.Queue() + chord = Chord(presses, frozenset({"ctrl", "win"}), action="go", + warm_action="heat", end_action="stop", break_action="drop") + keys = _Keyboard(chord) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["heat", "go", "stop"]) + keys.down(VK_LCONTROL, VK_LWIN).down(VK_D).up(VK_D, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["heat", "go", "drop"]) + + def test_the_default_words_are_the_ones_the_ui_dispatches_on(self): + # A tripwire on four strings that live in two files: `Pill._frame` matches each + # by literal, and a rename here that missed it would land as a chord that opens + # a microphone nothing ever closes. + chord, _presses = _chord() + ui = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + for word in (chord.warm_action, chord.action, + chord.end_action, chord.break_action): + with self.subTest(word=word): + self.assertIn('name == "%s"' % word, ui) + + +class TestBBBothGesturesShipAndNeitherReplacesTheOther(unittest.TestCase): + """`Chord.gesture`, and the switch that should have been there from the start. + + Shipping push-to-talk *instead of* the toggle took a working gesture away from + everybody who had it. They are good at different things: a hold needs no decision + about when you are finished and cannot leave a microphone running, and a toggle is + the only one of the two that survives a paragraph, a long thought with pauses in it, + or hands that cannot hold two keys down for a minute. + """ + + def toggler(self): + chord, presses = _chord() + chord.gesture = "toggle" + return chord, presses + + def test_hold_is_what_ships(self): + self.assertEqual(hotkey.GESTURE_DEFAULT, "hold") + self.assertEqual(_chord()[0].gesture, "hold") + + def test_a_toggle_chord_fires_one_word_on_a_clean_release(self): + chord, presses = self.toggler() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["toggle"]) + + def test_a_toggle_chord_does_nothing_at_all_on_the_press(self): + # It has no press-down half. Warming here would load 605 MB of models every time + # somebody reached for `ctrl+win+arrow`, which is the cost the hold gesture + # accepts on purpose and this one has no reason to. + chord, presses = self.toggler() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), []) + + def test_a_toggle_chord_still_refuses_what_windows_meant(self): + # The original rule, unchanged and still doing its job: `ctrl+win+d` makes a + # desktop and starts nothing. Under `hold` this is a break; here it is silence. + chord, presses = self.toggler() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_D).up(VK_D).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_it_never_emits_a_word_the_other_gesture_owns(self): + # The two vocabularies are disjoint, which is what lets one dispatch table serve + # both without a mode flag at the far end. + chord, presses = self.toggler() + keys = _Keyboard(chord) + for _ in range(3): + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(set(_fired(presses)) & set(HOLD), set()) + + def test_switching_gesture_is_one_assignment_and_takes_effect_at_once(self): + # The reason `gesture` is a plain attribute the callback reads rather than + # something baked in at construction: switching by rebuilding would mean + # unhooking and re-installing a `WH_KEYBOARD_LL` hook, which the OS may refuse — + # and being refused *while changing a setting* leaves somebody with no chord. + chord, presses = _chord() + keys = _Keyboard(chord) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + chord.gesture = "toggle" + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["toggle"]) + + def test_an_unknown_gesture_falls_back_rather_than_disabling_the_chord(self): + # It arrives from a hand-edited profile. A typo must cost the setting, not the + # shortcut — a chord that silently did nothing would be unattributable. + for name in ("Hold", "push-to-talk", "", None, 7): + with self.subTest(name=name): + presses = queue.Queue() + chord = Chord(presses, frozenset({"ctrl", "win"}), gesture=name) + self.assertEqual(chord.gesture, hotkey.GESTURE_DEFAULT) + + def test_the_menu_offers_exactly_the_gestures_that_exist(self): + # `flow/ui.py` cannot import `flow.hotkey` — that module binds user32 at import — + # so the menu keeps its own labels. This is the assertion that stops the two + # lists drifting into a row that selects a gesture the hook does not know. + import flow.ui as ui + + self.assertEqual(tuple(ui.GESTURE_LABELS), hotkey.GESTURES) + + +class TestCWindowsOwnsCtrlWinToo(unittest.TestCase): + """`ctrl+win` is a Windows prefix, and push-to-talk changed what that costs. + + Under the old toggle gesture this class asserted that `ctrl+win+d` did nothing at + all: nothing had started, so refusing to fire at the release was the whole + behaviour. Push-to-talk opens the microphone on the press-down, so a desktop switch + now genuinely starts capturing — and the promise has to be restated rather than + quietly kept: + + **A third key stops the capture on the keystroke, and sends nothing.** Not at the + release, which would record every desktop switch for as long as the user held the + keys; and never as a send, which is the half that would put words in a window. + + What is captured in the fifty milliseconds before the third key is whatever the gate + let through, which is nothing — nobody has begun speaking yet. `Session.talk_end` + commits it regardless rather than applying a minimum-length rule, on the grounds + that an empty utterance costs nothing and a threshold eventually eats a real word. + """ + + def test_ctrl_win_d_makes_a_virtual_desktop_and_sends_nothing(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_D).up(VK_D).up(VK_LWIN, VK_LCONTROL) + fired = _fired(presses) + self.assertEqual(fired, ["warm", "talk", "talk-break"]) + self.assertNotIn("talk-end", fired) + + def test_ctrl_win_left_switches_desktop_and_sends_nothing(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LEFT).up(VK_LEFT).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_the_break_happens_on_the_third_key_not_on_the_release(self): + # The timing is the whole difference. Waiting for the release would leave the + # microphone open across every desktop switch in a long hold. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), ["warm", "talk"]) + keys.down(VK_LEFT) + self.assertEqual(_fired(presses), ["talk-break"]) + + def test_switching_desktop_twice_breaks_once(self): + # `_talking` is cleared by the first break, so the second arrow finds nothing to + # stop. A second break would ask the session to close a microphone it has + # already closed. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LEFT).up(VK_LEFT).down(VK_LEFT).up(VK_LEFT) + keys.up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_an_extra_modifier_is_a_different_chord_and_never_starts_one(self): + # ctrl+shift+win is not ctrl+win. Nothing starts, so nothing needs breaking. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LSHIFT, VK_LCONTROL, VK_LWIN) + keys.up(VK_LWIN, VK_LCONTROL, VK_LSHIFT) + self.assertEqual(_fired(presses), []) + + def test_the_extra_modifier_blocks_it_whenever_it_went_down(self): + # Both orders, because the first version of this got one of them wrong: an + # unwanted modifier pressed *before* the chord formed was forgotten when arming + # reset the verdict. Held state and press history are different questions, and + # this is the pair that tells them apart. The one order that is not here is + # shift arriving last — that is a break, and it has its own test below. + for order in ((VK_LSHIFT, VK_LCONTROL, VK_LWIN), + (VK_LCONTROL, VK_LSHIFT, VK_LWIN)): + with self.subTest(order=order): + chord, presses = _chord() + keys = _Keyboard(chord).down(*order) + keys.up(*reversed(order)) + self.assertEqual(_fired(presses), []) + + def test_shift_arriving_last_is_a_break_like_any_other_third_key(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_LSHIFT) + keys.up(VK_LSHIFT, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_releasing_the_extra_modifier_first_frees_the_chord_for_the_next_hold(self): + # `_extra` is held state, so it has to clear on release. If it latched, one + # accidental Shift would kill the chord for the life of the process — a defect + # whose only symptom is that the feature stops working and never comes back. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LSHIFT).up(VK_LSHIFT) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_one_modifier_alone_never_starts_however_long_it_is_held(self): + for vk in (VK_LCONTROL, VK_LWIN): + with self.subTest(vk=vk): + chord, presses = _chord() + _Keyboard(chord).down(vk, vk, vk).up(vk) + self.assertEqual(_fired(presses), []) + + def test_ordinary_typing_never_starts_one(self): + chord, presses = _chord() + keys = _Keyboard(chord) + for vk in (VK_A, VK_D, VK_LEFT): + keys.down(vk).up(vk) + self.assertEqual(_fired(presses), []) + + def test_a_key_pressed_before_the_chord_formed_is_not_the_chords_business(self): + # Typing, then reaching for the chord, is somebody starting a new gesture — not + # a dirty hold. The history resets when the chord forms; only what is still + # *held* survives into the verdict. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_A).up(VK_A) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + +class TestDItLearnsNothingAboutTheKeysItRejects(unittest.TestCase): + """The narrowing of R16, asserted rather than asserted *about*. + + The claim being defended is not "Flow is trustworthy" — it is that a key outside the + chord changes one boolean and leaves nothing behind. So the test is a state + comparison: type different things, and check the object cannot tell them apart. + """ + + def test_two_different_keys_leave_the_object_in_identical_states(self): + # If a virtual key were being recorded anywhere, typing `a` and typing `d` would + # have to produce different objects. They must not. + # Skipped by name, and the list is short on purpose: these are the per-instance + # objects that can only ever compare unequal — a queue, a thread, a C callback. + # *Everything else* is compared, which is what makes this a trap rather than a + # restatement: a field added tomorrow to hold a virtual key would be compared by + # default and would fail here. + identity = {"presses", "_proc", "_thread", "_ready", "_hook", "_tid"} + states = [] + for vk in (VK_A, VK_D, VK_LEFT): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(vk).up(vk) + states.append(vars(chord).copy()) + for other in states[1:]: + for name, value in states[0].items(): + if name in identity: + continue + with self.subTest(field=name): + self.assertEqual(value, other[name]) + + def test_what_it_keeps_about_a_rejected_key_is_one_boolean(self): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertFalse(chord._other) + keys.down(VK_A) + self.assertIs(chord._other, True) + + def test_the_only_fields_it_has_are_the_ones_the_state_machine_needs(self): + # A guard on the shape rather than on today's code: a future field holding a + # keystroke would have to be added here first, which is the moment to argue + # about it. + chord, _presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_A).up(VK_A, VK_LWIN, VK_LCONTROL) + self.assertEqual( + set(vars(chord)), + {"presses", "mods", "action", "warm_action", "end_action", "break_action", + "toggle_action", "gesture", "installed", "_down", "_other", "_extra", + "_armed", "_talking", "_hook", "_tid", "_ready", "_proc", "_thread"}, + ) + + +class TestEItNeverSwallowsAKeystroke(unittest.TestCase): + """Ctrl and Win have real jobs, and Flow does not get to keep them.""" + + def test_every_event_reaches_the_next_hook(self): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_A) + keys.up(VK_A, VK_LWIN, VK_LCONTROL) + self.assertTrue(keys.passed) + for message, vk, passed_on in keys.passed: + with self.subTest(message=message, vk=vk): + self.assertTrue(passed_on) + + def test_including_the_release_that_fired_the_chord(self): + # The one most likely to be lost to a `return 1` added in a hurry — and losing it + # means the Ctrl release never reaches the app, which strands every ctrl- + # shortcut in whatever window had focus. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN) + self.assertEqual(_fired(presses), HOLD) + self.assertTrue(keys.passed[-1][2]) + + def test_a_negative_code_is_passed_on_without_being_looked_at(self): + # `nCode < 0` means "pass it on without looking", and it is not advice. A hook + # that inspected those events anyway is a hook that can fire on something the OS + # explicitly said not to read. + chord, presses = _chord() + chord._down["ctrl"] = chord._down["win"] = True + chord._armed = True + block = hotkey._KBDLLHOOKSTRUCT(vkCode=VK_LWIN, scanCode=0, flags=0, time=0, + dwExtraInfo=None) + with mock.patch.object(hotkey, "user32") as fake: + fake.CallNextHookEx.return_value = 0 + chord._on_key(-1, WM_KEYUP, ctypes.addressof(block)) + self.assertTrue(fake.CallNextHookEx.called) + self.assertEqual(_fired(presses), []) + self.assertTrue(chord._armed) + + +class TestFTheHookIsTornDownByWhoeverOwnsGlobalKeyInput(unittest.TestCase): + """A hook left installed is one the OS calls into a dead interpreter.""" + + def test_stopping_the_hotkeys_stops_the_chord(self): + # The pill already calls `hotkeys.stop()` on the way out, and there should be + # exactly one thing that owns the teardown of "global key input". Hanging the + # chord off `Hotkeys` is what makes that true without `flow/ui.py` learning a + # second name. + hotkeys = hotkey.Hotkeys(hotkey.DEFAULT_BINDINGS) + chord, _presses = _chord() + hotkeys.chord = chord + with mock.patch.object(chord, "stop") as stop: + with mock.patch.object(hotkey, "user32"): + hotkeys.stop() + self.assertTrue(stop.called) + + def test_hotkeys_without_a_chord_still_stop_cleanly(self): + # The `--no-chord` and hook-refused paths both leave `chord` as None, and the + # quit path must not care which one it is looking at. + hotkeys = hotkey.Hotkeys(hotkey.DEFAULT_BINDINGS) + self.assertIsNone(hotkeys.chord) + with mock.patch.object(hotkey, "user32"): + hotkeys.stop() + + def test_a_refused_hook_is_a_false_from_start_and_not_a_crash(self): + # `SetWindowsHookExW` answers NULL when the OS refuses — policy, another process, + # a desktop this one cannot reach into. Not fatal: the registered toggle is still + # there, and `flow/__main__.py` has a line to print about it. + chord, _presses = _chord() + with mock.patch.object(chord, "_install", return_value=None): + self.assertFalse(chord.start(timeout=5.0)) + self.assertFalse(chord.installed) + + def test_a_raising_install_is_the_same_event_as_a_refused_one(self): + # The failure that would otherwise surface as a traceback on a daemon thread's + # stderr — which on a windowed build is nowhere at all — while `start()` blocked + # for the whole timeout waiting for a flag nobody was going to set. + chord, _presses = _chord() + with mock.patch.object(hotkey, "user32") as fake: + fake.SetWindowsHookExW.side_effect = OSError("refused") + self.assertFalse(chord.start(timeout=5.0)) + self.assertFalse(chord.installed) + + def test_the_callback_is_held_so_the_os_cannot_call_a_collected_one(self): + # Without a reference on the instance the `WINFUNCTYPE` object is collected while + # still installed, and the process dies inside a keystroke somewhere unrelated — + # a crash with no connection to anything Flow was doing. + chord, _presses = _chord() + self.assertIsNotNone(chord._proc) + self.assertIn("_proc", vars(chord)) + + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_editor.py b/tests/test_editor.py index 19c137d..abac53e 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -605,6 +605,7 @@ def _bubble(self, note: str, text: str = "a draft"): #: is an unpackable that raises, which is the loud failure this would rather have #: than a fixture silently laying out against a screen of no particular size. b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text, b._sent, b._partial, b._note = text, "", "", note b._editor = None @@ -624,14 +625,26 @@ def test_the_error_clears_the_chip_row(self): f"the note runs to y={note_bottom} and the chips start at y={chips_top}", ) - def test_the_bubble_grew_to_make_room_rather_than_clipping(self): - # The other way to stop an overlap is to cut the text off, and for an error - # message that is the same defect wearing a different hat. + def test_the_bubble_grows_to_make_room_rather_than_clipping(self): + """The other way to stop an overlap is to cut the text off, and for an error + message that is the same defect wearing a different hat. + + Briefly untrue, while the panel was a fixed height — the room came out of the + body instead. The band is snug around its content again (`_settled_h`), so this + is back to what it always asserted, and the foot still does not move: the shell + grows upward. + """ short = self._bubble("ok") short._render() long_ = self._bubble(self.ERROR) long_._render() - self.assertGreater(long_._h, short._h) + # `assertGreaterEqual`, not `assertGreater`, and the reason is `_settled_h`: the + # band steps by a whole body line, so a note that grew by 14 px can land in the + # same 17 px bucket as the one before it. That is the point of the snap — the + # window stops changing size for every small thing — and the property this test + # is really about is the next two lines, which is that the note is drawn whole. + # The sibling test above asserts it clears the chips. + self.assertGreaterEqual(long_._h, short._h) drawn = next(i for i in long_.canvas.items if "WinError 2" in i["text"]) self.assertEqual(drawn["text"], self.ERROR, "the note must not be truncated") @@ -666,6 +679,7 @@ def _bubble(self, **session): b.pill.session = mock.Mock(**fields) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -743,6 +757,7 @@ def _bubble(self): ) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -832,6 +847,7 @@ def _bubble(self, partial: str, note: str = "local: replace('x' -> 'y')"): ) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "seconds, send the question. No auto-ask to press it yourself." b._sent, b._partial, b._note = "", partial, note @@ -861,7 +877,10 @@ def test_the_partial_clears_the_chip_row(self): f"the partial runs to y={bottom} and the chips start at y={chips_top}", ) - def test_the_bubble_grew_rather_than_clipping(self): + def test_the_bubble_grows_rather_than_clipping(self): + # Grows in whole-line steps rather than continuously (`_settled_h`), and upward, + # so a partial arriving while you speak moves the top edge and leaves every + # control where it was. one = self._bubble("part key towel control") one._render() many = self._bubble(self.PARTIAL) @@ -945,6 +964,7 @@ def bubble(self, first=0.0, last=0.3, lines=60, height=200): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock(mode="dictate", editing=True, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() @@ -1111,6 +1131,7 @@ def bubble(self, text="a draft", sent=""): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock(mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() diff --git a/tests/test_help.py b/tests/test_help.py index ac8f81c..a0d1ca5 100644 --- a/tests/test_help.py +++ b/tests/test_help.py @@ -29,9 +29,24 @@ class FakeHotkeys: """What `Hotkeys` looks like after `start()`: what registered, and what could not.""" - def __init__(self, chosen: dict, failed=()) -> None: + def __init__(self, chosen: dict, failed=(), chord=None) -> None: self.chosen = chosen self.failed = list(failed) + #: `None` on the launches that have no chord — `--no-chord`, `"chord": ""`, or a + #: hook the OS refused — which is most of the tests in this file and is why it + #: defaults that way. + self.chord = chord + + +class FakeChord: + """What `Chord` looks like once installed. It is not in `chosen` and never will be: + nothing registered it, because there is nothing `RegisterHotKey` could register.""" + + def __init__(self, spelling="ctrl+win", action="talk") -> None: + self.spelling, self.action = spelling, action + + def describe(self) -> str: + return self.spelling REGISTERED = {"toggle": "ctrl+shift+space", "send": "ctrl+alt+enter", @@ -62,6 +77,30 @@ def test_an_action_that_could_not_register_is_named_as_unavailable(self): line = next(ln for ln in text.splitlines() if "cancel" in ln) self.assertIn("owned by another app", line) + def test_the_chord_is_named_even_though_nothing_registered_it(self): + # The sheet is built from what `RegisterHotKey` accepted, and the chord is + # precisely the shortcut that call cannot express. A sheet built only from + # `chosen` would describe a machine on which the shape the user has been holding + # all week does not exist. + text = rendered(hotkeys=FakeHotkeys(REGISTERED, chord=FakeChord())) + self.assertIn("ctrl+win", text) + line = next(ln for ln in text.splitlines() if "ctrl+win" in ln) + self.assertIn("hold to talk, release to send", line) + self.assertIn("held", line) + + def test_a_launch_without_a_chord_does_not_mention_one(self): + # `--no-chord`, `"chord": ""` and a refused hook all land here, and they are the + # same rule the rest of the sheet follows: it says what works on this machine + # this launch, and nothing else. + self.assertNotIn("ctrl+win", rendered(hotkeys=FakeHotkeys(REGISTERED))) + + def test_the_chord_is_listed_before_the_hotkey_that_also_starts_dictation(self): + # They no longer do the same thing — the chord is a hold and the hotkey is a + # toggle — but they are the two ways to start talking, and the chord goes first + # because it is the one somebody opened this sheet to look up. + text = rendered(hotkeys=FakeHotkeys(REGISTERED, chord=FakeChord())) + self.assertLess(text.index("ctrl+win"), text.index("ctrl+shift+space")) + def test_no_hotkeys_at_all_is_a_sentence_rather_than_a_hole(self): # `--no-hotkeys` is a supported way to run, and an empty section reads as a bug # in the help rather than as a choice the user made at launch. diff --git a/tests/test_hotkey.py b/tests/test_hotkey.py index 3e51f95..8deb20d 100644 --- a/tests/test_hotkey.py +++ b/tests/test_hotkey.py @@ -313,6 +313,17 @@ def GetMessageW(self, *_args): def PostThreadMessageW(self, *_args): return 1 + #: The chord's hook, refused. This fake describes a machine with combos already + #: taken, and a hook is not a combo — nothing here is about `WH_KEYBOARD_LL`, and a + #: fake that granted one would have `main()` install a real global keyboard hook on + #: whoever is running the suite. NULL is a shape the app already handles: the + #: startup block says the chord is unavailable and the registered toggle stands. + def SetWindowsHookExW(self, *_args): + return None + + def UnhookWindowsHookEx(self, *_args): + return 1 + class Registered(unittest.TestCase): """One launch's worth of registration against a machine this test describes.""" diff --git a/tests/test_indicator.py b/tests/test_indicator.py index 73b3d63..9303353 100644 --- a/tests/test_indicator.py +++ b/tests/test_indicator.py @@ -363,7 +363,7 @@ def _pill(mode=CONVERSE, pinned=None): pill._flash = 0 pill.armed = True pill._clis = None - pill.levels = [0.0] * 18 + pill._meter_level = 0.0 pill.session = mock.Mock(mode=mode, state=State.IDLE, cli=pinned) return pill diff --git a/tests/test_inject_mac.py b/tests/test_inject_mac.py new file mode 100644 index 0000000..d6f6d48 --- /dev/null +++ b/tests/test_inject_mac.py @@ -0,0 +1,228 @@ +"""`flow.inject_mac`: the paste path that made Flow useful on a Mac. + +Off Windows Flow ran in Lite, where Send copies the draft and stops — and the owner's +verdict on that was the reason this module exists: "it seems on mac there is no actual +send it copies in clipboard that is making this no useful on mac". Every window fault had +been fixed by then and the thing they all led to still handed you a clipboard. + +No `osascript` runs here. What is asserted is the shape of what would be run, the order +things happen in, and what the user is told when the OS says no — which is the part that +cannot be tested on the machine this was written on and is the part most likely to be +wrong. +""" + +import subprocess +import unittest +from unittest import mock + +import flow.inject_mac as inject_mac + + +class Ran: + """Records the child processes `paste` would start, and answers for them.""" + + def __init__(self, front="TextEdit", clipboard="old text", fail=None, reason=""): + self.calls: list[tuple[list[str], str | None]] = [] + self.front, self.clipboard = front, clipboard + self.fail, self.reason = fail, reason + + def __call__(self, argv, stdin=None): + self.calls.append((argv, stdin)) + tool = argv[0] + if self.fail is not None and self.fail in " ".join(argv): + return False, self.reason + if tool == "pbpaste": + return True, self.clipboard + if tool == "osascript" and "frontmost" in argv[-1]: + return True, self.front + return True, "" + + @property + def tools(self) -> list[str]: + return [argv[0] for argv, _stdin in self.calls] + + def script(self) -> str: + """The keystroke script, which is the last osascript that is not the query.""" + for argv, _stdin in reversed(self.calls): + if argv[0] == "osascript" and "frontmost" not in argv[-1]: + return argv[-1] + return "" + + def copied(self) -> list[str]: + return [stdin for argv, stdin in self.calls if argv[0] == "pbcopy"] + + +class Paste(unittest.TestCase): + def setUp(self): + inject_mac.take_warnings() + self.addCleanup(inject_mac.take_warnings) + + def run_paste(self, ran=None, **kw): + ran = ran or Ran() + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later"): + ok = inject_mac.paste("hello", **kw) + return ok, ran + + def test_it_copies_then_sends_command_v(self): + ok, ran = self.run_paste() + self.assertTrue(ok) + self.assertEqual(ran.copied(), ["hello"]) + self.assertIn('keystroke "v" using command down', ran.script()) + + def test_the_clipboard_is_written_before_the_keystroke_is_attempted(self): + """The order is the decision, and `inject.py` made the same one. + + A keystroke can be refused; a clipboard write mostly cannot. Doing the fragile + half second means a refusal still leaves the words somewhere the user can reach + with their own Cmd-V, which is exactly what the permission note promises them. + """ + _ok, ran = self.run_paste() + # Against the keystroke script specifically, not against the first `osascript`: + # the frontmost query is one too, and it runs first on purpose so a refusal + # costs no clipboard at all. + copied = next(i for i, t in enumerate(ran.tools) if t == "pbcopy") + typed = next(i for i, (argv, _s) in enumerate(ran.calls) + if argv[0] == "osascript" and "frontmost" not in argv[-1]) + self.assertLess(copied, typed) + + def test_submit_presses_return_by_key_code_and_not_as_a_character(self): + # `keystroke return` sends the character, and an app that tells them apart gets a + # newline in the box instead of a send. + _ok, ran = self.run_paste(submit=True) + self.assertIn("key code 36", ran.script()) + self.assertNotIn("keystroke return", ran.script()) + + def test_the_paste_and_the_return_travel_in_one_script(self): + # Each `osascript` is a process launch, and the gap between them is where another + # window could come forward and take the Return. + _ok, ran = self.run_paste(submit=True) + self.assertEqual(len([a for a, _s in ran.calls if a[0] == "osascript"]), 2) + self.assertIn('keystroke "v"', ran.script()) + self.assertIn("key code 36", ran.script()) + + def test_no_submit_presses_nothing(self): + _ok, ran = self.run_paste() + self.assertNotIn("key code", ran.script()) + + def test_it_refuses_to_paste_into_flow_itself(self): + # The one outcome that would destroy the text being sent. `ui._bare_window` is + # what should make it impossible; this is the belt to those braces. + ok, ran = self.run_paste(Ran(front="Python")) + self.assertFalse(ok) + self.assertEqual(ran.copied(), []) + self.assertIn("had the focus", inject_mac.take_warnings()[0]) + + def test_a_refused_keystroke_names_the_permission_and_the_terminal(self): + """The only message most people will ever see from this module. + + Accessibility is granted to the *responsible* process — the terminal — so a note + naming Flow or Python sends the reader to a list Flow is not in. + """ + ok, _ran = self.run_paste(Ran(fail="System Events", reason="(-1719)")) + self.assertFalse(ok) + note = inject_mac.take_warnings()[0] + self.assertIn("Accessibility", note) + self.assertIn("terminal", note) + self.assertIn("Cmd-V", note) + + def test_every_refusal_code_is_recognised(self): + for code in inject_mac.DENIED_CODES: + with self.subTest(code=code): + self.assertTrue(inject_mac.denied(f"System Events got an error ({code})")) + self.assertFalse(inject_mac.denied("some other failure")) + + def test_a_failure_that_is_not_the_permission_says_what_it_was(self): + # Invariant 4: a send that did nothing must never do it quietly. + ok, _ran = self.run_paste(Ran(fail="System Events", reason="osascript exploded")) + self.assertFalse(ok) + self.assertIn("osascript exploded", inject_mac.take_warnings()[0]) + + def test_a_clipboard_that_refuses_stops_before_the_keystroke(self): + # Sending Cmd-V after a failed copy pastes whatever was there before, which is + # somebody else's text going into the window they were working in. + ok, ran = self.run_paste(Ran(fail="pbcopy", reason="no pasteboard")) + self.assertFalse(ok) + self.assertEqual(ran.script(), "") + self.assertIn("clipboard", inject_mac.take_warnings()[0]) + + def test_empty_text_does_nothing_at_all(self): + with mock.patch.object(inject_mac, "_run") as ran: + self.assertFalse(inject_mac.paste("")) + ran.assert_not_called() + + def test_the_old_clipboard_is_read_and_scheduled_to_go_back(self): + ran = Ran(clipboard="something the user had") + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later") as later: + inject_mac.paste("hello") + later.assert_called_once_with("something the user had") + + def test_restore_can_be_turned_off_and_then_nothing_is_read(self): + ran = Ran() + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later") as later: + inject_mac.paste("hello", restore_clipboard=False) + self.assertNotIn("pbpaste", ran.tools) + later.assert_not_called() + + def test_hwnd_is_accepted_and_ignored(self): + # It exists for the signature `__main__.on_send` is written against. macOS has no + # window handle to aim at and does not need one: Flow's windows never take focus. + ok, _ran = self.run_paste(hwnd=0x22) + self.assertTrue(ok) + + +class Running(unittest.TestCase): + """`_run` itself, which must turn every way a child can fail into a sentence.""" + + def test_a_missing_tool_is_a_reason_and_not_an_exception(self): + with mock.patch.object(subprocess, "run", side_effect=OSError("no such file")): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertIn("osascript", why) + + def test_a_wedged_child_is_given_up_on(self): + # Finite by construction: a hung System Events must not hang the send. + with mock.patch.object(subprocess, "run", + side_effect=subprocess.TimeoutExpired("osascript", 10)): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertIn("did not answer", why) + + def test_a_nonzero_exit_carries_the_stderr(self): + done = mock.Mock(returncode=1, stderr="it went wrong", stdout="") + with mock.patch.object(subprocess, "run", return_value=done): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertEqual(why, "it went wrong") + + def test_a_nonzero_exit_with_nothing_to_say_still_says_something(self): + done = mock.Mock(returncode=1, stderr="", stdout="") + with mock.patch.object(subprocess, "run", return_value=done): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertTrue(why) + + +class Warnings(unittest.TestCase): + def test_taking_them_clears_them(self): + # `inject.py`'s rule: a warning read twice is a failure reported twice, and one + # never read is the silence invariant 4 forbids. + inject_mac.take_warnings() + inject_mac._warn("something") + self.assertEqual(inject_mac.take_warnings(), ["something"]) + self.assertEqual(inject_mac.take_warnings(), []) + + +class TheStartupLine(unittest.TestCase): + def test_it_is_ascii_like_every_other_startup_line(self): + # `__main__.say` documents why: a redirected stdout on a legacy console code page + # cannot encode an en-dash, so a line carrying one crashes instead of printing. + for line in (inject_mac.permission_note(),): + line.encode("ascii") + line.encode("cp437") + + +if __name__ == "__main__": # pragma: no cover + unittest.main() diff --git a/tests/test_lite.py b/tests/test_lite.py index 6bfb969..ec3e1c5 100644 --- a/tests/test_lite.py +++ b/tests/test_lite.py @@ -184,7 +184,7 @@ class Pill: found this the hard way). The attribute has to exist. """ - def __init__(self, s: Session, lite: bool = True) -> None: + def __init__(self, s: Session, lite: bool = True, injector: bool = True) -> None: import flow.ui as ui self.copied: list[str] = [] @@ -193,7 +193,12 @@ def __init__(self, s: Session, lite: bool = True) -> None: self.pill = ui.Pill.__new__(ui.Pill) self.pill.session = s self.pill.lite = lite - self.pill.on_send = self._on_send + # `injector` is what `__main__` decides by importing a paste module or not, and + # it is no longer the same question as `lite`. A Mac is Lite — no global hotkeys, + # no window handles — and still pastes, through System Events. `injector=False` + # is the case with nothing to paste with: `--lite` on Windows, `--no-paste`, or a + # platform Flow has no injector for. + self.pill.on_send = self._on_send if injector else None self.pill.paste_target = 0x22 self.pill.bubble = mock.Mock() self.pill._flash = 0 @@ -223,9 +228,16 @@ def notes(self) -> str: class TestSendInLiteIsACopy(unittest.TestCase): + """Lite with nothing to paste with — `--lite` on Windows, or `--no-paste`. + + Lite used to mean this by definition. It does not any more: a Mac is Lite in every + other respect and pastes through System Events, so the copy is now the fallback for + *no injector* rather than the behaviour of a mode. See `TestLiteWithAnInjector`. + """ + def setUp(self): self.s = session() - self.p = Pill(self.s) + self.p = Pill(self.s, injector=False) self.s.draft.set(DRAFT) def test_the_draft_goes_to_the_clipboard_and_nowhere_else(self): @@ -286,7 +298,7 @@ class TestTheEnterVariantCollapses(unittest.TestCase): def setUp(self): self.s = session() - self.p = Pill(self.s) + self.p = Pill(self.s, injector=False) self.s.draft.set(DRAFT) def test_it_copies_rather_than_refusing(self): @@ -305,7 +317,7 @@ def test_both_spoken_triggers_reach_it_through_the_router(self): ("enter boom", "Enter is yours to press")): with self.subTest(said=said): s = session() - p = Pill(s) + p = Pill(s, injector=False) s.draft.set(DRAFT) p.say(said) self.assertEqual(p.copied, [DRAFT]) @@ -319,6 +331,12 @@ def _pill(self, lite: bool): pill = ui.Pill.__new__(ui.Pill) pill.lite = lite pill.paste_target = None + # `_track_target` writes the app name onto the session for `_app_note` to read + # later. A stand-in rather than a mock, because what the tests below check is the + # *value* that lands there — and on a `Pill` built by `__new__`, a missing + # attribute is not an AttributeError but a `tkinter` lookup that recurses until + # the interpreter gives up, which is a confusing way to learn this line exists. + pill.session = type("S", (), {"target_app": ""})() return pill def test_the_foreground_is_never_asked_about_in_lite(self): @@ -345,6 +363,45 @@ def test_full_mode_still_tracks_it(self): pill._track_target() self.assertEqual(pill.paste_target, 0x99) + def test_the_app_behind_the_window_is_named_for_the_per_app_note(self): + import flow.ui as ui + + pill = self._pill(lite=False) + with mock.patch.object(ui, "foreground_hwnd", return_value=0x99), mock.patch.object(ui, "owned_by_flow", return_value=False), mock.patch.object(ui, "classify") as named: + named.return_value = type("T", (), {"process": "code.exe"})() + pill._track_target() + self.assertEqual(pill.session.target_app, "code.exe") + + def test_it_is_resolved_on_the_edge_and_not_once_a_frame(self): + # `classify` opens a process handle and this runs at 30 fps. Paying that every + # frame is a cost paid forever to answer a question whose answer moves a few + # times an hour — so it is asked when the window changes and remembered between. + import flow.ui as ui + + pill = self._pill(lite=False) + with mock.patch.object(ui, "foreground_hwnd", return_value=0x99), mock.patch.object(ui, "owned_by_flow", return_value=False), mock.patch.object(ui, "classify") as named: + named.return_value = type("T", (), {"process": "code.exe"})() + for _ in range(10): + pill._track_target() + self.assertEqual(named.call_count, 1) + named.return_value = type("T", (), {"process": "slack.exe"})() + with mock.patch.object(ui, "foreground_hwnd", return_value=0xAB): + pill._track_target() + self.assertEqual(named.call_count, 2) + self.assertEqual(pill.session.target_app, "slack.exe") + + def test_lite_never_names_an_app_because_it_never_has_a_target(self): + # No target-window awareness at all (product.md), which reads downstream as an + # app with no note configured — the behaviour every launch had before per-app + # notes existed, rather than a gap. + import flow.ui as ui + + pill = self._pill(lite=True) + with mock.patch.object(ui, "classify") as named: + pill._track_target() + named.assert_not_called() + self.assertEqual(pill.session.target_app, "") + class TestTheWindowsOnlyTkAttributes(unittest.TestCase): """`-transparentcolor` and `-toolwindow` exist only on Windows. diff --git a/tests/test_longrun.py b/tests/test_longrun.py index 77b2ea0..74462f9 100644 --- a/tests/test_longrun.py +++ b/tests/test_longrun.py @@ -92,6 +92,52 @@ def test_model_is_dropped_after_idle(self): self.assertEqual(asr.unloads, 1) s.close() + def test_a_warm_holds_the_models_against_an_idle_that_is_already_due(self): + # The race the grace window exists for. `_last_activity` is moved by Flow's own + # milestones, so somebody who has just reached for the chord is still idle by + # that measure — and the health pump runs every tick, so without this it is free + # to unload between the press-down and the release that arms. + asr = TrackingAsr() + s = Session(asr=asr, mic=StubMic()) + s.start() + s.warm() + with mock.patch.object(session_mod, "IDLE_UNLOAD_SEC", 0.0): + s.tick() + self.assertTrue(asr.loaded) + self.assertEqual(asr.unloads, 0) + s.close() + + def test_and_lets_go_once_the_grace_is_spent(self): + # A window, not a veto. `ctrl+win` is also Windows' desktop-switch prefix, so a + # warm that reset the idle clock outright would mean anybody who switches + # desktops through the day never unloads at all — the setting would quietly stop + # existing for exactly the people using their machine most. + asr = TrackingAsr() + s = Session(asr=asr, mic=StubMic()) + s.start() + s.warm() + with mock.patch.object(session_mod, "IDLE_UNLOAD_SEC", 0.0), \ + mock.patch.object(session_mod, "WARM_GRACE_SEC", 0.0): + s.warm() + s.tick() + self.assertFalse(asr.loaded) + self.assertEqual(asr.unloads, 1) + s.close() + + def test_a_session_nobody_warmed_is_not_holding_anything_off(self): + # Zero is the state every session starts in and mostly stays in, so the guard + # must not be what decides the ordinary case. + s = Session(asr=TrackingAsr(), mic=StubMic()) + self.assertEqual(s._warm_until, 0.0) + s.close() + + def test_the_idle_threshold_is_the_gaps_in_a_day_and_not_five_minutes(self): + # Stated as a number because it is a judgement and not an accident: five minutes + # was inside the gaps of an ordinary working session, so the common case was not + # reclaiming memory from somebody who left, it was paying a reload in the middle + # of their first sentence back. + self.assertEqual(session_mod.IDLE_UNLOAD_SEC, 1800.0) + def test_model_is_kept_while_a_draft_is_held(self): # Unloading mid-draft would make the next correction pay a reload for nothing. asr = TrackingAsr() diff --git a/tests/test_main.py b/tests/test_main.py index 7810b31..f2b4946 100644 --- a/tests/test_main.py +++ b/tests/test_main.py @@ -94,7 +94,11 @@ def test_the_entry_point_imports_at_all_without_win32(self): def test_a_mac_gets_lite_rather_than_a_refusal(self): code, out, pill, session = launch("darwin") self.assertEqual(code, 0) - self.assertIn("Flow Lite on darwin", out) + # Not the Lite banner any more: a Mac pastes, which is the one thing Lite was + # defined by. It is still Lite in every other respect and the `lite` kwargs below + # are what say so. + self.assertIn("Flow on darwin", out) + self.assertIn("Send pastes", out) self.assertNotIn("Windows-only", out) self.assertTrue(pill.call_args.kwargs["lite"]) self.assertTrue(session.call_args.kwargs["lite"]) @@ -103,11 +107,14 @@ def test_it_names_whichever_platform_it_found(self): _code, out, _pill, _session = launch("linux") self.assertIn("linux", out) - def test_no_hotkey_is_registered_and_no_paste_handler_is_built(self): - # The two halves of "no hands", asserted where they are decided rather than - # where they would be felt. `on_send` is the paste closure, and in Lite it - # closes over names that were never imported — so handing one over would be a - # handler that fails on its first call. + def test_no_hotkey_is_registered_but_a_paste_handler_is(self): + """"No hands" was two halves and is now one. + + The hotkeys stay unregistered: there is no `RegisterHotKey` here and the pill is + the gesture. The paste closure is built, because `inject_mac` gives it something + to close over — `osascript` rather than `SendInput`. Handing one over used to be + a handler that would fail on its first call; it is not any more. + """ _code, out, pill, _session = launch("darwin") # The registration report is `hotkey ` per line. Matched on the # line rather than on the word, because the Lite banner says "no global hotkeys" @@ -115,12 +122,19 @@ def test_no_hotkey_is_registered_and_no_paste_handler_is_built(self): self.assertEqual( [ln for ln in out.splitlines() if ln.startswith("hotkey")], []) self.assertIsNone(pill.call_args.kwargs["hotkeys"]) + self.assertIsNotNone(pill.call_args.kwargs["on_send"]) + + def test_no_paste_puts_the_clipboard_back(self): + # The one way to get the old behaviour on a Mac, and it has to keep working: + # somebody who does not want Flow synthesising keystrokes should not have to + # choose between that and using Flow. + _code, out, pill, _session = launch("darwin", ["--no-paste"]) self.assertIsNone(pill.call_args.kwargs["on_send"]) + self.assertIn("Send copies the draft", out) - def test_the_mode_line_does_not_name_a_window_lite_cannot_see(self): + def test_the_mode_line_says_it_pastes(self): _code, out, _pill, _session = launch("darwin") - self.assertIn("Send copies the draft", out) - self.assertNotIn("focused window", out) + self.assertIn("Send pastes into the focused window", out) def test_every_lite_startup_line_is_ascii_like_the_rest(self): # `say()` documents why: a redirected stdout on a legacy console code page @@ -495,7 +509,7 @@ def test_a_mac_launch_reads_the_field_without_reaching_for_win32(self): # that stops Flow from starting on the platform Lite exists for. code, out = self.launch(self.OVERRIDES, ["--lite"], platform="darwin") self.assertEqual(code, 0) - self.assertIn("Flow Lite on darwin", out) + self.assertIn("on darwin", out) @unittest.skipUnless(sys.platform == "win32", "Windows-only: ctypes.WinDLL") def test_no_hotkeys_reads_no_override_because_it_registers_none(self): @@ -512,3 +526,29 @@ def test_and_an_unusable_block_is_not_named_under_that_flag_either(self): if __name__ == "__main__": unittest.main() + +class TestTheModelIsLoadedBeforeItIsAskedFor(unittest.TestCase): + """"loading the model" used to be the first thing a fresh Flow said back. + + It said it in the bubble, while somebody was already speaking, because the load lands + *inside* the first utterance rather than in front of it — first partial 1 230 ms + against ~570 ms for the four behind it. The chord's press-down has warmed the models + since push-to-talk shipped, which covers the second use and not the first. + """ + + def test_startup_warms_the_session(self): + _code, _out, _pill, session = launch("win32") + session.return_value.warm.assert_called_once() + + def test_it_happens_off_windows_too(self): + # The load is the same load and the wait is the same wait; nothing about it is + # platform-shaped. + _code, _out, _pill, session = launch("darwin") + session.return_value.warm.assert_called_once() + + def test_no_warm_leaves_it_for_the_first_word(self): + # For a launcher that starts with the machine, where paying a model load at + # login is the wrong trade — and for measuring the cold path on purpose. + _code, _out, _pill, session = launch("win32", ["--no-warm"]) + session.return_value.warm.assert_not_called() + diff --git a/tests/test_native.py b/tests/test_native.py new file mode 100644 index 0000000..01a6e3c --- /dev/null +++ b/tests/test_native.py @@ -0,0 +1,414 @@ +"""The macOS on-device decoder, and the rule that decides when Flow reaches for it. + +The engine decides what Flow can *hear*, so the interesting tests here are not about +Swift — none of this compiles a helper — they are about the choice. Two properties carry +it: + + **`auto` never switches a working machine.** Apple's recogniser is a different engine, + not a spare one: no `no_speech_prob`, so `clean.py` falls to the narrow filler check it + documents for exactly that; one quality tier where Whisper has two; no hotword biasing + for the rescue path. Reaching for it on a machine where Whisper was fine would change + what Flow hears for a reason nobody asked about. + + **Every refusal says why.** "Not available" is four different sentences — wrong + platform, no toolchain, a build that failed, a permission declined — and a user who + cannot see which one has a feature that is missing for no stated reason. + +The subprocess is faked at `Popen`, so the framing this file cares about — a +length-prefixed block of float32 out, one line of text back — is asserted against the +bytes actually written rather than against a Mac nobody in CI has. +""" + +import struct +import subprocess +import sys +import unittest +from pathlib import Path +from unittest import mock + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +import flow.native as native # noqa: E402 +from flow.__main__ import _engine, _models_present # noqa: E402 + + +def args(engine="auto"): + return mock.Mock(engine=engine) + + +def _only(present): + """A `Path.exists` that is true for exactly one path. + + `mock.patch.object` cannot patch an attribute on a `Path` *instance* — they are + read-only — so the class method is replaced and told which path is meant to be + there. Which is the distinction under test: the helper source exists, the built + binary does not, so a build is attempted. + """ + return lambda self: self == present + + +class TestTheEngineChoice(unittest.TestCase): + """`_engine`, which is the whole feature — the rest is plumbing.""" + + def test_asking_for_whisper_gets_whisper_and_asks_nothing(self): + # No probe, no build, no subprocess: somebody who named the engine has already + # answered the question this function exists to ask. + with mock.patch.object(native, "available") as probe: + self.assertEqual(_engine(args("whisper"), "base.en", "small.en"), + ("whisper", "")) + probe.assert_not_called() + + def test_auto_keeps_whisper_when_the_models_are_on_the_machine(self): + # The property that matters most. A working machine is never switched. + with mock.patch("flow.__main__._models_present", return_value=True), \ + mock.patch.object(native, "available") as probe: + self.assertEqual(_engine(args(), "base.en", "small.en"), ("whisper", "")) + probe.assert_not_called() + + def test_auto_reaches_for_native_only_when_whisper_has_nothing_to_run(self): + # The situation this was written for: a network that blocks huggingface.co, + # where the alternative is not a worse engine but no dictation at all. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", return_value=(True, "")): + engine, why = _engine(args(), "base.en", "small.en") + self.assertEqual(engine, "native") + self.assertIn("models not found", why) + + def test_and_says_so_on_the_startup_line(self): + # A silent engine change would be the one thing nobody could check. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", return_value=(True, "")): + _engine, why = _engine_result() + self.assertTrue(why.strip()) + + def test_neither_engine_available_still_returns_whisper_and_names_the_reason(self): + # Whisper will fail its own way, with its own message. What this must not do is + # return an engine that is not there, or fail silently between the two. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", + return_value=(False, "Dictation is off")): + engine, why = _engine(args(), "base.en", "small.en") + self.assertEqual(engine, "whisper") + self.assertIn("Dictation is off", why) + + def test_asking_for_native_off_a_mac_is_refused_out_loud(self): + said = [] + with mock.patch.object(sys, "platform", "win32"), \ + mock.patch("flow.__main__.say", said.append): + self.assertEqual(_engine(args("native"), "base.en", "small.en"), + ("whisper", "")) + self.assertIn("macOS only", " ".join(said)) + + def test_asking_for_native_on_a_mac_that_cannot_reports_the_reason(self): + said = [] + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__.say", said.append), \ + mock.patch.object(native, "available", + return_value=(False, "no Swift toolchain")): + self.assertEqual(_engine(args("native"), "base.en", "small.en"), + ("whisper", "")) + self.assertIn("no Swift toolchain", " ".join(said)) + + +def _engine_result(): + return _engine(args(), "base.en", "small.en") + + +class TestAutoNeverPaysForAnEngineItMayNotUse(unittest.TestCase): + """The regression CI found, pinned so it cannot come back. + + Asking `available()` unconditionally at startup took the macOS CI leg from **35 + seconds to 643**. Every launch on a Mac without Whisper models was compiling Swift + and then sitting on `--probe` until it timed out, because the probe waits for an + authorization dialog a headless machine never answers. + + A user's first launch would have done the same thing: a full minute of nothing + before a pill appeared, on a machine that had asked for none of it. So `auto` uses + what is *ready*, and naming the engine is what builds it. + """ + + def test_auto_will_not_compile_anything(self): + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["compile"] = compile_if_missing + seen["timeout"] = timeout + return False, "not built yet" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", fake): + _engine(args(), "base.en", "small.en") + self.assertFalse(seen["compile"]) + + def test_and_will_not_wait_a_minute_on_a_permission_dialog(self): + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["timeout"] = timeout + return False, "not built yet" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", fake): + _engine(args(), "base.en", "small.en") + self.assertLessEqual(seen["timeout"], 15.0) + + def test_naming_the_engine_is_what_builds_it(self): + # The other half of the rule. Somebody who typed `--engine native` has asked for + # the compile and is willing to wait for it. + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["compile"] = compile_if_missing + return True, "" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(native, "available", fake): + self.assertEqual(_engine(args("native"), "base.en", "small.en")[0], + "native") + self.assertTrue(seen["compile"]) + + def test_an_unbuilt_helper_says_how_to_build_it(self): + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", return_value=False): + ok, why = native.available(compile_if_missing=False) + self.assertFalse(ok) + self.assertIn("--engine native", why) + + def test_a_probe_that_hangs_is_named_as_the_dialog_it_is(self): + # "probe failed: TimeoutExpired" is true and useless. The fix is a click, and + # the sentence should say so. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", return_value=True), \ + mock.patch.object(native, "_run", + side_effect=subprocess.TimeoutExpired("probe", 10)): + ok, why = native.available(timeout=10.0) + self.assertFalse(ok) + self.assertIn("Speech Recognition", why) + + def test_every_reason_survives_the_console_the_startup_line_prints_to(self): + # `say()` writes to a cp437 console on Windows, and these strings reach it + # through `_engine`. An em dash here is a launch that dies on its own + # explanation — which is exactly how this was found. + for reason in ("not built yet", "no Swift toolchain", "probe timed out"): + with self.subTest(reason=reason): + pass + import inspect + + source = inspect.getsource(native) + for line in source.splitlines(): + stripped = line.strip() + if stripped.startswith("#") or '"' not in line: + continue + for chunk in line.split('"')[1::2]: + with self.subTest(chunk=chunk[:40]): + chunk.encode("cp437") + + +class TestTheModelPresenceCheckNeverDownloads(unittest.TestCase): + """It is asked *because* the network may be unusable; it must not use it.""" + + def test_a_directory_holding_a_model_counts_without_asking_the_hub(self): + # What `--model /some/path` gives. Nothing to look up. + with mock.patch.object(Path, "exists", return_value=True): + with mock.patch.dict(sys.modules, {"huggingface_hub": None}): + self.assertTrue(_models_present("/somewhere/base.en")) + + def test_a_hub_lookup_is_made_offline(self): + seen = {} + fake = mock.Mock() + + def snapshot(repo, local_files_only=False): + import os + seen["offline"] = os.environ.get("HF_HUB_OFFLINE") + seen["local_only"] = local_files_only + return "/cache" + + fake.snapshot_download = snapshot + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + self.assertTrue(_models_present("base.en")) + self.assertEqual(seen["offline"], "1") + self.assertTrue(seen["local_only"]) + + def test_a_missing_model_is_absent_rather_than_an_exception(self): + fake = mock.Mock() + fake.snapshot_download = mock.Mock(side_effect=OSError("not cached")) + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + self.assertFalse(_models_present("base.en")) + + def test_the_environment_is_left_as_it_was_found(self): + # It sets HF_HUB_OFFLINE to ask its question. A launch that then downloaded + # nothing for the rest of the session would be this function's fault. + import os + + fake = mock.Mock() + fake.snapshot_download = mock.Mock(return_value="/cache") + was = os.environ.get("HF_HUB_OFFLINE") + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + _models_present("base.en") + self.assertEqual(os.environ.get("HF_HUB_OFFLINE"), was) + + +class TestItRefusesOffAMacWithAReason(unittest.TestCase): + """Pinned to a non-Mac platform rather than reading the runner's. + + These assert the *refusal*, and on a macOS runner there is nothing to refuse — it + would go and build a real binary instead, which is a different test and a slow one. + The suite runs on both platforms, so a test that means one thing on Windows and + another on macOS is a test that is only half run wherever it passes. + """ + + def test_build_names_the_platform(self): + with mock.patch.object(sys, "platform", "win32"): + with self.assertRaises(native.NotAvailable) as caught: + native.build() + self.assertIn("macOS", str(caught.exception)) + + def test_available_answers_false_and_why_rather_than_raising(self): + # `available()` is called during startup. A raise there is a launch that dies + # over a feature the machine was never going to have. + with mock.patch.object(sys, "platform", "win32"): + ok, why = native.available() + self.assertFalse(ok) + self.assertTrue(why) + + def test_a_missing_toolchain_is_named_with_the_command_that_fixes_it(self): + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", _only(native.SOURCE)), \ + mock.patch.object(native, "_run", + return_value=mock.Mock(returncode=1)): + ok, why = native.available() + self.assertFalse(ok) + self.assertIn("xcode-select --install", why) + + def test_a_build_error_carries_the_compilers_last_word(self): + fail = mock.Mock(returncode=1, stderr="flow_stt.swift:9: error: no such module", + stdout="") + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", _only(native.SOURCE)), \ + mock.patch.object(Path, "mkdir", lambda *a, **k: None), \ + mock.patch.object(native, "_run", + side_effect=[mock.Mock(returncode=0), fail]): + ok, why = native.available() + self.assertFalse(ok) + self.assertIn("no such module", why) + + +class TestTheWireFormat(unittest.TestCase): + """A length-prefixed block of float32 out, one line of text back. + + Asserted against the bytes actually written, because the Swift on the other end is + reading them with `load(as: UInt32.self).littleEndian` and a disagreement about + endianness or element size is a silent mis-decode rather than an error. + """ + + def transcriber(self, reply=b"hello there\n"): + proc = mock.Mock() + proc.poll.return_value = None + proc.stdout.readline.return_value = reply + proc.stdin = mock.Mock() + a = native.NativeTranscriber(binary=Path("/fake/flow-stt")) + a._proc = proc + return a, proc + + def test_it_writes_a_little_endian_count_then_the_samples(self): + a, proc = self.transcriber() + audio = np.arange(4, dtype=np.float32) + a.text(audio) + count, payload = [c[0][0] for c in proc.stdin.write.call_args_list] + self.assertEqual(count, struct.pack(" dict: + return {name: getattr(ui, name) for name in SHIPPED} + + +class TestTheShippedWidthIsUntouched(PanelWidth): + def test_the_module_starts_where_it_always_did(self): + # Before anything calls `apply_panel_width` at all — the state of a launch that + # never got as far as reading a profile, and of every test module that imports + # `flow.ui` for some other reason. + self.assertEqual(self.widths(), SHIPPED) + + def test_and_asking_for_regular_puts_it_back_exactly(self): + # The promise to everybody who never opens this menu. A size setting that also + # re-flowed their draft would be two changes shipped as one. + ui.apply_panel_width(ui.PANEL_WIDTHS["large"]) + ui.apply_panel_width(ui.PANEL_WIDTHS["regular"]) + self.assertEqual(self.widths(), SHIPPED) + + def test_the_default_name_means_the_shipped_width(self): + self.assertEqual(ui.PANEL_WIDTHS[ui.PANEL_DEFAULT], SHIPPED["BUBBLE_W"]) + + +class TestTheFloorHoldsBecauseSendMustStayReachable(PanelWidth): + def test_a_width_below_the_floor_is_clamped_to_it(self): + # The measured rows are 345 px (bubble) and 377 px (card) of chips before gaps + # and padding. Below 420 the row either loses its gaps or loses a label, and the + # label it loses first is Send. + for asked in (0, 1, 100, 379, 419, -50): + with self.subTest(asked=asked): + ui.apply_panel_width(asked) + self.assertEqual(ui.BUBBLE_W, SHIPPED["BUBBLE_W"]) + + def test_no_offered_size_is_below_the_floor(self): + # A guard on the menu rather than on the clamp: an entry that had to be clamped + # would be a row somebody could pick that silently did nothing. + for name, width in ui.PANEL_WIDTHS.items(): + with self.subTest(name=name): + self.assertGreaterEqual(width, SHIPPED["BUBBLE_W"]) + + def test_the_sizes_only_go_up(self): + # Stated as a property because it is a design decision and not an accident: + # there is no "small", and the reason is the chip row rather than restraint. + self.assertEqual(min(ui.PANEL_WIDTHS.values()), SHIPPED["BUBBLE_W"]) + + +class TestTheTwoWindowsMoveTogether(PanelWidth): + def test_the_card_follows_the_bubble_at_every_size(self): + # One window at two moments, docked to the same pill. A width that moved one + # without the other is visible the first time somebody switches mode. + for width in ui.PANEL_WIDTHS.values(): + with self.subTest(width=width): + ui.apply_panel_width(width) + self.assertEqual(ui.CARD_W, ui.BUBBLE_W) + + +class TestTheMeasurementsScaleWithTheColumn(PanelWidth): + def test_a_wider_panel_lays_out_more_text(self): + # `BODY_CHARS_PER_LINE` frozen at one width would under-feed the canvas at 640 px + # put the bottom of the draft below the fold — the setting would make the thing + # it exists to improve worse. + seen = [] + for width in sorted(ui.PANEL_WIDTHS.values()): + ui.apply_panel_width(width) + seen.append((ui.BODY_CHARS_PER_LINE, ui.BODY_TAIL_CHARS)) + self.assertEqual(seen, sorted(seen)) + self.assertLess(seen[0][0], seen[-1][0]) + + def test_the_tail_stays_the_same_number_of_lines(self): + # What `BODY_TAIL_CHARS` really encodes is ~28 lines, which is what keeps render + # cost flat (invariant 7). Holding lines rather than characters is what carries + # that invariant across a width change instead of re-measuring it. + lines = [] + for width in ui.PANEL_WIDTHS.values(): + ui.apply_panel_width(width) + lines.append(round(ui.BODY_TAIL_CHARS / ui.BODY_CHARS_PER_LINE)) + self.assertEqual(len(set(lines)), 1, lines) + + def test_a_line_is_never_zero_characters(self): + # The clamp exists because this number is divided by. It cannot be reached + # through the menu; it can be reached by a future width and a smaller font. + ui.apply_panel_width(0) + self.assertGreaterEqual(ui.BODY_CHARS_PER_LINE, 1) + + +class TestANameIsTurnedIntoAWidth(PanelWidth): + def test_every_offered_name_resolves_to_its_own_width(self): + for name, width in ui.PANEL_WIDTHS.items(): + with self.subTest(name=name): + self.assertEqual(ui.panel_width(name), width) + + def test_case_and_spacing_are_the_writers_business(self): + # This is a value a hand-edit can put in `profile.json`, so it reads the way + # every other hand-written value in that file does. + for text in ("LARGE", "Large", " large ", "lArGe"): + with self.subTest(text=text): + self.assertEqual(ui.panel_width(text), ui.PANEL_WIDTHS["large"]) + + def test_anything_unknown_is_the_shipped_width_rather_than_a_refusal(self): + # Deliberately the opposite of how `hotkey.parse` treats a bad combo, and the + # difference is what each costs when wrong. A hotkey that silently fell back + # leaves somebody pressing keys that do nothing with no way to find out; a panel + # that falls back is a window visibly not the size they asked for, with the + # evidence on the screen. + for value in ("enormous", "", None, 7, ["large"], {"large": 1}, True): + with self.subTest(value=value): + self.assertEqual(ui.panel_width(value), SHIPPED["BUBBLE_W"]) + + +class TestTheProfileRemembersIt(PanelWidth): + def test_the_two_modules_agree_on_the_default_name(self): + # `flow/profile.py` spells the default itself rather than importing it, because + # it is read on every launch including Lite's and `flow.ui` is not something it + # may need. That buys platform reach and costs exactly this risk, so the risk is + # bought back here. + self.assertEqual(PANEL_DEFAULT, ui.PANEL_DEFAULT) + self.assertIn(PANEL_DEFAULT, ui.PANEL_WIDTHS) + + def test_it_survives_a_save_and_a_load(self): + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + p = Profile(path) + p.panel = "large" + self.assertTrue(p.save()) + again = Profile(path) + self.assertTrue(again.load()) + self.assertEqual(again.panel, "large") + self.assertNotIn("panel", again.faults) + + def test_a_fresh_profile_asks_for_the_shipped_width(self): + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + p = Profile(Path(tmp) / "profile.json") + self.assertEqual(ui.panel_width(p.panel), SHIPPED["BUBBLE_W"]) + + def test_a_nonsense_value_in_the_file_is_named_and_still_launches(self): + # Both halves matter. `faults` is how `--stats` and the startup block say a + # setting degraded, and the launch has to survive it: a profile is not a thing + # somebody can be locked out of the app by. + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + path.write_text(json.dumps({"schema": 1, "panel": 7}), encoding="utf-8") + p = Profile(path) + self.assertTrue(p.load()) + self.assertIn("panel", p.faults) + self.assertEqual(ui.panel_width(p.panel), SHIPPED["BUBBLE_W"]) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_pill.py b/tests/test_pill.py index afdda73..78c6076 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -35,6 +35,17 @@ def __init__(self) -> None: def delete(self, *a, **kw) -> None: ... + #: `_sync_shell` places this canvas at the foot of a window whose top edge moves + #: when a panel opens. Recorded rather than ignored, so a test can check the row + #: really is at the bottom of whatever height the shell currently is. + def place(self, **kw) -> None: + self.placed = kw + + #: `_sync_dock` resizes the canvas when a panel docks or goes away. Accepted and + #: recorded rather than ignored, so a test can tell a widened pill from a moved one. + def configure(self, **kw) -> None: + self.width = kw.get("width", getattr(self, "width", None)) + def create_polygon(self, *a, **kw) -> None: ... def create_arc(self, *a, **kw) -> None: ... @@ -63,7 +74,7 @@ def pill(state=State.IDLE, *, armed=True, mode=DICTATE, hearing=True, p = ui.Pill.__new__(ui.Pill) p.canvas = Canvas() p.armed = armed - p.levels = [0.0] * ui.BARS + p._meter_level = 0.0 p.session = mock.Mock( mode=mode, state=state, hearing=hearing, editing=editing, cli=None, mic=mock.Mock(active=mic_active), @@ -410,6 +421,737 @@ def test_the_glyph_and_the_label_travel_together(self): self.assertEqual({o[-1] for o in p.canvas.ovals if o[0] < ui.METER_X}, {half}) +class TestTheMeterBloomsFromItsCentre(unittest.TestCase): + """The shape taken from FluidVoice's `BottomWaveformView`, asserted as a shape. + + Flow's meter used to be a scrolling history — one level per frame through a deque, + travelling right to left. It is a symmetric bloom now: every bar reads the same + current level, and the envelope is what makes the middle ones tallest. These are + the properties that distinguish the two, so that a future edit which quietly + restored the old behaviour would fail here rather than merely look different. + """ + + def heights(self, level: float) -> list[float]: + return [ui.Pill._bar_half_height(i, level) for i in range(ui.BARS)] + + def test_the_middle_is_the_tallest_part_of_the_shape(self): + h = self.heights(1.0) + self.assertEqual(max(h), max(h[ui.BARS // 2 - 1:ui.BARS // 2 + 1])) + + def test_it_falls_away_toward_both_ends(self): + # Monotonic out from the centre in each direction. The per-bar variation is + # deliberately small enough not to break this — a wobble that reordered the + # envelope would be a comb, not a bloom. + h = self.heights(1.0) + mid = ui.BARS // 2 + self.assertEqual(h[:mid], sorted(h[:mid])) + self.assertEqual(h[mid:], sorted(h[mid:], reverse=True)) + + def test_the_ends_still_move_rather_than_sitting_dead(self): + # `_ENVELOPE_MIN` is 18% and not 0. An end bar pinned at the minimum would read + # as a broken meter rather than a shaped one. + self.assertGreater(self.heights(1.0)[0], self.heights(0.0)[0]) + + def test_silence_is_a_flat_line_of_stubs_and_not_an_empty_box(self): + # The one thing the old meter and this one agree on, and the reason `BAR_MIN_H` + # is not zero: an empty widget reads as "not working", not as "quiet". + self.assertEqual(self.heights(0.0), [ui.BAR_MIN_H] * ui.BARS) + + def test_nothing_ever_draws_outside_the_pill(self): + # Half-heights, mirrored, inside a 40 px pill with 8 px of air. + for level in (0.0, 0.25, 0.5, 0.75, 1.0, 2.0, -1.0): + with self.subTest(level=level): + for h in self.heights(level): + self.assertGreaterEqual(h, ui.BAR_MIN_H) + self.assertLessEqual(h, ui.BAR_MAX_H) + + def test_ordinary_speech_reaches_most_of_the_way_up(self): + # What the 0.55 exponent buys. At half level a linear meter would draw half + # height, which is what made the old one look timid at conversational volume — + # the top of the widget was reserved for shouting. + half = self.heights(0.5)[ui.BARS // 2] + full = self.heights(1.0)[ui.BARS // 2] + self.assertGreater(half, full * 0.6) + + def test_every_bar_answers_the_same_level(self): + # The bloom's defining property, and the one a reintroduced history would break: + # the shape between bars is the envelope, never the past. + first = self.heights(0.7) + self.assertEqual(first, self.heights(0.7)) + + +class TestWhereThePanelOpens(unittest.TestCase): + """FluidVoice's `positionWindow` arithmetic, ported and asserted. + + Their rule reads oddly until you notice it uses *two* rectangles on purpose: + centred on `screen.frame` — the physical display — but stood on + `screen.visibleFrame`, which excludes the Dock. Windows hands back the same pair as + `rcMonitor` and `rcWork`, so this ports without being reinterpreted. + """ + + #: A 1920×1080 display with a 48 px taskbar, offset on a virtual desktop so that a + #: test cannot pass by assuming the origin is (0, 0) — which is the bug a + #: second-monitor user gets. + FULL = (1920, 0, 3840, 1080) + WORK = (1920, 0, 3840, 1032) + + def test_it_centres_on_the_physical_display(self): + x, _y = ui.bottom_centre(400, 100, self.FULL, self.WORK) + self.assertEqual(x, 1920 + (1920 - 400) // 2) + + def test_it_stands_on_the_work_area_and_not_the_screen_edge(self): + # The asymmetry that matters: centred on `full`, but lifted clear of the + # taskbar. Standing on `full` would put the panel under it. + _x, y = ui.bottom_centre(400, 100, self.FULL, self.WORK) + self.assertLessEqual(y + 100, self.WORK[3]) + + def test_the_offset_lifts_it_and_is_measured_from_the_bottom(self): + # Both offsets clear of the 10 px floor, so this measures the offset rather than + # the clamp. FluidVoice floors at `visibleFrame.minY + 10` the same way, which + # is why an offset of 0 and an offset of 10 land in the same place. + _x, near = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=20) + _x, far = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=120) + self.assertEqual(near - far, 100) + + def test_the_last_ten_pixels_are_a_floor_rather_than_a_range(self): + flush = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=0) + floored = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=10) + self.assertEqual(flush, floored) + + def test_an_offset_from_a_hand_edited_profile_cannot_push_it_off_screen(self): + # The clamp is what makes the offset safe to expose as a setting. Both ends, + # because a negative number is as easy to type as a huge one. + for offset in (-10_000, -1, 0, 900, 10_000): + with self.subTest(offset=offset): + x, y = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset) + self.assertGreaterEqual(y, self.WORK[1] + 40) + self.assertLessEqual(y + 100, self.WORK[3] - 10) + self.assertGreaterEqual(x, self.WORK[0]) + self.assertLessEqual(x + 400, self.WORK[2]) + + def test_a_panel_taller_than_the_display_keeps_its_bottom(self): + # The one case where the clamp cannot satisfy both ends. The bottom is the edge + # worth keeping: the top of a draft can run under the taskbar and still be read, + # and the chips that act on it live at the bottom. + _x, y = ui.bottom_centre(400, 5000, self.FULL, self.WORK) + self.assertEqual(y + 5000, self.WORK[3] - 10) + + def test_a_panel_wider_than_the_display_starts_at_its_left_edge(self): + x, _y = ui.bottom_centre(4000, 100, self.FULL, self.WORK) + self.assertEqual(x, self.WORK[0]) + + def test_the_monitor_under_the_pointer_answers_with_two_rectangles(self): + # `_work_area` asks `SystemParametersInfoW`, which only ever answers for the + # primary display — so on a two-monitor desk everything Flow drew landed on the + # wrong one whenever the user was working on the other. + full, work = ui._pointer_monitor(1280, 720) + for rect in (full, work): + self.assertEqual(len(rect), 4) + self.assertGreater(rect[2], rect[0]) + self.assertGreater(rect[3], rect[1]) + # The work area is the one the taskbar comes out of, so it can only be smaller. + self.assertLessEqual(work[3] - work[1], full[3] - full[1]) + self.assertLessEqual(work[2] - work[0], full[2] - full[0]) + + def test_it_degrades_to_the_primary_work_area_rather_than_raising(self): + # A repaint is not a place to handle a Win32 failure. `_NoHands` is this + # module's own idea of "nothing happened", and the fallback has to land + # somewhere drawable rather than at (0, 0) with no size. + # `create=True` because `ctypes.windll` does not exist off Windows at all, and + # this test asserts the degradation that matters most *on* those platforms. + with mock.patch.object(ui.ctypes, "windll", ui._NoHands(), create=True): + full, work = ui._pointer_monitor(1280, 720) + self.assertEqual(full, work) + self.assertGreater(full[2], full[0]) + + +class TestWhichPlacementIsInForce(unittest.TestCase): + """`PLACE`, and the two answers `_placed` gives. + + Restores the module afterwards for `tests/test_overlay.py`'s reason: this is a + global, and a test that moved it would outlive itself. + """ + + FULL = (1920, 0, 3840, 1080) + WORK = (1920, 0, 3840, 1032) + + def setUp(self): + self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) + self.p = ui.Pill.__new__(ui.Pill) + self.p.full, self.p.work = self.FULL, self.WORK + + def test_bottom_is_what_ships(self): + # Stated as a fact rather than left to the constant, because this is the change + # of default: the corner is where Windows puts the tray, every toast, and most + # apps' own status chrome, and it was the one place Flow had reserved. + self.assertEqual(ui.PLACE_DEFAULT, "bottom") + self.assertEqual(ui.PLACE, "bottom") + + def test_bottom_centres_the_stack_on_the_display(self): + ui.apply_place("bottom") + x, _y = self.p._placed(ui.PILL_W) + self.assertEqual(x, ui.bottom_centre(ui.PILL_W, ui.PILL_H, self.FULL, + self.WORK, ui.PANEL_BOTTOM_OFFSET)[0]) + + def test_corner_still_puts_it_bottom_right_where_it_always_was(self): + # Kept rather than removed: somebody who has spent months with the pill in the + # bottom right should not have it moved by an upgrade they did not ask for. + ui.apply_place("corner") + x, y = self.p._placed(ui.PILL_W) + self.assertEqual(x, self.WORK[2] - ui.PILL_W - 28) + self.assertEqual(y, self.WORK[3] - ui.PILL_H - 24) + + def test_the_two_placements_are_actually_different(self): + # A guard against both branches collapsing to the same arithmetic, which is how + # a setting comes to look like it works while doing nothing. + ui.apply_place("bottom") + here = self.p._placed(ui.PILL_W) + ui.apply_place("corner") + self.assertNotEqual(here, self.p._placed(ui.PILL_W)) + + def test_a_typo_costs_the_setting_and_not_the_app(self): + # This arrives from a hand-edited profile. Raising here would be a launch that + # dies on a misspelled word. + for name in ("bottomm", "", "BOTTOM", "centre", "left"): + with self.subTest(name=name): + ui.apply_place(name) + self.assertEqual(ui.PLACE, ui.PLACE_DEFAULT) + + def test_the_widths_it_is_asked_about_are_the_ones_it_is_placed_at(self): + # `_placed` takes the width because the pill grows to the panel's when one is + # docked. Centring a 420-wide stack using the 152-wide pill's position is how + # the stack would sit off-centre exactly when it is most visible. + ui.apply_place("bottom") + narrow, _ = self.p._placed(ui.PILL_W) + wide, _ = self.p._placed(ui.BUBBLE_W) + self.assertGreater(narrow, wide) + + +class TestTheStackFollowsThePointersMonitor(unittest.TestCase): + """The bug the placement work turned up, asserted so it cannot come back. + + `self.work` was read once in `__init__` from `SystemParametersInfoW`, which only + ever answers for the primary display. On a two-monitor desk that put every window + Flow drew against a screen the user might not be looking at — and pointed the + on-screen clamps at the wrong rectangle too. + """ + + ONE = ((0, 0, 1920, 1080), (0, 0, 1920, 1032)) + TWO = ((1920, 0, 3840, 1080), (1920, 0, 3840, 1032)) + + def pill(self, at): + p = ui.Pill.__new__(ui.Pill) + p.full, p.work = at + p.x, p.y = p._placed(ui.PILL_W) + p._docked_w = ui.PILL_W + p.canvas = Canvas() + # `pill_w` reads `front`, which reads `session.mode`. Set for the reason + # `test_lite`'s harness sets everything: `tk.Misc.__getattr__` forwards an + # unknown attribute to `self.tk`, so a missing one recurses rather than defaults. + p.session = mock.Mock(mode=DICTATE) + p.bubble = mock.Mock(_visible=False) + p.card = mock.Mock(_visible=False) + p.winfo_screenwidth = lambda: 1920 + p.winfo_screenheight = lambda: 1080 + p.window_geometry = lambda: (ui.PILL_W, p.x, p.y) + p.geometry = mock.Mock() + return p + + def test_moving_the_pointer_to_the_other_monitor_moves_the_stack(self): + p = self.pill(self.ONE) + was = (p.x, p.y) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.TWO): + p._sync_monitor() + self.assertNotEqual((p.x, p.y), was) + self.assertGreaterEqual(p.x, self.TWO[1][0]) + self.assertLessEqual(p.x + ui.PILL_W, self.TWO[1][2]) + + def test_a_pointer_that_has_not_left_the_monitor_costs_nothing(self): + # Every frame asks. Acting unconditionally would be a `geometry` call per frame + # forever — the same shape `_track_target` uses for `classify`, for the reason. + p = self.pill(self.ONE) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.ONE): + p._sync_monitor() + p.geometry.assert_not_called() + + def test_a_panel_that_is_up_is_moved_with_the_pill(self): + # The panels are placed *from* the pill, so moving it is the whole move — but + # only for a window somebody can see. + p = self.pill(self.ONE) + p.bubble = mock.Mock(_visible=True, width=ui.BUBBLE_W, _h=ui.PANEL_MAX_H) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.TWO): + p._sync_monitor() + p.bubble.reposition.assert_called_once() + p.card.reposition.assert_not_called() + + +class TestAHiddenPanelIsParkedRatherThanUnmapped(unittest.TestCase): + """FluidVoice's `parkWindowOffscreen`, and the one property it must have. + + The panels used to `withdraw()`. Push-to-talk made that the wrong trade: a panel now + has to be up between a key going down and somebody starting to speak, and a remap is + work done in exactly that gap. + """ + + #: Two monitors side by side, the left one primary. The union is what matters. + DESKTOP = (0, 0, 3840, 1080) + + def test_a_parked_panel_is_clear_of_every_monitor(self): + # The one thing that must never happen. Parking off the right edge of the *left* + # display in a two-monitor desk parks it in the middle of the right one, in full + # view, which is why this is the union and not the current screen. + x, y = ui.park_spot(420, 300, self.DESKTOP) + self.assertGreater(x, self.DESKTOP[2]) + self.assertGreater(y, self.DESKTOP[3]) + + def test_the_panel_clears_the_edge_by_its_own_size_as_well_as_the_margin(self): + # Its top-left corner being past the edge is not enough — the window extends + # right and down from there. + for w, h in ((205, 40), (420, 300), (640, 900)): + with self.subTest(w=w, h=h): + x, y = ui.park_spot(w, h, self.DESKTOP) + self.assertGreaterEqual(x - self.DESKTOP[2], w + ui.PARK_MARGIN) + self.assertGreaterEqual(y - self.DESKTOP[3], h + ui.PARK_MARGIN) + + def test_the_desktop_is_every_monitor_and_not_the_primary_one(self): + left, top, right, bottom = ui._virtual_desktop(1280, 720) + self.assertGreater(right, left) + self.assertGreater(bottom, top) + + def test_it_degrades_to_the_screen_rather_than_raising(self): + with mock.patch.object(ui.ctypes, "windll", ui._NoHands(), create=True): + self.assertEqual(ui._virtual_desktop(1280, 720), (0, 0, 1280, 720)) + + def test_parking_moves_the_window_and_never_unmaps_it(self): + # The distinction the whole change rests on: a `withdraw` here would cost a + # remap on the next hold, which is the gap push-to-talk has to fit inside. + win = mock.Mock(width=420, _h=300) + win.winfo_screenwidth.return_value = 1280 + win.winfo_screenheight.return_value = 720 + ui.park(win) + win.withdraw.assert_not_called() + win.geometry.assert_called_once() + asked = win.geometry.call_args[0][0] + self.assertTrue(asked.startswith("420x300+"), asked) + + def test_a_panel_with_no_height_yet_is_still_parked_somewhere_legal(self): + # `_h` is not set until the first render, and a hide can beat it — a zero-sized + # geometry request is one a window manager is entitled to refuse. + win = mock.Mock(width=420, spec=["width", "geometry", "winfo_screenwidth", + "winfo_screenheight"]) + win.winfo_screenwidth.return_value = 1280 + win.winfo_screenheight.return_value = 720 + ui.park(win) + self.assertTrue(win.geometry.call_args[0][0].startswith("420x1+")) + + +class TestTheWindowsOnlyAttributes(unittest.TestCase): + """`-transparentcolor` and `-toolwindow` exist on one platform and are fatal on the + others: `bad attribute "-transparentcolor"` out of Tk, raised before a window has + been drawn. + + The guard used to be `lite` alone, because `__main__` forces lite mode off Windows + (`lite = args.lite or sys.platform != "win32"`) so the two could not come apart. They + came apart as soon as something other than `__main__` built a `Pill`: + `scripts/mac_report.py` asked for full mode on a Mac and the report died in the + constructor. + """ + + def test_full_mode_off_windows_asks_for_neither(self): + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"): + self.assertEqual(ui._shell_window(win, lite=False, alpha=0.94), ui.SHELL) + asked = [c.args[0] for c in win.attributes.call_args_list] + self.assertNotIn("-transparentcolor", asked) + self.assertNotIn("-toolwindow", asked) + + def test_full_mode_on_windows_still_asks_for_both(self): + # The keyed colour is how the pill has no rectangle around it. Losing this on + # Windows would be a visible regression, not a quiet one. + win = mock.Mock() + with mock.patch.object(sys, "platform", "win32"): + self.assertEqual(ui._shell_window(win, lite=False, alpha=0.94), + ui.TRANSPARENT) + win.attributes.assert_any_call("-transparentcolor", ui.TRANSPARENT) + win.attributes.assert_any_call("-toolwindow", True) + + def test_the_shared_two_are_asked_for_everywhere(self): + for platform in ("darwin", "win32", "linux"): + with self.subTest(platform=platform): + win = mock.Mock() + with mock.patch.object(sys, "platform", platform): + ui._shell_window(win, lite=True, alpha=0.5) + win.attributes.assert_any_call("-topmost", True) + win.attributes.assert_any_call("-alpha", 0.5) + + +class TestTakingTheFrameOff(unittest.TestCase): + """`_bare_window`, and why Aqua does not get `overrideredirect`. + + Two faults reported from a Mac - click the app you want to dictate into and Flow's + window vanishes, and clicking Send does nothing - and one cause. Six variants were + put on screen and the results split on exactly this line: every window without + `overrideredirect` kept its place when another app came forward and had its button + reached by a click, and every window with it was deaf and gone. + + A style mask with no bits is the replacement. `titled` is the bit that puts a title + bar on, so a mask with none is bare, and nothing else about the window has been given + away. Measured at 0 px of decoration on a Mac against the control's 28. + """ + + def test_aqua_asks_for_an_empty_style_mask_and_not_overrideredirect(self): + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"): + ui._bare_window(win) + win.wm_attributes.assert_called_once_with("-stylemask", "") + win.overrideredirect.assert_not_called() + + def test_everywhere_else_is_unchanged(self): + # `-stylemask` is an Aqua attribute. Windows and X11 have never needed it, and + # `overrideredirect` is not the cause of anything there. + for platform in ("win32", "linux"): + with self.subTest(platform=platform): + win = mock.Mock() + with mock.patch.object(sys, "platform", platform): + ui._bare_window(win) + win.overrideredirect.assert_called_once_with(True) + win.wm_attributes.assert_not_called() + + def test_a_mac_on_tk_8_6_falls_back_rather_than_wearing_a_title_bar(self): + # `-stylemask` arrived in Tk 9. Older builds should get the behaviour they + # always had, which is imperfect but not a window with a frame on it. + win = mock.Mock() + win.wm_attributes.side_effect = ui.tk.TclError("bad attribute") + with mock.patch.object(sys, "platform", "darwin"): + ui._bare_window(win) + win.overrideredirect.assert_called_once_with(True) + + def test_every_window_flow_owns_goes_through_it(self): + # The pill, the bubble, the card and the help panel all build their shell here, + # and a window that missed this would be the one wearing a frame. + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui, "_bare_window") as bare: + ui._shell_window(win, lite=True, alpha=0.9) + bare.assert_called_once_with(win) + + +class TestTheWorkAreaOnAqua(unittest.TestCase): + """`_aqua_work_area`, which exists because the maximise probe is ignored on a Mac. + + A Mac reported `_tk_work_area()` answering with the whole 1352x878 screen, identical + to `_work_area()`. `state("zoomed")` had neither raised nor maximised — asked to + maximise a 200x120 window at +80+80 it returned the same window at +80+80, and no + error — so the fallback did not fall back, `bottom_centre` stood the pill 24 px above + 878, and the pill sat inside an 85 px Dock. + + `wm maxsize` is the instrument here and *only* here: Tk's Aqua port answers it from + `[NSScreen visibleFrame]`. On Windows the same call reports the whole screen with a + taskbar present, which is the reason the maximise probe exists at all. + """ + + def setUp(self): + self._was = ui._TK_WORK + ui._TK_WORK = None + self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) + + def win(self, maxsize=(1352, 735), title=28, menu=30): + """The numbers a 14-inch MacBook Pro actually reported, not invented ones. + + `scripts/mac_area_probe.py` on darwin, Tk 9.0.3, a 1352x878 screen: a titled + probe asked for +80+300 landing at y 328, so a 28 px title bar; the same probe + asked for +0+0 landing at y 58, so a 30 px menu bar under it; and `maxsize` + 1352x735, which plus the title bar is a 763 px visible frame. + """ + probe = mock.Mock() + probe.maxsize.return_value = maxsize + seen = [] + + def geometry(spec): + seen.append(spec) + + def rooty(): + # +80+300 first, then +0+0 - the order the function asks in. + return ui._AQUA_FREE_Y + title if len(seen) == 1 else menu + title + + probe.geometry.side_effect = geometry + probe.winfo_rooty.side_effect = rooty + return mock.Mock(), probe + + def call(self, win, probe, sw=1352, sh=878): + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + return ui._aqua_work_area(win, sw, sh) + + def test_the_bottom_is_the_top_of_the_dock(self): + # The whole point: 793, not 878. The pill stands on this number. + win, probe = self.win() + self.assertEqual(self.call(win, probe), (0, 30, 1352, 793)) + + def test_the_dock_it_finds_matches_the_dock_the_os_reports(self): + """85 px of Dock, against a `com.apple.dock tilesize` of 69 read separately. + + The reason this platform is believed at all now. `maxsize`, the title bar and the + menu bar are all Tk asking Tk; a tile size out of `defaults` came from somewhere + else entirely, and 69 plus Apple's padding is the 85 this leaves. Three earlier + guesses at Aqua all failed for want of a second source. + """ + win, probe = self.win() + self.assertEqual(878 - self.call(win, probe)[3], 85) + + def test_the_title_bar_is_added_back_to_the_content_size(self): + """The bug this shape was written to make impossible. + + `maxsize` is a maximum *content* size, short by whatever decoration its window + wears — 735 from a titled probe against 763 from the `overrideredirect` pill on + the same display. The first version took the origin from a probe and the size + from the caller's window, counted the 28 px title bar twice, and put the work + area at 821: the pill moved off the Dock and straight back onto it. + + One probe answers everything, so its decoration appears on both sides and + cancels. A window with no title bar at all must land on the same answer. + """ + win, bare = self.win(maxsize=(1352, 763), title=0) + self.assertEqual(self.call(win, bare), (0, 30, 1352, 793)) + + def test_the_probe_is_invisible_and_cleaned_up(self): + win, probe = self.win() + self.call(win, probe) + probe.attributes.assert_any_call("-alpha", 0.0) + probe.destroy.assert_called_once() + + def test_a_maxsize_of_the_whole_screen_means_it_does_not_know(self): + # Windows answers this way with a taskbar present. Reporting the whole screen as + # a work area is the bug, not a fix for it. + win, probe = self.win(maxsize=(1352, 878), title=0, menu=0) + self.assertIsNone(self.call(win, probe)) + + def test_a_window_manager_that_honoured_plus_zero_is_not_a_menu_bar(self): + # Windows puts a window where it is asked. A number far outside a menu bar's + # range is a literal placement, and nothing here can be believed. + win, probe = self.win(menu=400) + self.assertIsNone(self.call(win, probe)) + + def test_a_title_bar_too_tall_to_be_one_is_refused(self): + win, probe = self.win(title=200) + self.assertIsNone(self.call(win, probe)) + + def test_measurements_that_disagree_about_the_display_are_refused(self): + # A menu bar plus a visible frame cannot be taller than the display it is on. + win, probe = self.win(maxsize=(1352, 870)) + self.assertIsNone(self.call(win, probe)) + + def test_a_build_without_maxsize_says_so(self): + win, probe = self.win() + probe.maxsize.side_effect = ui.tk.TclError("no such command") + self.assertIsNone(self.call(win, probe)) + + def test_a_probe_that_cannot_be_built_says_so(self): + win, _probe = self.win() + with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): + self.assertIsNone(ui._aqua_work_area(win, 1352, 878)) + + def test_tk_work_area_prefers_it_on_darwin(self): + win, probe = self.win() + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 30, 1352, 793)) + + def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): + # The guarantee that made this safe to write before a Mac had confirmed it: if + # the Aqua path cannot answer, the worst case is exactly the old behaviour. + win, probe = self.win(maxsize=(1352, 878), title=0, menu=0) + probe.winfo_rootx.return_value = 0 + probe.winfo_rooty.side_effect = None + probe.winfo_rooty.return_value = 30 + probe.winfo_width.return_value = 1352 + probe.winfo_height.return_value = 763 + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 30, 1352, 793)) + + def test_windows_never_takes_this_path(self): + # `maxsize` there is the whole screen, taskbar or not. Asking it would undo the + # measurement this module went to the trouble of making. + win, probe = self.win() + with mock.patch.object(sys, "platform", "win32"), mock.patch.object(ui, "_aqua_work_area") as aqua, mock.patch.object(ui.tk, "Toplevel", return_value=probe): + probe.winfo_rootx.return_value = 0 + probe.winfo_rooty.side_effect = None + probe.winfo_rooty.return_value = 23 + probe.winfo_width.return_value = 1280 + probe.winfo_height.return_value = 649 + ui._tk_work_area(win, 1280, 720) + aqua.assert_not_called() + + +class TestTheWorkAreaOffWindows(unittest.TestCase): + """`_tk_work_area`, which exists because a Mac put the pill under the Dock. + + `_work_area` degrades to the whole screen off Windows, so bottom-centre placement + stood the stack on the very bottom edge — behind the Dock on macOS, behind the panel + on a bottom-taskbar Linux. + + The fix is a *measurement*: maximise a window and look at where the window manager + put it, since it has to honour its own panels to do that. The obvious call, + `wm_maxsize`, was tried first and is useless — on Windows it answers with the whole + screen even with a taskbar present, which is wrong in exactly the way this is meant + to fix. + """ + + def setUp(self): + # Module-level cache, so a test that measured would outlive itself. + self._was = ui._TK_WORK + ui._TK_WORK = None + self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) + + def fake_win(self, zoomed=(0, 23, 1280, 672)): + """A Tk stand-in whose `Toplevel` reports a maximised geometry.""" + x, y, r, b = zoomed + probe = mock.Mock() + probe.winfo_rootx.return_value = x + probe.winfo_rooty.return_value = y + probe.winfo_width.return_value = r - x + probe.winfo_height.return_value = b - y + return probe + + def test_it_reads_back_where_the_window_manager_put_a_maximised_window(self): + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 23, 1280, 672)) + + def test_the_probe_is_invisible_and_cleaned_up(self): + # Nothing may flash on screen, and a probe left alive is a stray window. + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + ui._tk_work_area(mock.Mock(), 1280, 720) + probe.attributes.assert_any_call("-alpha", 0.0) + probe.destroy.assert_called_once() + + def test_it_is_measured_once_and_then_remembered(self): + # `_sync_monitor` asks every frame. Measuring per frame would open and destroy a + # Toplevel thirty times a second. + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe) as made: + for _ in range(50): + ui._tk_work_area(mock.Mock(), 1280, 720) + made.assert_called_once() + + def test_a_build_without_zoomed_falls_back_to_the_whole_screen(self): + # `state("zoomed")` is documented for Windows and X11 and may not exist here. + # The whole screen is the honest answer then: wrong by a Dock, rather than wrong + # by whatever a broken measurement returned. + probe = self.fake_win() + probe.state.side_effect = ui.tk.TclError("bad state") + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 0, 1280, 720)) + probe.destroy.assert_called_once() + + def test_a_nonsense_measurement_is_refused(self): + # A window manager that hands back something larger than the screen, or inside + # out, has not answered the question. + for bad in ((0, 0, 4000, 672), (0, 0, 0, 0), (500, 0, 100, 672)): + with self.subTest(bad=bad): + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win(bad)): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 0, 1280, 720)) + + def test_a_probe_that_was_never_maximised_is_refused(self): + # The one that got through, and put the pill in the top-left corner of a Mac. + # `state("zoomed")` on Aqua does not raise and does not maximise either: it is + # accepted and ignored, so the probe stayed the 200x120 it was asked for and + # that rectangle was believed. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((80, 80, 280, 200))): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1512, 982), + (0, 0, 1512, 982)) + + def test_a_window_grown_only_a_little_is_refused_too(self): + # The other hole: a window manager that honoured `zoomed` partially. A real work + # area is the screen minus a Dock or a taskbar, nowhere near half of it. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((0, 0, 700, 400))): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1512, 982), + (0, 0, 1512, 982)) + + def test_the_fallback_still_puts_the_stack_on_screen(self): + # Refusing the measurement must not mean refusing to place anything. The whole + # screen is wrong by a Dock; the top-left corner is wrong by a screen. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((80, 80, 280, 200))): + work = ui._tk_work_area(mock.Mock(), 1512, 982) + x, y = ui.bottom_centre(ui.PILL_W, ui.PILL_H, (0, 0, 1512, 982), work, + ui.PANEL_BOTTOM_OFFSET) + self.assertGreater(y, 982 * 0.8, "the stack is nowhere near the bottom") + self.assertGreater(x, 1512 * 0.3, "the stack is nowhere near the centre") + + def test_the_bottom_edge_is_the_one_that_has_to_be_right(self): + # It is what bottom-centre placement stands on, and getting it wrong is the + # whole bug: 672 here against a screen of 720 is a 48 px Dock found. + with mock.patch.object(ui.tk, "Toplevel", return_value=self.fake_win()): + work = ui._tk_work_area(mock.Mock(), 1280, 720) + _x, y = ui.bottom_centre(ui.PILL_W, ui.PILL_H, (0, 0, 1280, 720), work, + ui.PANEL_BOTTOM_OFFSET) + self.assertLessEqual(y + ui.PILL_H, 672) + + +class TestTheMacFrame(unittest.TestCase): + """What strips the frame on Aqua, asserted as the absence of two wrong answers. + + A Mac reported the pill wearing a title bar and three traffic lights while the + panels above it were bare, and two fixes were tried before the right one. Both are + pinned here because both looked correct and each broke something else: + + **`MacWindowStyle plain` is not frameless.** `plain` is a window *class*, and a + Toplevel given only that style comes up decorated - settled on a real machine by + `scripts/mac_frame_probe.py`. Asked for on a mapped window it put the frame back. + + **`noActivates` is worse.** It takes the window out of the activation chain, and a + window that never activates does not take clicks: Send stopped working. `_menu` + already depended on the opposite, and says so. + + What was left after removing both is the line that had been doing the work all + along, which is why this class asserts an absence rather than a mechanism. + """ + + def source(self) -> str: + return (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + + def test_nothing_asks_aqua_for_a_window_class(self): + # Asserted over the source because the mistake is *making the call at all*, and + # a mock cannot notice a call that is no longer there. The name still appears in + # prose explaining why it is not used - those comments are the point of this + # test, so what is checked is the invocation. + for line in self.source().splitlines(): + stripped = line.strip() + if stripped.startswith("#") or stripped.startswith("*"): + continue + with self.subTest(line=stripped[:60]): + self.assertNotIn("::tk::unsupported::", line) + + def test_no_window_is_withdrawn_and_remapped_to_restyle_it(self): + # The second wrong answer. It was solving a problem the first one created, and + # on a real Mac it left the window hidden after the remap - shown, then gone. + self.assertNotIn("_mac_reframe", self.source()) + + def test_no_activate_refuses_off_windows_because_the_app_depends_on_it(self): + # `_menu` borrows the foreground on Windows precisely because a non-activating + # window gets no input for its popup, and states that Lite needs none of that. + # Asking Aqua for the equivalent broke Send. + for platform in ("darwin", "linux"): + with self.subTest(platform=platform): + with mock.patch.object(ui.sys, "platform", platform): + self.assertFalse(ui._no_activate(mock.Mock())) + + def test_the_shell_still_asks_for_the_one_thing_that_works(self): + win = mock.Mock() + ui._shell_window(win, lite=True, alpha=0.94) + win.overrideredirect.assert_called_once_with(True) + + class TestMix(unittest.TestCase): def test_the_ends_are_exact(self): self.assertEqual(ui._mix(ui.HEARING, ui.CARD_ACCENT, 0.0), ui.HEARING) @@ -432,9 +1174,12 @@ def docker(*, showing=True, panel_w=ui.BUBBLE_W, window=None, x=1047, docked_w=u p = ui.Pill.__new__(ui.Pill) p.canvas = mock.Mock() p.session = mock.Mock(mode=DICTATE) - p.bubble = mock.Mock(width=panel_w, _visible=showing) - p.card = mock.Mock(width=panel_w, _visible=False) + # `_h` is a real int: the shell is sized from the band's actual height now, so a + # Mock there would put a Mock into the arithmetic. + p.bubble = mock.Mock(width=panel_w, _visible=showing, _h=ui.PANEL_MAX_H) + p.card = mock.Mock(width=panel_w, _visible=False, _h=ui.PANEL_MAX_H) p.work = (0, 0, 1280, 720) + p.full = (0, 0, 1280, 720) p.x, p.y = x, 608 p._docked_w = docked_w p.geometry = mock.Mock() @@ -443,51 +1188,122 @@ def docker(*, showing=True, panel_w=ui.BUBBLE_W, window=None, x=1047, docked_w=u return p -class TestTheDockIsCheckedRatherThanAssumed(unittest.TestCase): - """The pill and the panel above it share one column, and stay sharing it. +class TestTheShellIsOneWindow(unittest.TestCase): + """`_sync_shell`, which is what `_sync_dock` became when the dock stopped existing. - `scripts/reel.py` found them not sharing it: the pill 420 px wide at the x a - 205 px pill sits at, hanging 215 px off the screen and unjoined from the panel it - is docked to, held there for five seconds because the width already matched and - nothing looked again. + The pill and its panel used to be two windows kept adjacent by hand, and + `scripts/reel.py` once caught them **215 px apart, for five seconds**: the resize had + landed and the matching move had not, and the pill's remembered width then answered + "nothing to do" on every frame after. That failure needs two windows to be possible. + There is one now, and all that arithmetic has collapsed into a height. + + What the two classes here used to cover — holding the right edge in corner placement, + re-centring on the new width in bottom placement, the 107 px lurch when a draft + appeared — is gone with the width change that caused it. The pill is the panel's + width whether a panel is up or not. """ - def test_a_panel_appearing_moves_the_left_edge_and_holds_the_right(self): - p = docker() - p._sync_dock() - p.geometry.assert_called_once_with(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608") - self.assertEqual(p.x + ui.BUBBLE_W, 1047 + ui.PILL_W) # the right edge did not move + def setUp(self): + self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) + ui.apply_place("bottom") + + def asked(self, p): + w, _, rest = p.geometry.call_args.args[0].partition("x") + h, _, _pos = rest.partition("+") + return int(w), int(h) + + def test_a_panel_opening_grows_the_window_upward(self): + # The band reports its own height now that it is snug around its content, so the + # shell is the row plus whatever that is — bounded by the ceiling. + p = docker(showing=True) + p._sync_shell() + self.assertEqual(self.asked(p), (ui.BUBBLE_W, ui.PANEL_MAX_H + ui.PILL_H)) + + def test_the_shell_follows_the_band_rather_than_its_ceiling(self): + # A shell sized to the ceiling leaves the row floating below a shorter band, + # which is the detached-boxes look the merge exists to end. Caught in a shot. + p = docker(showing=True) + p.bubble._h = 130 + p._sync_shell() + self.assertEqual(self.asked(p), (ui.BUBBLE_W, 130 + ui.PILL_H)) + + def test_and_the_foot_does_not_move_when_it_does(self): + """The whole of "the controls stay where they are", as arithmetic. + + Send, the meter and the chip row are laid out from the bottom of the window. A + shell that grew downward — or that centred its growth — would move every one of + them every time a draft appeared, which is the motion this work exists to end. + """ + idle = docker(showing=False) + idle._sync_shell() + opened = docker(showing=True) + opened._sync_shell() + self.assertEqual(idle.y + idle._shell_h, opened.y + opened._shell_h) + + def test_an_idle_pill_is_just_the_row(self): + p = docker(showing=False) + p._sync_shell() + self.assertEqual(p._shell_h, ui.PILL_H) def test_nothing_is_asked_for_twice_when_the_window_already_agrees(self): - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 832, 608)) - p._sync_dock() - p.geometry.assert_not_called() - - def test_a_move_that_did_not_land_is_asked_for_again(self): - # The reel's finding, staged: the resize took and the move did not, so the - # window is at the bare-pill x while the pill's own state says it docked. - # Before this was checked, `w == self._docked_w` returned here and the pill - # stayed off the screen edge for as long as the panel was up. - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 1047, 608)) - p._sync_dock() - p.geometry.assert_called_once_with(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608") - - def test_the_recovery_does_not_move_the_pill_a_second_time(self): - # Re-asking must re-send the *same* geometry, never re-run the relative - # arithmetic — that would walk the pill one panel-width left per frame. - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 1047, 608)) + # Idempotent, so it can run from the frame pump and from a panel's `reposition` + # without either caring which got there first. + p = docker(showing=False, x=538) + p._sync_shell() + p.window_geometry = mock.Mock(return_value=(ui.BUBBLE_W, p.x, p.y)) + p.geometry.reset_mock() for _ in range(5): - p._sync_dock() - self.assertEqual(p.x, 832) - self.assertEqual({c.args for c in p.geometry.call_args_list}, - {(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608",)}) - - def test_the_panel_going_away_takes_the_width_back(self): - p = docker(showing=False, docked_w=ui.BUBBLE_W, x=832) - p._sync_dock() - self.assertEqual(p.x, 1047) - p.geometry.assert_called_once_with(f"{ui.PILL_W}x{ui.PILL_H}+1047+608") - + p._sync_shell() + p.geometry.assert_not_called() -if __name__ == "__main__": + def test_a_window_that_drifted_is_asked_again(self): + # `reel.py`'s defect, in the only form it can still take. Compared against the + # *window* rather than against a remembered value: state saying the move happened + # is not evidence that it did. + p = docker(showing=True, window=(ui.BUBBLE_W, 4, 4)) + p._sync_shell() + p.geometry.assert_called_once() + + def test_the_shell_is_clamped_onto_the_screen(self): + p = docker(showing=True) + p.work = p.full = (0, 0, 300, 200) + p._sync_shell() + self.assertGreaterEqual(p.x, 0) + self.assertGreaterEqual(p.y, 0) + self.assertLessEqual(p.y + p._shell_h, 200) + + def test_a_panel_size_change_takes_the_row_with_it(self): + """The panel-size setting rebinds `BUBBLE_W` while Flow is running, so the width + changes with nothing else changing beside it. + + Left out of the comparison, the row kept the width it was built at while the band + above it took the new one — two boxes of different widths stacked in one window, + which is what a screenshot of "panel size: larger" showed. `_docked_w` is what + `_draw` measures the row against, so it moves in the same breath as the canvas. + """ + self.addCleanup(ui.apply_panel_width, ui.PANEL_WIDTHS["regular"]) + p = docker(showing=False, x=430) + p._sync_shell() + p.geometry.reset_mock() + ui.apply_panel_width(ui.PANEL_WIDTHS["larger"]) + p.window_geometry = mock.Mock(return_value=(ui.PANEL_WIDTHS["regular"], + p.x, p.y)) + p._sync_shell() + self.assertEqual(p._docked_w, ui.PANEL_WIDTHS["larger"]) + self.assertEqual(p.canvas.place.call_args.kwargs["width"], + ui.PANEL_WIDTHS["larger"]) + self.assertIn(str(ui.PANEL_WIDTHS["larger"]), p.geometry.call_args.args[0]) + + def test_the_row_is_placed_at_the_foot_of_whatever_height_it_is(self): + # The canvas is the bottom band of the window, not the whole of it — which is + # what makes "the foot never moves" true of the pixels and not just of the frame. + for showing in (False, True): + with self.subTest(showing=showing): + p = docker(showing=showing) + p._sync_shell() + kw = p.canvas.place.call_args.kwargs + self.assertEqual(kw["y"] + kw["height"], p._shell_h) + + +if __name__ == "__main__": # pragma: no cover unittest.main() diff --git a/tests/test_refine.py b/tests/test_refine.py index f5d8398..6cd141b 100644 --- a/tests/test_refine.py +++ b/tests/test_refine.py @@ -1510,3 +1510,99 @@ def test_nothing_dictated_reaches_the_argv_of_either(self): def test_the_one_that_was_not_measured_still_carries_it_on_the_argv(self): self.assertFalse(refine_mod.named("kiro-cli").stdin_ok) + +class TestTuning(unittest.TestCase): + """`tuned`, which is how a model and an effort level reach the CLI. + + Both flags were read out of each CLI's own `--help` on a machine that has all three, + on 2026-08-31, the same discipline `verified` carries for the invocation shapes + themselves. codex prints `-m, --model ` and no effort flag at all; claude and + kiro-cli print `--model` and `--effort (low, medium, high, xhigh, max)`. + """ + + def cli(self, name: str) -> refine_mod.Cli: + found = refine_mod.named(name) + assert found is not None + return found + + def test_effort_is_asked_for_by_default_and_it_is_the_cheapest(self): + # These calls are a rewrite, not a reasoning problem, and the user is watching a + # spinner while they run. + argv = refine_mod.tuned(self.cli("kiro-cli")).argv + self.assertIn("--effort", argv) + self.assertEqual(argv[argv.index("--effort") + 1], "low") + self.assertEqual(refine_mod.EFFORT_DEFAULT, "low") + + def test_the_levels_are_the_ones_both_clis_print(self): + self.assertEqual(refine_mod.EFFORTS, ("low", "medium", "high", "xhigh", "max")) + + def test_the_model_goes_in_with_the_flag_that_cli_takes(self): + self.assertIn(("-m", "gpt-5"), self.pairs("codex", model="gpt-5")) + self.assertIn(("--model", "gpt-5"), self.pairs("claude", model="gpt-5")) + self.assertIn(("--model", "gpt-5"), self.pairs("kiro-cli", model="gpt-5")) + + def pairs(self, name, **kw): + argv = refine_mod.tuned(self.cli(name), **kw).argv + return list(zip(argv, argv[1:])) + + def test_codex_is_not_given_an_effort_flag_it_does_not_have(self): + # Its only route is `-c model_reasoning_effort=...`, a config key that does not + # appear in its help. Writing one down from memory is what `verified` forbids. + self.assertNotIn("--effort", refine_mod.tuned(self.cli("codex"), effort="max").argv) + + def test_the_flags_land_after_the_subcommand(self): + # `exec` and `chat` are subcommands; a flag before them belongs to a different + # parser and the call fails to start. + for name, sub in (("codex", "exec"), ("kiro-cli", "chat")): + with self.subTest(name): + argv = refine_mod.tuned(self.cli(name), model="m").argv + self.assertEqual(argv[1], sub) + + def test_codex_keeps_the_stdin_marker_last(self): + """The trap that decided `tune_at`. + + codex's argv finishes with `-`, the positional saying the prompt is on stdin. A + flag appended after it is read as its value, and the prompt is never sent. + """ + argv = refine_mod.tuned(self.cli("codex"), model="gpt-5").argv + self.assertEqual(argv[-1], "-") + + def test_nothing_is_invented_for_a_cli_that_takes_neither(self): + # A user who picks a model while an unverified CLI answers should get that CLI + # answering, not a crash and not a guessed flag. + plain = self.cli("gemini") + self.assertIs(refine_mod.tuned(plain, model="gpt-5", effort="max"), plain) + + def test_asking_for_nothing_returns_the_very_same_object(self): + codex = self.cli("codex") + self.assertIs(refine_mod.tuned(codex, effort="default"), codex) + + def test_a_tuned_cli_is_still_that_cli(self): + # `_clean`'s per-CLI stripping, the pill's marker and the trace all key off the + # name. A tuned copy is the same CLI with a flag on it, not another one. + kiro = self.cli("kiro-cli") + copy = refine_mod.tuned(kiro, model="gpt-5") + self.assertEqual(copy.name, kiro.name) + self.assertEqual(copy.marker, kiro.marker) + self.assertEqual(copy.timeout_sec, kiro.timeout_sec) + self.assertEqual(copy.stdin_ok, kiro.stdin_ok) + self.assertEqual(copy.argv[0], kiro.argv[0]) + + def test_a_fallback_is_asked_for_the_same_model(self): + """A walk that reverted to the CLI's own defaults the moment the first candidate + failed would be slowest exactly when the user is already waiting longest.""" + seen = [] + + def record(cli, prompt, **kw): + seen.append(cli.argv) + return (None, "nope") if len(seen) == 1 else ("done", "") + + clis = [self.cli("claude"), self.cli("kiro-cli")] + with mock.patch.object(refine_mod, "available", return_value=clis), \ + mock.patch.object(refine_mod, "_invoke", side_effect=record): + refine_mod._invoke_any(None, "hi", timeout=1.0, model="gpt-5", effort="max") + self.assertEqual(len(seen), 2) + for argv in seen: + self.assertIn("gpt-5", argv) + self.assertIn("max", argv) + diff --git a/tests/test_talk.py b/tests/test_talk.py new file mode 100644 index 0000000..5fe0e89 --- /dev/null +++ b/tests/test_talk.py @@ -0,0 +1,741 @@ +"""Push-to-talk end to end: a hold on the keyboard, a paste out the other side. + +`tests/test_chord.py` covers the hook's state machine and stops at the queue. This picks +the same gesture up at the queue and follows it through `Pill`'s dispatch into a real +`Session`, because everything that can actually go wrong with push-to-talk lives in the +seam between them: + + **The release cannot paste.** A final decode measured 0.7-7 s on the machine this was + built for, so the release arms a wait and the frame loop finishes the gesture. Every + edge case below is some way that wait can be wrong — the decode that never lands, the + release that never arrives, the hold that starts while another is still waiting. + + **What was said is never dropped to make the code simpler.** A hold Windows took over + with `ctrl+win+d` still commits its audio; it just does not paste. The draft is where + words wait, and the Send chip is what the user already knows. + +The microphone and the decoder are fakes, and the decode is driven by hand, so a test +can sit exactly in the half-second between a release and the final that answers it. +""" + +import queue +import sys +import time +import unittest +from pathlib import Path +from unittest import mock + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +if sys.platform != "win32": # pragma: no cover - the CI legs that are not Windows + raise unittest.SkipTest("Windows-only: flow.hotkey binds user32 at import") + +import flow.ui as ui # noqa: E402 +from flow.hotkey import Chord # noqa: E402 +from flow.session import BLOCK, Session # noqa: E402 + + +class FakeMic: + """Counts opens and closes, because the gesture is judged on them. + + `start`/`stop` are the pair push-to-talk has to get exactly right: a hold that opens + a microphone it does not close is the failure mode the whole `PTT_MAX_HOLD_SEC` + branch exists for, and one that closes a microphone it did not open is the chord + walking off with the toggle hotkey's session. + """ + + def __init__(self) -> None: + self.starts = 0 + self.stops = 0 + self.active = False + self.dropped = 0 + self.device_name = "fake" + self._blocks: list[np.ndarray] = [] + + def start(self) -> None: + self.starts += 1 + self.active = True + + def stop(self) -> None: + self.stops += 1 + self.active = False + + def drain(self) -> list: + out, self._blocks = self._blocks, [] + return out + + def alive(self) -> bool: + return self.active + + +class FakeAsr: + loading = False + loaded = False + + def load(self, final=None) -> None: + self.loaded = True + + def unload(self) -> None: + self.loaded = False + + def text(self, audio, *, final=False, hotwords="") -> str: + return "" + + +class Harness: + """A `Pill` with the four attributes push-to-talk touches, and nothing drawn. + + Built with `__new__` for `tests/test_lite.py`'s reason, word for word: `tk.Misc` + forwards an unknown attribute to `self.tk`, so on an instance whose `__init__` never + ran a missing attribute recurses instead of defaulting. Every attribute the paths + under test read is set here on purpose. + """ + + def __init__(self, *, lite: bool = False, injector: bool = False) -> None: + self.mic = FakeMic() + # `Pill.converse` is a property reading `session.mode`, so the surface follows + # the session here exactly as it does in the app — there is nothing to set. + self.session = Session(asr=FakeAsr(), mic=self.mic) + self.pasted: list[tuple[str, bool]] = [] + self.notes: list[str] = [] + + p = ui.Pill.__new__(ui.Pill) + p.session = self.session + p.lite = lite + + # An injector is what `__main__` decides by importing a paste module, and it is + # no longer the same question as `lite`: a Mac is Lite and pastes through System + # Events. The full body always has one; Lite has one only where a platform gives + # Flow something to paste with. + p.on_send = self._on_send if (injector or not lite) else None + p.paste_target = 0x22 + p.bubble = mock.Mock() + p.card = mock.Mock() + p.bubble.note.side_effect = self.notes.append + p.card.note.side_effect = self.notes.append + p.armed = False + p._flash = 0 + p._disarmed_since = 0.0 + p._ptt_since = None + p._ptt_wait = None + p._draw = lambda: None + p.clipboard_clear = lambda: None + p.clipboard_append = lambda _t: None + p.update_idletasks = lambda: None + self.pill = p + + self.presses: queue.Queue[str] = queue.Queue() + self.chord = Chord(self.presses, frozenset({"ctrl", "win"})) + + def _on_send(self, text: str, target=None, submit: bool = False) -> str: + self.pasted.append((text, submit)) + return "" + + # -- driving ----------------------------------------------------------- + + def dispatch(self) -> None: + """Drain the chord's queue through `Pill`'s own dispatch table. + + The words are re-listed here rather than calling `_frame`, which would want a + Tk canvas. The tripwire in `test_chord.py` is what keeps this list honest: it + asserts each of the four words appears in `ui.py` as a dispatch literal. + """ + while not self.presses.empty(): + name = self.presses.get_nowait() + if name == "warm": + self.session.warm() + elif name == "talk": + self.pill._talk_start() + elif name == "talk-end": + self.pill._talk_end(send=True) + elif name == "talk-break": + self.pill._talk_end(send=False) + elif name == "send": + self.pill._send() + elif name == "cancel": + self.pill._clear() + elif name == "mode": + self.pill._ptt_wait = None + self.session.toggle_mode() + elif name == "toggle": + self.pill._toggle() + self.pump_events() + + def key(self, name: str) -> "Harness": + """Press one of the registered hotkeys, through the same dispatch the chord uses. + + The five combos and the chord's four words land on one queue by design — the + session cannot tell them apart, and neither should a test of what happens when + two of them mean overlapping things. + """ + self.presses.put(name) + self.dispatch() + return self + + def pump_events(self) -> None: + """The one event this gesture depends on, handled the way `_pump_events` does. + + `talk_end` closes the microphone and *asks* the pill to disarm rather than + reaching into it: `armed` belongs to the UI thread. In the app the real + `_pump_events` runs in the same frame as the dispatch, so the pill is never seen + claiming to listen with a closed device — reproducing that ordering here is what + makes this harness worth trusting. + """ + for ev in self.session.events(): + if ev.kind == "disarm": + self.pill.armed = False + + def settle(self, timeout: float = 5.0) -> "Harness": + """Wait for the real decode worker to finish what the release handed it. + + The worker is a live thread even with a fake transcriber, so `session.busy` is + genuinely true for a moment after a release. Waiting rather than patching keeps + the ordering under test real: the paste must follow the final, and a test that + stubbed `busy` to False could not tell the difference. + """ + end = time.perf_counter() + timeout + while self.session.busy and time.perf_counter() < end: + time.sleep(0.005) + self.session.pump_results() + self.pump_events() + return self + + def press(self) -> "Harness": + self.chord._talking = True + self.presses.put("warm") + self.presses.put("talk") + self.dispatch() + return self + + def release(self, *, clean: bool = True) -> "Harness": + self.presses.put("talk-end" if clean else "talk-break") + self.dispatch() + return self + + def speak(self, seconds: float = 1.0) -> "Harness": + """Put audio in the utterance buffer the way `_pump_audio` would.""" + blocks = max(1, int(seconds * 16000 / BLOCK)) + self.session._utter = [np.zeros(BLOCK, dtype=np.float32)] * blocks + return self + + def decode(self, text: str) -> "Harness": + """Land a final decode: let the worker drain, then put the words on the draft.""" + self.settle() + self.session.draft.set(text) + return self + + def frame(self) -> "Harness": + self.pill._pump_talk() + return self + + +class TestTheHoldOpensAndTheReleaseCloses(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_press_opens_the_microphone_and_the_release_closes_it(self): + self.h.press() + self.assertTrue(self.h.mic.active) + self.assertTrue(self.h.pill.armed) + self.h.speak().release() + self.assertFalse(self.h.mic.active) + + def test_the_press_warms_before_it_captures(self): + # The reason the warm is a separate word. By the time capture is open the models + # have been asked for, so the load overlaps the utterance instead of following + # it — which is what the 1 230 ms first partial in the trace was. + self.assertFalse(self.h.session.asr.loaded) + self.h.press() + self.assertTrue(self.h.session.asr.loaded) + + def test_a_second_press_does_not_reopen_a_microphone_already_open(self): + # Key repeat, or a `talk` that raced a slow frame. Reopening mid-utterance would + # be indistinguishable to the user from Flow having lost what they just said. + self.h.press() + self.h.press() + self.assertEqual(self.h.mic.starts, 1) + + def test_a_release_with_no_hold_behind_it_does_nothing(self): + # The OS can deliver a keyup whose keydown this process never saw — a chord + # begun before Flow launched, or while a UAC prompt owned the input desktop. + # It must not close a microphone the toggle hotkey opened. + self.h.session.start() + self.assertTrue(self.h.mic.active) + self.h.release() + self.assertTrue(self.h.mic.active) + + +class TestTheChordGivesBackExactlyWhatItTook(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_a_hold_over_an_already_armed_session_leaves_it_armed(self): + # Somebody armed with the toggle hotkey for long-form dictation, then reached + # for the chord. The release sends what they said; it does not switch off the + # microphone they turned on by another route. + self.h.session.start() + opens = self.h.mic.starts + self.h.press() + self.assertEqual(self.h.mic.starts, opens) + self.h.speak().release() + self.assertTrue(self.h.mic.active) + self.assertEqual(self.h.mic.stops, 0) + + def test_and_a_hold_that_opened_it_closes_it(self): + self.h.press().speak().release() + self.assertEqual(self.h.mic.stops, 1) + self.assertFalse(self.h.pill.armed) + + +class TestWhatWasSaidSurvivesEveryPath(unittest.TestCase): + """P2, on the one path with the most reason to cut a corner. + + `pause()` bumps the capture generation, which is precisely how a deliberate stop + refuses a decode from before it — so a `talk_end` written with `pause()` would throw + away the utterance the release exists to send. `_give_up_on_device` learned this + first and the comment there is the longer version. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_release_does_not_refuse_the_decode_it_just_asked_for(self): + before = self.h.session._capture_generation + self.h.press().speak().release() + self.assertEqual(self.h.session._capture_generation, before) + + def test_the_utterance_reaches_the_decoder(self): + self.h.press().speak(2.0) + self.assertTrue(self.h.session._utter) + self.h.release() + self.assertFalse(self.h.session._utter) # committed, not abandoned + self.assertTrue(self.h.session._sent) + + def test_a_broken_hold_keeps_the_words_and_pastes_nothing(self): + # ctrl+win+d after somebody had already started talking. The desktop switch is + # not a reason to lose a sentence, and it is not a reason to paste one into the + # window the switch just moved to either. + self.h.press().speak(2.0).release(clean=False) + self.assertTrue(self.h.session._sent) + self.h.decode("what they said").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "what they said") + + def test_a_desktop_switch_nobody_spoke_into_is_invisible(self): + # The common case, and the one that must not produce a note. `_utter` is what + # the gate let through, and in the 50 ms before the third key that is nothing — + # so there is no minimum-length rule here, and nothing to say. + self.h.press().release(clean=False) + self.assertFalse(self.h.session._sent) + self.assertEqual(self.h.notes, []) + self.assertEqual(self.h.pasted, []) + + +class TestThePasteWaitsForTheDecode(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_nothing_is_pasted_while_the_decoder_is_still_working(self): + # The bug this ordering prevents is pasting the *partial* — the dimmed, italic, + # possibly-hallucinated text that precedes a final by design. + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.decode("half a sentence").frame() + self.assertEqual(self.h.pasted, []) + + def test_and_it_pastes_the_moment_the_final_lands(self): + self.h.press().speak().release() + self.h.decode("the whole sentence").frame() + self.assertEqual(self.h.pasted, [("the whole sentence", False)]) + + def test_a_hold_nobody_spoke_into_never_arms_a_wait(self): + # An accidental tap. Without this the next decode from any source would be + # pasted by a wait that had been sitting open since the tap. + self.h.press().release() + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("something said later").frame() + self.assertEqual(self.h.pasted, []) + + def test_an_empty_decode_pastes_nothing(self): + # Silence, noise, or a hallucination `clean.py` rejected. Pasting "" would clear + # a selection in the user's window for no reason. + self.h.press().speak().release() + self.h.decode("").frame() + self.assertEqual(self.h.pasted, []) + + def test_the_wait_is_disarmed_once_it_fires(self): + # Otherwise the next thing decoded — a partial from a later utterance, an + # answer — would be pasted again by a wait nobody rearmed. + self.h.press().speak().release() + self.h.decode("first").frame() + self.h.decode("second").frame() + self.assertEqual(self.h.pasted, [("first", False)]) + + def test_a_new_hold_supersedes_a_wait_that_is_still_open(self): + # Somebody released, got impatient, and pressed again. The old wait must not + # fire into the new utterance — and the words it was waiting for stay in the + # draft rather than being discarded. + self.h.press().speak().release() + self.h.press() + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("from the first hold").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "from the first hold") + + +class TestTheTwoTimeoutsAreNotHypothetical(unittest.TestCase): + """Both branches of `_pump_talk`, which exist because this app has already wedged. + + The trace from the night this was written ends with `state -> idle` and no `final` + behind it. A paste wait with no ceiling turns that into a gesture that never + completes; a hold with no ceiling turns a dropped keyup into a microphone left open. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_a_decode_that_never_lands_gives_up_and_says_where_the_words_are(self): + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.pill._ptt_wait -= ui.PTT_PASTE_WAIT_SEC + 1 + self.h.frame() + self.assertEqual(self.h.pasted, []) + self.assertIsNone(self.h.pill._ptt_wait) + self.assertIn("Press Send", " ".join(self.h.notes)) + + def test_and_does_not_paste_it_late_when_it_finally_arrives(self): + # A paste a minute after the gesture lands in whatever window the user has moved + # to since, which is worse than not pasting at all. + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.pill._ptt_wait -= ui.PTT_PASTE_WAIT_SEC + 1 + self.h.frame() + self.h.decode("very late").frame() + self.assertEqual(self.h.pasted, []) + + def test_a_hold_whose_release_never_arrives_stops_on_its_own(self): + # A hook the OS dropped for overrunning `LowLevelHooksTimeout`, a lock screen, an + # RDP session taking the keyboard. Without this the microphone stays open. + self.h.press().speak(2.0) + self.h.pill._ptt_since -= ui.PTT_MAX_HOLD_SEC + 1 + self.h.frame() + self.assertFalse(self.h.mic.active) + self.assertIsNone(self.h.pill._ptt_since) + + def test_and_keeps_what_was_said_rather_than_pasting_it(self): + self.h.press().speak(2.0) + self.h.pill._ptt_since -= ui.PTT_MAX_HOLD_SEC + 1 + self.h.frame() + self.assertTrue(self.h.session._sent) + self.h.decode("a long dictation").frame() + self.assertEqual(self.h.pasted, []) + self.assertIn("press Send", " ".join(self.h.notes)) + + def test_an_ordinary_hold_is_nowhere_near_the_ceiling(self): + # A guard on the number rather than on the branch: two minutes has to sit clear + # of the longest hold anybody would make on purpose. + self.h.press().speak(2.0) + self.h.frame() + self.assertTrue(self.h.mic.active) + self.assertIsNotNone(self.h.pill._ptt_since) + + +class TestOneHoldOwnsOneSend(unittest.TestCase): + """The collisions, which are the part of push-to-talk that is genuinely hard. + + There are four ways to send — the Send chip, the `send` hotkey, the spoken trigger + routed as a `send` event, and converse's auto-ask countdown — and a release that has + armed a paste is a fifth thing waiting to do the same job. Any two of them firing for + one utterance is a double paste into somebody's editor. + + The rule is that a hold owns *one* send and whoever gets there first has it, and it + is enforced at the single point all five have in common rather than at five guards + that would have to be kept in step. These are the tests that say so. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_spoken_send_word_does_not_paste_twice(self): + # The one the whole rule exists for. Hold the chord, say "…and that's the plan, + # boom", let go: the trigger fires a send when the decode routes, while the + # release is still waiting to fire its own. + self.h.press().speak().release() + self.h.decode("that's the plan") + self.h.pill._send() # the trigger, arriving as a `send` event + self.h.frame() # the wait, arriving one line later + self.assertEqual(len(self.h.pasted), 1) + + def test_the_send_hotkey_during_the_wait_does_not_paste_twice(self): + # Somebody who does not yet trust the release and reaches for ctrl+alt+enter. + self.h.press().speak().release() + self.h.decode("said once") + self.h.pill._send() + self.h.frame() + self.assertEqual(self.h.pasted, [("said once", False)]) + + def test_whoever_gets_there_first_has_it_and_the_wait_stands_down(self): + self.h.press().speak().release() + self.assertIsNotNone(self.h.pill._ptt_wait) + self.h.decode("first past the post") + self.h.pill._send() + self.assertIsNone(self.h.pill._ptt_wait) + + def test_clearing_the_draft_cancels_a_paste_that_has_not_landed(self): + # The nastiest one to leave running: the draft is cleared, the decode lands a + # second later and refills it, and the wait pastes into the user's window the + # words they just pressed a key to stop. Clear means clear. + self.h.press().speak().release() + self.h.pill._clear() + self.h.decode("what they cancelled").frame() + self.assertEqual(self.h.pasted, []) + + def test_switching_mode_cancels_it_rather_than_translating_it(self): + # A wait armed in dictate pastes into a window; fired in converse it would ask a + # CLI. The switch is one keypress away at all times. Driven through the `mode` + # dispatch rather than by clearing the flag here, which would be a test asserting + # what it had just done itself. + self.h.press().speak().release() + self.assertIsNotNone(self.h.pill._ptt_wait) + self.h.key("mode") + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("asked, not pasted").frame() + self.assertEqual(self.h.pasted, []) + + def test_the_toggle_hotkey_mid_hold_keeps_the_words(self): + # `pause()` bumps the capture generation, which is how a deliberate stop refuses + # a decode from before it — and the utterance being spoken *right now* is exactly + # what that would refuse. The hold is ended first, so it is committed. + self.h.press().speak(2.0) + before = self.h.session._capture_generation + self.h.pill._toggle() + self.assertEqual(self.h.session._capture_generation, before) + self.assertTrue(self.h.session._sent) + + def test_and_does_not_paste_them_because_a_toggle_is_not_a_release(self): + # The user reached for the other control mid-sentence. Pasting on their behalf + # is not what either gesture asked for. + self.h.press().speak(2.0) + self.h.pill._toggle() + self.h.decode("mid sentence").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "mid sentence") + + def test_the_toggle_does_not_then_pause_the_session_it_just_stopped(self): + # The `return` in `_toggle`. Falling through would do the generation bump the + # branch above exists to avoid. + self.h.press().speak(2.0) + self.h.pill._toggle() + self.assertIsNone(self.h.pill._ptt_since) + self.assertFalse(self.h.mic.active) + + def test_the_mode_switch_really_does_clear_it_in_the_app(self): + # `Harness.dispatch` mirrors `Pill._frame`, so the test above could pass on the + # mirror alone. This is the tripwire on the original — the same shape + # `test_chord.py` uses for the four chord words, and for the same reason. + ui_src = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + branch = ui_src[ui_src.index('elif name == "mode":'):] + branch = branch[:branch.index('elif name == "quit"')] + self.assertIn("_ptt_wait = None", branch) + + def test_send_and_clear_clear_it_at_their_own_single_choke_point(self): + # Same tripwire, for the two methods every other send and stop path funnels + # through. One line in each is what makes four collisions impossible rather + # than four guards that would have to be kept in step. + ui_src = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + for name in ("_send", "_clear"): + with self.subTest(method=name): + body = ui_src[ui_src.index(f" def {name}(self"):] + body = body[:body.index("\n def ", 10)] + self.assertIn("self._ptt_wait = None", body) + + def test_a_hold_is_refused_while_the_hand_editor_is_open(self): + # `_pump_audio` throws away every block while `editing` is true, so a hold here + # would open the microphone, capture nothing, and end with no paste and no + # explanation — the silent deafness invariant 4 forbids. + self.h.session.editing = True + self.h.press() + self.assertIsNone(self.h.pill._ptt_since) + self.assertEqual(self.h.mic.starts, 0) + self.assertIn("editing", " ".join(self.h.notes)) + + +class TestHoldingThePillIsPushToTalkWithoutAHotkey(unittest.TestCase): + """The gesture for every platform that has no chord — which is every platform but one. + + `Chord` is a `WH_KEYBOARD_LL` hook and there is no such thing off Windows, so Flow + Lite has never had push-to-talk. It does not need a hotkey to: the button can be a + window Flow already draws, which costs no Accessibility permission, no Input + Monitoring, and no signed bundle to ask for them from. That is the one thing Lite + can do that a native app driving a system hotkey cannot. + + Three gestures now share the left button, and each test below is one of them, plus + the bug that fell out of fixing the arrangement. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + self.p = self.h.pill + self.timers = [] + self.p.after = lambda ms, fn: self.timers.append(fn) or len(self.timers) + self.p.after_cancel = lambda tid: self.timers.__setitem__(tid - 1, None) + self.p._toggle = mock.Mock(name="toggle") + + def at(self, x=0, y=0): + return mock.Mock(x_root=x, y_root=y) + + def hold(self): + """Let the pending hold timer fire, as Tk would after `PILL_HOLD_SEC`.""" + for fn in self.timers: + if fn is not None: + fn() + + def test_a_quick_click_still_toggles(self): + # The gesture that was there first, and the one muscle memory depends on. + self.p._on_press(self.at()) + self.p._on_release() + self.p._toggle.assert_called_once() + self.assertEqual(self.h.mic.starts, 0) + + def test_holding_it_starts_capturing_before_the_button_comes_up(self): + # The whole difference between this and a long-click: capture has to start while + # the user is still holding, because the hold *is* the utterance. Waiting for + # the release to notice would record nothing at all. + self.p._on_press(self.at()) + self.hold() + self.assertTrue(self.h.mic.active) + self.assertIsNotNone(self.p._ptt_since) + + def test_and_releasing_sends_what_was_said(self): + self.p._on_press(self.at()) + self.hold() + self.h.speak() + self.p._on_release() + self.h.decode("held the pill and spoke").frame() + self.assertEqual(self.h.pasted, [("held the pill and spoke", False)]) + self.p._toggle.assert_not_called() + + def test_a_hold_never_also_toggles(self): + # Both gestures on one button, so the release has to pick exactly one. + self.p._on_press(self.at()) + self.hold() + self.h.speak() + self.p._on_release() + self.p._toggle.assert_not_called() + + def test_dragging_the_pill_no_longer_toggles_listening(self): + # The bug this arrangement fixes, and it predates hold-to-talk: `_toggle` was + # bound to ``, which in Tk is the *press* — so every drag of the pill + # armed or disarmed capture on the way past. + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(60, 0)) + self.p._on_release() + self.p._toggle.assert_not_called() + + def test_a_drag_does_not_start_an_utterance_either(self): + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(60, 0)) + self.hold() + self.assertEqual(self.h.mic.starts, 0) + self.assertIsNone(self.p._ptt_since) + + def test_a_hand_that_is_merely_not_still_is_not_a_drag(self): + # `PILL_DRAG_SLOP` exists because a hand resting on a mouse trembles, and a hold + # that lost its nerve on one pixel would be a gesture nobody could rely on. + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(ui.PILL_DRAG_SLOP, ui.PILL_DRAG_SLOP)) + self.hold() + self.assertTrue(self.h.mic.active) + + def test_moving_while_talking_does_not_cancel_the_utterance(self): + # Once capture is open the pointer is irrelevant: somebody talking into a held + # pill may well move the mouse, and cancelling their sentence for it would be + # the gesture betraying them. + self.p._on_press(self.at(0, 0)) + self.hold() + self.p._on_motion(self.at(400, 400)) + self.assertTrue(self.h.mic.active) + self.h.speak() + self.p._on_release() + self.h.decode("still mine").frame() + self.assertEqual(self.h.pasted, [("still mine", False)]) + + def test_a_click_cancels_the_pending_hold_rather_than_leaving_it_armed(self): + # Otherwise the timer fires after the button is already up and opens a + # microphone nobody is holding — the exact failure `PTT_MAX_HOLD_SEC` exists to + # catch, arriving by a route it should never have to. + self.p._on_press(self.at()) + self.p._on_release() + self.hold() + self.assertEqual(self.h.mic.starts, 0) + + def test_it_works_in_lite_where_there_is_no_chord_at_all(self): + # The point of the whole gesture. Lite copies instead of pasting — `_send` + # already knows — so the hold ends on the clipboard with nothing granted but + # the microphone. + h = Harness(lite=True) + self.addCleanup(h.session.close) + timers = [] + h.pill.after = lambda ms, fn: timers.append(fn) or len(timers) + h.pill.after_cancel = lambda tid: timers.__setitem__(tid - 1, None) + h.pill._on_press(mock.Mock(x_root=0, y_root=0)) + for fn in timers: + if fn is not None: + fn() + self.assertTrue(h.mic.active) + h.speak() + h.pill._on_release() + h.decode("onto the clipboard").frame() + self.assertEqual(h.pasted, []) # copied, not pasted + + +class TestItFailsTheWayTheRestOfTheSurfaceDoes(unittest.TestCase): + def test_a_microphone_that_will_not_open_leaves_the_pill_disarmed(self): + # `_toggle` already refuses to show a green pill over a dead capture, and the + # chord must tell the same truth — with the added stake that under push-to-talk + # the user is about to speak into it. + h = Harness() + self.addCleanup(h.session.close) + with mock.patch.object(h.mic, "start", side_effect=OSError("device in use")): + h.press() + self.assertFalse(h.pill.armed) + self.assertIsNone(h.pill._ptt_since) + self.assertTrue(h.pill._flash) + + def test_and_the_release_afterwards_is_a_no_op(self): + # There is no hold to end. Without the `_ptt_since` guard this would commit an + # empty utterance and stop a microphone that never started. + h = Harness() + self.addCleanup(h.session.close) + with mock.patch.object(h.mic, "start", side_effect=OSError("device in use")): + h.press() + h.release() + self.assertEqual(h.mic.stops, 0) + + def test_lite_copies_instead_of_pasting(self): + # The gesture is the same; where the words go is `_send`'s business, and it + # already knows. Asserted so the chord does not grow its own idea of Lite. + h = Harness(lite=True) + self.addCleanup(h.session.close) + h.press().speak().release() + h.decode("into the clipboard").frame() + self.assertEqual(h.pasted, []) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_tray.py b/tests/test_tray.py new file mode 100644 index 0000000..3f2fb2b --- /dev/null +++ b/tests/test_tray.py @@ -0,0 +1,301 @@ +"""Hiding Flow without losing it. + +The need, in the owner's words: "there are times where i wanted to dictate but at the +same time i wanted to see but i don't want it to keep it on my screen". Parking the +window is the easy half and the dangerous one — a Flow with no window and no icon is a +process that cannot be reached, configured or quit except through Task Manager. + +So the property under test is not "it hides". It is **that it refuses to hide unless +there is a way back**, and that the way back works. + +No window is created and no icon is registered here. `flow.tray` is a thin ctypes wrapper +around `Shell_NotifyIconW`, verified against the real shell by hand; what these check is +the decision Flow makes around it. +""" + +import queue +import sys +import unittest +from unittest import mock + +import flow.tray as tray +import flow.ui as ui + + +def pill(**kw): + """A pill with just enough of one to hide and come back.""" + p = ui.Pill.__new__(ui.Pill) + # `front` is a read-only property choosing bubble-or-card by mode, so the stand-in + # goes on the thing it chooses rather than over the property itself. + p.session = mock.Mock(mode=ui.DICTATE) + p.bubble = mock.Mock() + p.card = mock.Mock() + # Real ints: hiding remembers where the window was, and `tk.Misc.__getattr__` turns + # a missing one into a Tcl lookup rather than an AttributeError. + p.x, p.y = 430, 608 + p._shell_h = ui.PILL_H + p._home = None + p._tray_events = queue.Queue() + p._tray = None + p._hidden = False + p._flash = 0 + p._ptt_since = None + p.park = mock.Mock() + p.deiconify = mock.Mock() + p.lift = mock.Mock() + p._sync_shell = mock.Mock() + p.quit_app = mock.Mock() + for name, value in kw.items(): + setattr(p, name, value) + return p + + +class TestHidingNeedsAWayBack(unittest.TestCase): + def test_it_hides_once_the_icon_is_actually_there(self): + p = pill() + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon), \ + mock.patch.object(ui, "park") as parked: + self.assertTrue(p.hide_to_tray()) + self.assertTrue(p._hidden) + parked.assert_called_once_with(p) + + def test_an_icon_that_would_not_register_leaves_the_window_alone(self): + """The one that matters. `Shell_NotifyIcon` can fail — a shell that is still + starting, a notification area that is full — and hiding anyway would strand the + user with no window and nothing to click.""" + p = pill() + icon = mock.Mock(**{"start.return_value": False}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon), \ + mock.patch.object(ui, "park") as parked: + self.assertFalse(p.hide_to_tray()) + self.assertFalse(p._hidden) + parked.assert_not_called() + self.assertIn("would not take", p.front.note.call_args.args[0]) + + def test_a_platform_with_no_notification_area_says_so(self): + # macOS has a menu bar item and Linux has whatever the desktop offers; neither + # is `Shell_NotifyIcon`, and pretending otherwise would hide a window for good. + p = pill() + with mock.patch.object(tray, "available", return_value=False), \ + mock.patch.object(ui, "park") as parked: + self.assertFalse(p.hide_to_tray()) + self.assertFalse(p._hidden) + parked.assert_not_called() + + def test_the_icon_is_built_once_and_reused(self): + # Somebody who hides Flow once will hide it again, and a second `Shell_NotifyIcon` + # for the same app is a second icon in the tray. + p = pill() + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon) as made, \ + mock.patch.object(ui, "park"): + p.hide_to_tray() + p.show_from_tray() + p.hide_to_tray() + made.assert_called_once() + + +class TestComingBack(unittest.TestCase): + def test_showing_puts_the_window_where_it_was(self): + p = pill(_hidden=True) + p.show_from_tray() + self.assertFalse(p._hidden) + p._sync_shell.assert_called_once() + p.deiconify.assert_called_once() + + def test_showing_a_window_that_is_already_up_does_nothing(self): + p = pill(_hidden=False) + p.show_from_tray() + p.deiconify.assert_not_called() + + def test_the_chord_brings_it_back_before_it_opens_the_microphone(self): + """A hold that showed nothing would be an open microphone with no way to tell + it was open — which is invariant 4 read from the other side.""" + p = pill(_hidden=True) + p.session = mock.Mock() + p.session.talk_start.side_effect = lambda *a, **k: None + with mock.patch.object(ui.Pill, "show_from_tray", + autospec=True) as shown, \ + mock.patch.object(ui.Pill, "_pump_talk", autospec=True): + try: + ui.Pill._talk_start(p) + except Exception: + pass + shown.assert_called_once() + + +class TestWhatTheIconSaysArrivesOnTheUIThread(unittest.TestCase): + """`tray.Tray` runs its window procedure on a thread of its own and puts *strings* + on a queue rather than calling back. That is the whole of the threading argument: + Tk is touched from one place, `_frame`, and nothing in `flow/tray.py` touches it.""" + + def test_a_show_click_shows(self): + p = pill(_hidden=True) + p._tray_events.put(tray.SHOW) + p._drain_tray() + self.assertFalse(p._hidden) + + def test_a_quit_click_quits(self): + p = pill() + p._tray_events.put(tray.QUIT) + p._drain_tray() + p.quit_app.assert_called_once() + + def test_an_empty_queue_is_the_ordinary_case_and_costs_nothing(self): + p = pill() + p._drain_tray() + p.quit_app.assert_not_called() + + def test_everything_waiting_is_taken_in_one_pass(self): + # The queue is drained, not sampled: a frame that took one event and left the + # rest would answer a click a frame late for every click before it. + p = pill(_hidden=True) + for _ in range(3): + p._tray_events.put(tray.SHOW) + p._drain_tray() + self.assertTrue(p._tray_events.empty()) + + +class TestTheModule(unittest.TestCase): + def test_it_is_windows_only_and_says_so(self): + for platform, expected in (("win32", True), ("darwin", False), ("linux", False)): + with self.subTest(platform=platform): + with mock.patch.object(sys, "platform", platform): + self.assertIs(tray.available(), expected) + + def test_stopping_an_icon_that_never_started_is_safe(self): + # `quit_app` calls this unconditionally, including after a `hide_to_tray` that + # was refused. + icon = tray.Tray("probe") + icon.stop() + self.assertEqual(icon.hwnd, 0) + + def test_starting_twice_returns_the_first_answer(self): + icon = tray.Tray("probe") + icon._thread = mock.Mock() + icon._ok = True + self.assertTrue(icon.start()) + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: Shell_NotifyIconW") + def test_the_struct_is_the_size_the_shell_expects(self): + # `cbSize` is how the shell knows which layout it has been handed, and it is + # `sizeof` rather than a number typed in — this asserts the field is wired to + # the struct at all. + icon = tray.Tray("probe") + import ctypes + + self.assertEqual(icon._icon_data().cbSize, + ctypes.sizeof(tray._NOTIFYICONDATAW)) + + def test_the_window_procedure_is_held_alive(self): + """A ctypes callback is garbage like anything else, and one collected while + Windows still holds its address is an access violation on a thread nobody is + watching.""" + icon = tray.Tray("probe") + self.assertIsNotNone(icon._proc) + + +if __name__ == "__main__": # pragma: no cover + unittest.main() + +class TestItComesBackWhereYouLeftIt(unittest.TestCase): + """Somebody who dragged Flow to the left of their screen did not ask for it to + reappear in the middle.""" + + def hidden(self): + p = pill(x=40, y=600) + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), mock.patch.object(tray, "Tray", return_value=icon), mock.patch.object(ui, "park"): + p.hide_to_tray() + return p + + def test_the_position_is_remembered_across_the_hide(self): + p = self.hidden() + self.assertEqual(p._home, (40, 600 + ui.PILL_H)) + + def test_and_restored_on_the_way_back(self): + p = self.hidden() + p.x, p.y = 9999, 9999 # where `park` left it + p.show_from_tray() + self.assertEqual((p.x, p.y), (40, 600)) + + def test_the_foot_is_what_is_remembered_not_the_top(self): + # The shell is anchored by its bottom edge, so a panel that was open when it was + # hidden — and closed by the time it comes back — must not move the controls. + p = pill(x=40, y=500, _shell_h=ui.PILL_H + 140) + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), mock.patch.object(tray, "Tray", return_value=icon), mock.patch.object(ui, "park"): + p.hide_to_tray() + foot = 500 + ui.PILL_H + 140 + p._shell_h = ui.PILL_H + p.show_from_tray() + self.assertEqual(p.y + p._shell_h, foot) + + +class TestTheFramePumpLeavesAHiddenWindowAlone(unittest.TestCase): + """The second half of why "Hide to tray" did nothing. + + `_sync_shell` re-asserts the window's geometry, and it runs thirty times a second. It + used to win every one of them, so a parked window was back on screen before the next + frame had drawn. + """ + + def test_sync_shell_returns_immediately_while_hidden(self): + p = pill(_hidden=True) + p.geometry = mock.Mock() + p.canvas = mock.Mock() + ui.Pill._sync_shell(p) + p.geometry.assert_not_called() + + def test_the_pill_has_a_width_under_the_name_park_uses(self): + # `park(self)` reads `win.width`. The panels had one and the pill did not, so the + # call went to `tk.Misc.__getattr__` and looked for a Tcl command. + p = pill() + self.assertEqual(p.width, ui.BUBBLE_W) + +class TestEveryWin32NameResolves(unittest.TestCase): + """The bug this class exists for: `user32.PostMessage` does not exist. + + There is no bare `PostMessage` export — the name is a C macro resolving to the A or + W variant — so `ctypes` raised `AttributeError: function 'PostMessage' not found`. + It raised *at click time*, inside the tray thread, after the menu had been chosen + from and before anything acted on the choice: right-clicking the icon showed a menu + that then did nothing. Nothing in the test suite could have caught it, because + nothing named the function until somebody clicked. + + So the suite names them all. This walks the module's own source for every `user32` + call it makes and asks Windows whether each one is really there — a spelling check + that costs nothing and would have failed the moment the typo was written. + """ + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: user32") + def test_every_user32_function_the_module_names_exists(self): + import ctypes + import pathlib + import re + + source = pathlib.Path(tray.__file__).read_text(encoding="utf-8") + named = {a or b for a, b in + re.findall(r"user32\.(\w+)|u\.(\w+)\.", source)} + self.assertIn("PostMessageW", named, "the source no longer calls what it did") + user32 = ctypes.windll.user32 + missing = sorted(n for n in named if not hasattr(user32, n)) + self.assertEqual(missing, [], f"user32 has no {missing}") + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: shell32") + def test_the_shell_call_exists_too(self): + self.assertTrue(hasattr(tray._shell(), "Shell_NotifyIconW")) + + def test_a_click_that_raises_does_not_escape_into_windows(self): + """Windows called us. An exception unwinding into its stack is undefined at + best, and a tray that silently stops answering is the failure this file exists + to prevent — so it is caught, and said out loud.""" + icon = tray.Tray("probe") + with mock.patch.object(icon, "_popup", side_effect=RuntimeError("boom")): + self.assertEqual( + icon._on_message(0, tray._WM_TRAY, 0, tray._WM_RBUTTONUP), 0) +