From 4a2ac66131871e1f6752605858139af91fd08ce8 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:22:04 -0500 Subject: [PATCH 01/41] Make the chord a hold, and give every platform the same gesture without a hotkey The chord was a toggle that fired one word on release. It is push-to-talk now: press warms the models and opens the microphone, the hold is the utterance, and the release stops capture and pastes what was said. Both gestures ship and Settings > Chord switches between them while Flow runs, because they are good at different things - a hold needs no decision about when you are finished, and a toggle is the only one of the two that survives a paragraph or hands that cannot hold two keys down. The same gesture is on the pill itself, which is the part that matters off Windows: `Chord` is a WH_KEYBOARD_LL hook and there is no such thing on a Mac, so Lite has never had push-to-talk. It does not need a hotkey to. Hold the pill, speak, let go - one button now carries three gestures, judged on what you did: a quick click toggles, a 300 ms hold talks, motion drags. That fixes something older on the way past: `_toggle` was bound to the button *press*, so every drag of the pill used to toggle listening. Five things can send one utterance - the chip, the hotkey, the spoken trigger, converse's countdown, and now a release waiting on a decode. A hold owns one send and whoever gets there first has it, enforced at the single point all five share rather than five guards that would drift. Clear cancels a pending paste, a mode switch drops it, the toggle hotkey mid-hold commits rather than letting `pause()` refuse the decode it just asked for, and a hold is refused while the hand editor is open instead of recording nothing. Two ceilings, neither hypothetical: a hold whose release never arrives stops itself after two minutes, and a paste that never lands gives up after fifteen seconds and says where the words are. Both keep what was said. Placement moved to bottom centre of the monitor under the pointer, taken from FluidVoice's positionWindow - centred on the physical display, stood on the work area. `"place": "corner"` restores the old bottom right. Doing it turned up a real bug: the work area was read once at launch from SystemParametersInfoW, which only ever answers for the primary monitor, so on a two-monitor desk every window Flow drew could land on the screen you were not looking at. Hidden panels park past every display instead of unmapping, so a hold shows the draft in the time a window takes to move. The meter blooms from its centre rather than scrolling a history past, which is FluidVoice's BottomWaveformView arithmetic - envelope, 0.55 response curve and rounded caps. Re-measured the body column at the shipped 420 px rather than the 380 it was taken at: 62 characters a line, not 56, and the tail is 1750 characters for the same 28 laid-out lines. Idle unload went 5 min to 30 - five minutes is inside the gaps of a working session, so the common case was a reload in the middle of somebody's first sentence back rather than memory reclaimed from somebody gone. The comment defending it said "the 141 MB model" while two tiers were resident; it is ~605 MB. Carries the per-app prompt sets already in the tree - refine.py, the apps table and tests/test_apps.py - because profile.py, README.md and guide.md hold both and splitting them needs hunk surgery. Co-Authored-By: Claude Opus 5 --- README.md | 16 +- docs/analysis.md | 10 +- docs/architecture.md | 4 +- docs/development.md | 4 +- docs/guide.md | 222 +++++++++- flow/__main__.py | 76 +++- flow/help.py | 14 + flow/hotkey.py | 380 +++++++++++++++++ flow/profile.py | 92 ++++ flow/refine.py | 45 ++ flow/session.py | 192 ++++++++- flow/ui.py | 912 ++++++++++++++++++++++++++++++++++++++-- tests/test_apps.py | 271 ++++++++++++ tests/test_chord.py | 680 ++++++++++++++++++++++++++++++ tests/test_help.py | 41 +- tests/test_hotkey.py | 11 + tests/test_indicator.py | 2 +- tests/test_lite.py | 45 ++ tests/test_longrun.py | 46 ++ tests/test_overlay.py | 205 +++++++++ tests/test_pill.py | 396 ++++++++++++++++- tests/test_talk.py | 737 ++++++++++++++++++++++++++++++++ 22 files changed, 4344 insertions(+), 57 deletions(-) create mode 100644 tests/test_apps.py create mode 100644 tests/test_chord.py create mode 100644 tests/test_overlay.py create mode 100644 tests/test_talk.py diff --git a/README.md b/README.md index f1c99da..5eff6c3 100644 --- a/README.md +++ b/README.md @@ -52,10 +52,16 @@ mode are all there, unchanged. What changes is the last inch: **Send copies the you press Ctrl+V yourself.** Four things it does not do — exclusions, not gaps: no injection into another -application's window, no global hotkeys (arming is a click on the pill), no auto-paste, -and no target-window awareness. What that buys is the property full Flow cannot have: -**nothing to grant but the microphone** — no accessibility permission, no input -monitoring, no trusted-application prompt. +application's window, no global hotkeys, no auto-paste, and no target-window awareness. +What that buys is the property full Flow cannot have: **nothing to grant but the +microphone** — no accessibility permission, no input monitoring, no trusted-application +prompt. + +**Push-to-talk works here anyway, and it does not need a hotkey.** Hold the pill, speak, +let go — the words land on your clipboard. A quick click still toggles listening, and +dragging still moves the pill. It is the same gesture Windows gets from `ctrl+win`, on a +button Flow already draws, which is why it costs no permission: a system hotkey is the +part that needs Accessibility and Input Monitoring, and this is not one. Two requirements Lite cannot meet, named rather than dropped: P7 (safe paste into a terminal) is a promise about a paste Flow performs, and Lite performs none; and P9's loop @@ -158,7 +164,7 @@ where Flow mishears them. That is the one thing I cannot measure alone. ```bash git clone https://github.com/samartomar/flow && cd flow uv sync && uv run flow # run it -uv run python -m unittest discover -s tests # 1,881 tests, ~40 s, no mic needed +uv run python -m unittest discover -s tests # 1,965 tests, ~42 s, no mic needed uv run python scripts/selfdrive.py # the end-to-end harness ``` diff --git a/docs/analysis.md b/docs/analysis.md index 73df480..5b94c59 100644 --- a/docs/analysis.md +++ b/docs/analysis.md @@ -173,9 +173,15 @@ Guards that enforce R11 ("no heavy lifting"), **as built**: - The **input device** is health-checked every 5 s and reopened if it dies. The decode worker instead swallows and reports per-decode exceptions, so it cannot die and needs no restart. -- Idle > 5 min → **unload the model only**. The mic stays open, a deliberate narrowing +- Idle > 30 min → **unload the models only**. The mic stays open, a deliberate narrowing of the "release the mic" idea above: releasing it would leave the app unable to hear - its own wake-up, and the mic is cheap while the model is 141 MB. + its own wake-up, and the mic is cheap while the models are ~605 MB (`base.en` 141 MB + for partials plus `small.en` 464 MB for finals — this line said "the model is 141 MB" + while there were two tiers resident, understating its own case fourfold). Was 5 min, + which sat *inside* the gaps of an ordinary session: the common case was not reclaiming + memory from somebody who had left, it was paying a reload in the middle of their first + sentence back. The chord now also warms on press-down, so the load happens during the + hold rather than inside the first utterance. - Undo history is bounded by **both** snapshot count and total characters, since 30 copies of a long draft is where undo quietly becomes megabytes. diff --git a/docs/architecture.md b/docs/architecture.md index 8418d2a..be465ca 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1500,14 +1500,14 @@ Only the ones with a measurement or a failure behind them. Everything else is in |---|---|---| | `MAX_UTTERANCE_SEC` | 24.0 s | Whisper pads to one 30 s mel window, so cost is flat below it and climbs past it. Cut before the boundary keeps latency constant in a long session | | `PARTIAL_MIN_GROWTH_SEC` | 0.7 s | Paired with the worker-idle check, this is what bounds partial latency | -| `IDLE_UNLOAD_SEC` | 300 s | Release the models, keep the mic. Releasing the mic would leave the app unable to hear its own wake-up | +| `IDLE_UNLOAD_SEC` / `WARM_GRACE_SEC` | 1800 s / 60 s | Release the models (~605 MB: `base.en` 141 + `small.en` 464), keep the mic. Releasing the mic would leave the app unable to hear its own wake-up. 1800 rather than the original 300 because five minutes is inside the gaps of a working session, so the unload's common case was a reload in the middle of somebody's first sentence back rather than memory reclaimed from somebody gone. The grace is what the chord's press-down buys: `Session.warm` starts the load during the hold, and holds off the idle unload for a minute so the health pump cannot drop the models between the press and the release that arms. A window rather than a touch of `_last_activity`, because `ctrl+win` is also Windows' `ctrl+win+arrow` prefix and letting a desktop switch reset the idle clock would retire the unload for anyone who uses virtual desktops | | `STALL_SEC` | 2.0 s | The liveness *backstop*: how long PortAudio may go without handing over a block before the stream counts as dead whatever it says about itself. Not a silence heuristic — what is timed is delivery, and a quiet room delivers blocks at exactly the rate a loud one does. Measured here: the gap between callbacks is a median **63.0 ms** (p99 79 ms, max 79 ms) idle, and a worst case of **453 ms** over ~109 s under four CPU-bound threads standing in for a decode. 2.0 s is 4.4x that worst case and 31 block periods; a false positive tears down a working stream mid-sentence, so the bias is toward late. The actual detector is `Pa_IsStreamActive`, polled every frame at **0.43 us** a call | | `MIC_RETRY_SEC` | 1.0 s | Spacing between reopen attempts. Measured: `sd._terminate()` + `sd._initialize()` costs **12.2 ms** (11.5-13.8 over six rounds), the open behind it **20.6 ms** (20.1-23.0), and the first block arrives **111-266 ms** after the open begins — a full `Mic.restart()` on real hardware measured **103 ms**. So a second is roughly 4x the worst end-to-end cost of an attempt: long enough that a retry is a fresh chance at a device still settling rather than the same failure re-timed, short enough that the whole ordeal fits inside `AUTO_ASK_SEC`. Spent as a deadline the pump checks, never as a sleep | | `MIC_RETRIES` | 3 | Attempts, counting the immediate one. The first covers the commonest failure, which is not a device dying but the *default moving* — a headset plugged in, the old stream orphaned, the replacement already there. The other two cover a device that is coming back but is not back yet (USB re-enumeration, a Bluetooth re-pair). Past ~2 s a device is not settling, and the honest end is the disarmed state a failed startup produces: pill off, reason on screen, clicking it opens fresh against whatever is plugged in by then. This replaced a 5 s heartbeat that reopened *forever*, one note and one error every five seconds, over a pill still claiming to be armed | | `FORCE_NEXT_TTL_SEC` | 30 s | A Refine/Continue chip means "the next thing I say"; after this long the next thing someone says is a different thought. The chips also toggle, because a one-way door that lasts 30 s reads as the app being stuck | | `AUTO_ASK_SEC` | 4 s | Converse mode only. Measured: the pauses a speaker leaves between separate spoken items run 1.4–3.3 s (median 2.5 s) on the one recording where every item was located, and each gap also contains a spoken item number, so real silence is shorter — under ~3.3 s fires mid-thought. R5 still holds where it matters: pasting into a window is irreversible and stays manual, asking is not | | `ui.SENT_LINGER_SEC` | 4 s | How long the bubble holds what a dictate-mode Send just handed over, with the chip that puts it back. Deliberately **not** `AUTO_ASK_SEC`, which is also 4 s and is a different four seconds: that one is how long a settled draft waits before asking itself, this is how long words stay recoverable after they have gone, and either could move without the other. The number it replaces was zero — the bubble was withdrawn on Send, so a Send that went nowhere and a Send that worked left the same empty screen | -| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1600 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1600 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 56 (measured: 3 160 characters wrapped to 56 lines at 352 px) — an average rather than a layout, because the layout is the cost | +| `ui.BODY_MAX_H` / `BODY_TAIL_CHARS` | 340 px / 1750 chars | How tall the draft may draw, and how much of it is laid out per event. Measured on the real canvas before the fix: **2.4 ms at 1 000 characters, 32.7 ms at 10 000, 476.7 ms at 50 000** — per partial, on the UI thread — and a 50 000-character draft sized the bubble **15 153 px tall inside a 672 px work area**, which is where the Send chip was at the one moment the spoken exits had already died with the microphone. Only the tail is laid out now, with `… N earlier lines` above it; the cap is 20 lines at the 17 px the body font measures, and 1750 characters is about 28 of them, so the visible window is always full. After: **2.5 / 4.2 / 4.3 ms**, and 414 px at 50 000. The line count is wraps plus explicit breaks from `BODY_CHARS_PER_LINE` = 62 (re-measured at the shipped 392 px column: 3 160 characters wrapped to 51 lines; the earlier 56 was the same prose at the pre-Phase-6 352 px column) — an average rather than a layout, because the layout is the cost | | `ui.EDGE_AIR` | 8 px | Air between the bubble and every edge of the work area — one number, because the window is *fitted* to `work − 2 × air` and *clamped* by `air`, and those two have to agree. They did not: item 37's cap bounded the draft body and the reply path kept its full-text probe, so a 4 000-character answer sized the window **1 459 px** and a 12 000-character artifact **4 179 px** on a 672 px desktop, and `reposition` pinned both at `top + 8` and let the rest run off the bottom — **12 of 36 corner placements outside the work area**, chip row at screen y 1 427 and 4 147. Fitting the height in `_render` is what makes the clamp a guarantee rather than a best effort: **0 of 36** after, chip row at 624. The top edge was never the breach, at any corner or in any state | | `ui.DOT_SEC` | 0.4 s | One dot of the indeterminate-wait animation. The bubble renders on events and a wait has no events, so the frame is computed and compared before anything is drawn — at this cadence that is ~2.5 repaints a second instead of the 33 that redrawing every pump would cost. Same discipline as the auto-ask countdown | | `DEAF_DB` | −120.0 | What `level_db` reports while the microphone is not evidence. Below any real room — a quiet room with a good USB mic measures −96.7 dB — so every meter maps it to silence without having to know why | diff --git a/docs/development.md b/docs/development.md index 5ad34f3..c99b904 100644 --- a/docs/development.md +++ b/docs/development.md @@ -26,7 +26,7 @@ flow/ hotkey.py RegisterHotKey on its own message-loop thread (ctypes) diag.py the wordless trace, and the identity block every benchmark records scripts/ benchmarks, probes, the soak test and the self-drive harness -tests/ 1,881 tests: routing, state machine, filters, phonetics, resilience +tests/ 1,965 tests: routing, state machine, filters, phonetics, resilience docs/ what Flow is for, the roadmap, the analysis, the recording kit ``` @@ -42,7 +42,7 @@ the event stream and the tuning constants with the measurements behind them. uv run python -m unittest discover -s tests ``` -1,881 tests, ~40 s, no microphone or model required — the fakes are injectable precisely so +1,965 tests, ~42 s, no microphone or model required — the fakes are injectable precisely so the routing logic, where the subtle bugs live, can be tested without either. **The interpreter is pinned.** `.python-version` holds `3.12`, which is what CI installs and diff --git a/docs/guide.md b/docs/guide.md index 689d8b2..88ce824 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -228,6 +228,7 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--arm` | start listening immediately, no click needed | | `--no-paste` | print the draft to stdout instead of pasting it | | `--no-hotkeys` | skip global hotkey registration | +| `--no-chord` | skip the modifier-only chord, so no low-level keyboard hook is installed ([The chord](#the-chord-ctrlwin)) | | `--calibrate` | measure this room and this voice, store the profile, and exit ([P8](#calibration-p8)) | | `--no-profile` | ignore the stored profile and learn nothing this session | | `--converse` | start in converse mode: Send asks the agent CLI instead of pasting ([P9](#converse-mode-p9)) | @@ -406,6 +407,220 @@ for you to fix. `--no-hotkeys` and [Lite](#install) register nothing with the OS, so a `hotkeys` block is not read on those launches and nothing is said about it. +### Hold the pill (every platform) + +**Hold the pill, speak, let go.** Same gesture as the chord below, on a button Flow +already draws rather than on a system hotkey — so it works in Lite, on macOS and Linux, +where there is no chord at all and nothing to grant but the microphone. + +One button, three gestures, each judged on what you actually did: + +| | | +|---|---| +| **Quick click** | toggles listening, as it always has | +| **Hold ~300 ms, then speak** | push-to-talk — release sends | +| **Press and move** | drags the pill somewhere else | + +Moving the mouse *while already talking* does not cancel anything: once capture is open +the pointer is irrelevant, and a sentence lost to a twitch would be the gesture betraying +you. The 300 ms and the 4 px of slop are `PILL_HOLD_SEC` and `PILL_DRAG_SLOP`. + +This also fixed something older: `_toggle` was bound to the button *press*, so **every +drag of the pill used to toggle listening on the way past**. A click is judged on +release now, like a button anywhere else. + +### The chord (ctrl+win) + +**Settings ▸ Chord (ctrl+win)** picks which of two gestures it is. Both ship, because +neither replaces the other, and you can switch between them while Flow is running — the +change takes effect on the next press, with no restart. + +| | | +|---|---| +| **Hold to talk, release to send** | The shipped default. Press both keys, speak while they are down, let go. What you said is pasted into whatever window has focus — no third key, no clicking anything, no second shortcut to send. | +| **Press to start, press again to stop** | The original. A clean press-and-release starts listening and the next one stops it, exactly like the `toggle` hotkey. | + +**Pick the hold for a sentence and the toggle for a paragraph.** The hold needs no +decision about when you are finished and cannot leave a microphone running by accident. +The toggle is the only one of the two that survives a long thought with pauses in it, a +phone call you are transcribing, or hands that would rather not hold two keys down for a +minute. It is remembered in `profile.json` as `"gesture": "toggle"`. + +The rest of this section describes the hold, which is the default. + +The press-down does two things at once: it opens the microphone, and it starts loading +the models if they had been idle long enough to be released. That second half is why the +gesture is a *hold* rather than a tap — the second or so you spend pressing the keys and +drawing breath is the second the models need to come back, so it costs nothing instead of +landing in the middle of your first sentence. + +It exists because every combo in the table above ends in a key. `RegisterHotKey`, the +Windows call behind them, takes a virtual key and has no way to say "these modifiers and +nothing else" — so `ctrl+alt+space` is a shape your left hand makes *plus* a reach. Ctrl +and Win are neighbours, and holding two neighbours is one movement. + +**The `toggle` hotkey still works and still toggles.** The two are for different things +now rather than being two doors to the same room: hold the chord for a sentence, press +`ctrl+alt+space` for a paragraph you want to say hands-free. + +#### What happens when Windows wants ctrl+win too + +Windows uses ctrl+win as a prefix — `ctrl+win+d` makes a virtual desktop, `ctrl+win+←/→` +switch between them. Those still work, and Flow gets out of the way as soon as it sees +the third key: the capture stops on that keystroke, and **nothing is ever pasted**. + +Under the old toggle gesture Flow could refuse these outright, because nothing had +started until you let go. Push-to-talk opens the microphone on the press-down, so a +desktop switch does open it for a moment. Two things bound what that costs. The +microphone closes on the third key rather than at the release, so holding the keys +through several desktop switches does not record them. And if you *had* already started +speaking when the third key landed, your words are kept — they go to the draft, where the +Send chip picks them up — rather than being thrown away or pasted somewhere you did not +mean. + +Adding a modifier is a different chord and starts nothing at all: ctrl+shift+win does +nothing here. + +#### The two things that stop on their own + +Both are ceilings on a failure, not limits on you: + +- **A hold of two minutes ends itself.** A key release can genuinely go missing — a lock + screen or a remote-desktop session takes the keyboard mid-hold, or Windows drops the + keyboard hook for being slow. Without a ceiling that leaves a microphone open. What you + said is kept and put on screen; it is not pasted, because two minutes later you are + somewhere else. +- **A paste waits up to fifteen seconds for the decode.** Transcribing the final takes + from under a second to several, so the release cannot paste immediately — it waits. + Past fifteen seconds Flow stops waiting and tells you the words are in the draft. It + never pastes late: text arriving a minute after the gesture would land in whatever + window you had moved to. + +Change it or turn it off with `chord` in `~/.flow/profile.json`, beside the `hotkeys` +table: + +```json +{ + "schema": 1, + "chord": "ctrl+shift" +} +``` + +Two or three of `ctrl`, `alt`, `shift` and `win`. **One is refused** — a chord of one +modifier fires every time you tap that key and let go, which is a thing hands do all day +without meaning anything by it. Four is refused as well, because it is not a shape you +can hold. `"chord": ""` turns it off entirely, and is deliberately different from +deleting the line: Flow writes every field back when it saves, so a deleted `chord` +comes back as the default. + +**What this costs, stated plainly.** A modifier-only chord cannot be done with +`RegisterHotKey`, so Flow installs a low-level keyboard hook (`WH_KEYBOARD_LL`) — which +means Windows calls into Flow for every keystroke on the machine, not just Flow's own. +Three things are true about what that code does, and they are the reason it is +considered acceptable here: + +- **It never learns which key you pressed.** The key code is compared against eleven + modifier constants and against nothing else. Anything not in that set flips a single + true/false — not stored, not logged, not compared to anything, and it never leaves the + function. It is about sixty lines in `flow/hotkey.py` (`Chord`), written to be read. +- **It never swallows a keystroke.** Every event is passed straight on, including the + release that ends it. Ctrl and Win keep their normal jobs. +- **Nothing is sent anywhere.** Same as the rest of Flow: no network, no file. + +If you would rather not have that hook at all, `--no-chord` skips it for one launch and +`"chord": ""` skips it for good. The `toggle` hotkey is unaffected either way. + +If the OS refuses the hook — some policies and some elevated desktops do — the startup +block says so on one line and Flow carries on with the registered combos: + +``` +chord unavailable (keyboard hook refused); the toggle hotkey still works +``` + +### Where the panels open + +**Bottom centre of whichever monitor your mouse is on.** That is the shipped placement, +and it changed: Flow used to sit in the bottom-right corner of the primary display. + +Two things were wrong with the corner. The bottom right is the busiest part of a Windows +desktop — the tray lives there, every toast notification opens there, and plenty of apps +park their own status chrome there — so the one place Flow had reserved for itself was +the place most likely to be covered. And the primary display is not necessarily the one +you are working on: the work area was read once at launch from `SystemParametersInfoW`, +which only ever answers for the primary monitor, so on a two-monitor desk everything +Flow drew could land on the screen you were not looking at. + +The panel now follows the pointer's monitor and re-places itself when you move between +displays. Set `"place": "corner"` in `profile.json` to put it back in the bottom right; +it is kept for anybody who has spent months with it there, and only the position +changes. + +`"place"` takes `"bottom"` or `"corner"`. Anything else falls back to `"bottom"` rather +than refusing to launch — this is a file people edit by hand. + +**A hidden panel is moved off-screen, not closed.** It is parked past the far corner of +every monitor you have and pulled back when it is needed, so a push-to-talk hold shows +the draft in the time a window takes to move rather than the time one takes to open. +Nothing about that is visible except the speed. + +### Panel size + +**Settings ▸ Panel size** draws the draft bubble and the conversation card wider: +**Regular** (420 px, the shipped width), **Large** (520) or **Larger** (640). It applies +straight away — no restart — and is remembered in `profile.json` as `"panel": "large"`. + +**There is no "small", and the reason is the chip row rather than restraint.** The +bubble's five-chip row — Refine, Continue, Edit, Was a command, Send — measures 345 px, +and the card's runs to 377. Below 420 the row loses its gaps and then loses a label, and +the first label to go is Send. A draft panel must never put its own exit off the edge, so +420 is a floor: a hand-edited width below it is clamped back up rather than honoured. + +Wider panels lay out proportionally more text per line, so the tail of a long draft stays +as full at 640 px as it is at 420 — the number of lines handed to the canvas is what is +held constant, which is what keeps render cost flat on a two-hour dictation. + +### Per-app notes + +**A standing instruction that depends on which app you are dictating into.** Add an +`apps` table to `~/.flow/profile.json`, keyed by the executable name: + +```json +{ + "schema": 1, + "apps": { + "slack.exe": "Keep it conversational. No headings, no bullet lists.", + "code.exe": "Be terse. Prefer imperative mood.", + "outlook.exe": "Use British spelling and a full greeting." + } +} +``` + +The name is the executable of the window in front — the same one Flow already looks at +to tell a terminal from an editor. Case does not matter. An entry for an app you have not +installed is not an error and is not reported: there is no list of every program in the +world to check a name against, so a key that never matches simply never fires. `""` as +the instruction switches one app off without deleting the line you wrote. + +**It applies to rewrites, not to plain dictation.** Both the semantic rewrite and the +prompt-shaping polish ("make it a proper prompt") pick it up. Ordinary dictation never calls a +CLI at all, so there is nothing there for a note to change. + +**It cannot out-shout what you just said.** The note is phrased as a destination — *"this +text is going into slack.exe, bear this in mind, without letting it override anything +asked for below"* — and it is placed *before* your instruction rather than after it. That +ordering is the point: a per-app note is a standing preference, and speaking an +instruction is how you override a standing preference on this one occasion. Asking for +something formal in Slack gets you something formal. + +When a note applies, the pill says so, on the same line that names the CLI: + +``` +using your slack.exe note +``` + +Nothing is said when none applies, which is almost every rewrite until you write a table. +[Lite](#install) has no target-window awareness at all, so no note ever fires there. + ### The pill and the bubble Right-click the pill for **Listen / Stop listening**, **Send**, **Converse/Dictate @@ -413,8 +628,9 @@ mode**, **Clear draft** and **Quit**, plus any corrections Flow is offering. Lis the same toggle as clicking the pill, given a label — and it is the way in when a Hyper-V console or an RDP session keeps every hotkey for its guest and the mouse is what still reaches Flow. Everything you set once — **Trigger -word**, **Agent CLI**, **Voice**, **Mute/Speak replies** (only when a speech engine was -found), the auto-ask toggle and **Open settings folder** — lives under **Settings ▸**, and +word**, **Panel size**, **Agent CLI**, **Voice**, **Mute/Speak replies** (only when a +speech engine was found), the auto-ask toggle and **Open settings folder** — lives under +**Settings ▸**, and **Help ▸** has the command sheet and this guide. Drag the pill anywhere — it stays inside the desktop work area. @@ -1296,7 +1512,7 @@ Verified on this machine while writing this document: | | | |---|---| -| Test suite | **1,881 tests, 39.6 s**, no mic or model needed | +| Test suite | **1,965 tests, 42.1 s**, no mic or model needed | | End-to-end | `scripts/selfdrive.py`, **64/64 checks**, live CLI round trip | | Build | `uv build` → wheel + sdist; wheel installs into a clean venv and its `flow` command runs | | Dependencies | 3 declared, **28 installed**, 243.9 MB venv | diff --git a/flow/__main__.py b/flow/__main__.py index 08d3ae0..e268ccc 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -46,7 +46,8 @@ #: who did not type `--lite` deserves to be told which one they got. LITE_LINE = ( "Flow Lite on {platform}: Send copies the draft and you paste it - no injection, " - "no global hotkeys, nothing to grant but the microphone." + "no global hotkeys, nothing to grant but the microphone. " + "Hold the pill to talk, let go to send; a quick click toggles listening." ) @@ -86,6 +87,56 @@ def _timeout_arg(text: str) -> float: return value + +def _chord(profile, hotkeys, Chord, parse_chord, echo, ignored_line, + unavailable, default): + """Install the modifier-only chord, and say what happened either way. + + Separated from `main()` for the reason the rest of that function is not: this is the + one startup step that can fail *silently and invisibly*. A hotkey that could not + register has `Hotkeys.failed` to name it; a chord whose hook was refused looks + exactly like a chord nobody pressed. So every path out of here prints a line, and the + lines sit in the `hotkey` block where somebody already looks when a shortcut is dead. + + Returns the `Chord` or None. The caller does not need the value — `hotkeys.chord` + holds it, which is what keeps the callback alive and gets it torn down — but a + returned object is what a test can look at. + """ + wanted = profile.chord if profile is not None else default + # The empty string is how somebody turns it off in the file, and it is deliberately + # not the same as deleting the key: the next save writes every field back, so a + # deleted `chord` would reappear as the default and the setting would look ignored. + if not (wanted or "").strip(): + return None + mods, reason = parse_chord(wanted) + if mods is None: + say(ignored_line.format(combo=echo(wanted), reason=reason)) + return None + chord = Chord(hotkeys.presses, mods, + gesture=getattr(profile, "gesture", None) if profile else None) + if not chord.start(): + # The hook is refusable — by policy, by another process, by an elevated window + # this process cannot see into. Not fatal and not silent: the registered toggle + # is still there, and the line says so rather than leaving a shape that does + # nothing with no explanation. + say(unavailable) + return None + hotkeys.chord = chord + # "hold" and not "toggle": the word in this column is what the shortcut *does*, and + # the chord stopped doing the same thing as the toggle hotkey when it became + # push-to-talk. A startup block that still said toggle would be the only place a + # user could check, quietly describing the gesture they no longer have. + # The word in this column is what the shortcut *does*, and the chord does two + # different things now depending on a setting. A startup block that named only one + # of them would be the one place a user could check, describing a gesture half of + # them do not have. + if chord.gesture == "hold": + say(f"chord {'hold':8s} {chord.describe()} (hold to talk, release to send)") + else: + say(f"chord {'toggle':8s} {chord.describe()} (press to start, again to stop)") + return chord + + def main(argv: list[str] | None = None) -> int: # First, before anything resolves anything. Windows searches the current directory # ahead of PATH for a bare executable name, and Flow is launched *inside* project @@ -142,6 +193,10 @@ def main(argv: list[str] | None = None) -> int: ap.add_argument( "--no-hotkeys", action="store_true", help="skip global hotkey registration" ) + ap.add_argument( + "--no-chord", action="store_true", + help="skip the modifier-only chord (no low-level keyboard hook)", + ) ap.add_argument( "--no-paste", action="store_true", help="print the draft instead of pasting" ) @@ -261,9 +316,13 @@ def main(argv: list[str] | None = None) -> int: say(f"version: {version()} (nothing checks for updates on its own; " "--check-update asks GitHub)") if not lite: - from .hotkey import BAD_BLOCK_LINE, DEFAULT_BINDINGS, Hotkeys + from .hotkey import ( + BAD_BLOCK_LINE, CHORD_IGNORED_LINE, CHORD_UNAVAILABLE, + DEFAULT_BINDINGS, Chord, Hotkeys, _echo, parse_chord, + ) + from .profile import CHORD_DEFAULT from .inject import paste, take_warnings - from .ui import Pill + from .ui import Pill, apply_panel_width, apply_place, panel_width from .asr import WhisperTranscriber @@ -475,6 +534,13 @@ def main(argv: list[str] | None = None) -> int: say("mode: DICTATE - Send pastes into the focused window " "(--converse, or ctrl+alt+M, to ask instead)") + # Before anything is drawn, and that is the whole contract: `apply_panel_width` + # rebinds module globals that two window classes and the chrome functions read, so a + # call after the first frame would leave a window whose parts disagree about how wide + # it is. Here is the last moment that is safely true. + apply_panel_width(panel_width(profile.panel if profile is not None else None)) + apply_place(profile.place if profile is not None else "bottom") + hotkeys = None if not args.no_hotkeys and not lite: # Read only where registration happens, which is why `--no-hotkeys` and Lite are @@ -499,6 +565,10 @@ def main(argv: list[str] | None = None) -> int: else: say("hotkey thread did not start; continuing without hotkeys") hotkeys = None + if hotkeys is not None: + if not args.no_chord: + _chord(profile, hotkeys, Chord, parse_chord, _echo, + CHORD_IGNORED_LINE, CHORD_UNAVAILABLE, CHORD_DEFAULT) # Assigned rather than passed: the session is built before `RegisterHotKey` has been # asked for anything, and what the session needs is the answer, not the request. It # reads this only to say what still works when voice stops working. diff --git a/flow/help.py b/flow/help.py index 08d4502..6990970 100644 --- a/flow/help.py +++ b/flow/help.py @@ -138,6 +138,10 @@ def auto_ask_notice(seconds: float) -> str: #: cannot find here is worse than one described badly. _ACTIONS = { "toggle": "start and stop listening", + # The chord's word, and the one row here that describes a *hold* rather than a + # press. Spelled out to the end — release and all — because the half people get + # wrong is that there is no second shortcut to send. + "talk": "hold to talk, release to send", "send": "hand the draft over (Send, or Ask in converse mode)", "cancel": "clear the draft, and cut a spoken reply short", "mode": "switch between dictate and converse", @@ -218,6 +222,16 @@ def _hotkey_rows(hotkeys) -> list[tuple[str, str, str]]: failed = list(getattr(hotkeys, "failed", []) or []) rows = [("pair", combo, _ACTIONS.get(action, action)) for action, combo in chosen.items()] + # The chord goes first among the rows it duplicates, because it is the one somebody + # is looking for: it does not appear in `chosen` — nothing registered it — so a sheet + # built only from what `RegisterHotKey` accepted would describe a machine where the + # shape they have been using does not exist. Absent when the hook was refused or + # `--no-chord` was passed, which is the same rule the rest of this function follows: + # the sheet says what works on this machine this launch. + chord = getattr(hotkeys, "chord", None) + if chord is not None: + rows.insert(0, ("pair", f"{chord.describe()} (held)", + _ACTIONS.get(chord.action, chord.action))) # Named as unavailable rather than left out. A shortcut that silently does nothing is # the defect `Hotkeys.failed` was built to report, and a sheet that omitted it would # send somebody looking for a key that cannot exist on this machine. diff --git a/flow/hotkey.py b/flow/hotkey.py index df6d23d..09aa553 100644 --- a/flow/hotkey.py +++ b/flow/hotkey.py @@ -95,6 +95,13 @@ def __init__(self, bindings: dict[str, list[tuple[int, int]]], self.failed: list[str] = [] #: action name -> the combo that actually registered, e.g. "ctrl+alt+space" self.chosen: dict[str, str] = {} + #: The modifier-only `Chord`, once one has been installed, or None. + #: + #: It hangs here rather than being tracked separately because the pill already + #: calls `hotkeys.stop()` on the way out and there is exactly one thing that + #: should own the teardown of "global key input". A chord left installed after + #: the window is gone is a hook the OS still calls into a dead interpreter. + self.chord = None self._ids: dict[int, str] = {} self._tid: int | None = None self._ready = threading.Event() @@ -105,6 +112,8 @@ def start(self, timeout: float = 2.0) -> bool: return self._ready.wait(timeout) def stop(self) -> None: + if self.chord is not None: + self.chord.stop() if self._tid is not None: user32.PostThreadMessageW(self._tid, WM_QUIT, 0, 0) @@ -347,3 +356,374 @@ def overridden( continue out[action].insert(0, binding) return out, ignored + + +# -- the modifier-only chord ------------------------------------------------ +# +# `RegisterHotKey` cannot express "ctrl+win, and no third key". It takes a virtual key +# and there is no VK for "nothing" — so every combo above the fold has to end in a key +# somebody presses with their other hand. That third key is the whole complaint: ctrl, +# shift and win sit together at the bottom-left and are one *shape*, while ctrl+shift+z +# is a shape plus a reach. +# +# Which leaves the low-level keyboard hook, and R16 said no to exactly that — no +# `keyboard`, no `pynput`, nothing sitting on the global input path. This narrows R16 +# rather than reversing it, and the narrowing is the thing to check when reading: +# +# **It never learns which key you pressed.** `vkCode` is compared against `_CHORD_VKS` +# — eleven modifier constants — and against nothing else. A key that is not in that set +# sets a *boolean*. It is not stored, not logged, not compared to anything, and does +# not leave the callback. That is a materially smaller claim than "Flow can see your +# keystrokes", and it is small enough to audit in one sitting, which is the point. +# +# **It is allocation-free on the hot path.** This callback runs on the input path of +# every keystroke on the machine, and a slow one makes *all* typing feel late — the +# worst possible failure for a dictation tool, because it would look like the OS. So +# the steady state is integer comparisons and attribute writes. The one allocation is +# `presses.put`, which happens when the chord actually fires. +# +# **It never swallows anything.** `CallNextHookEx` is called on every event, including +# the ones that fire the chord. Ctrl and Win are real keys with real jobs and Flow does +# not get to keep them. +# +# **Why the "some other key" flag is correctness and not only privacy.** Windows already +# uses ctrl+win as a *prefix*: ctrl+win+d makes a virtual desktop, ctrl+win+left and +# +right switch between them. Every one of those presses a third key, so requiring a +# clean release — both modifiers held, nothing else touched, then let go — is what keeps +# switching desktops from also starting dictation. It is the same flag serving both ends. +# +# Start never opens on the release either, and that is Windows' own rule rather than +# something arranged here: the Start menu comes up on a Win keyup only when Win was +# pressed alone, and holding Ctrl is already enough to suppress it. + +WH_KEYBOARD_LL = 13 +WM_KEYDOWN, WM_KEYUP, WM_SYSKEYDOWN, WM_SYSKEYUP = 0x0100, 0x0101, 0x0104, 0x0105 + +VK_SHIFT, VK_CONTROL, VK_MENU = 0x10, 0x11, 0x12 +VK_LWIN, VK_RWIN = 0x5B, 0x5C +VK_LSHIFT, VK_RSHIFT = 0xA0, 0xA1 +VK_LCONTROL, VK_RCONTROL = 0xA2, 0xA3 +VK_LMENU, VK_RMENU = 0xA4, 0xA5 + +#: vk -> which modifier name it is. The hook reads `vkCode` against this and nothing +#: else; see the block above for why that sentence is the design and not a detail. +#: +#: Both the generic constants (`VK_CONTROL`) and the sided ones (`VK_LCONTROL`) are in +#: here because the hook is fed the *sided* code for a physical press, while an injected +#: keystroke — `SendInput`, which `flow/inject.py` itself uses — may carry the generic +#: one. Listening for only one of the two would make the chord's behaviour depend on +#: whether a human or a program pressed it. +_CHORD_VKS: dict[int, str] = { + VK_CONTROL: "ctrl", VK_LCONTROL: "ctrl", VK_RCONTROL: "ctrl", + VK_LWIN: "win", VK_RWIN: "win", + VK_SHIFT: "shift", VK_LSHIFT: "shift", VK_RSHIFT: "shift", + VK_MENU: "alt", VK_LMENU: "alt", VK_RMENU: "alt", +} + +#: The shipped chord — two modifiers, one hand, no reach, which is the entire reason +#: this code path exists rather than a sixth entry in `DEFAULT_BINDINGS` — lives in +#: `flow/profile.py` as `CHORD_DEFAULT`, because that module can be imported on a Mac +#: and this one cannot. What a chord *means* is judged here; what it defaults to is +#: written there, the same split `hotkeys` already uses. + +#: Modifier names a chord may be written from, in the order `describe_chord` prints them +#: so that "win+ctrl" and "ctrl+win" report as the same thing. +CHORD_NAMES = ("ctrl", "alt", "shift", "win") + +#: The two gestures a chord can be. `"hold"` is push-to-talk — press to start capturing, +#: speak while it is down, release to send what was said. `"toggle"` is the original: +#: a clean press-and-release starts hands-free listening, and the next one stops it. +#: +#: Both ship because they are good at different things and neither replaces the other. +#: A hold is the better gesture for a sentence — it needs no decision about when you are +#: finished, and it cannot leave a microphone running. A toggle is the only one of the +#: two that survives a paragraph, a long thought with pauses in it, or a pair of hands +#: that cannot hold two keys down for a minute. Shipping only the hold, which is what +#: this did first, took the second case away from everybody who had it. +GESTURES = ("hold", "toggle") +GESTURE_DEFAULT = "hold" + +#: Said when the hook cannot be installed. Shaped like the `hotkey` lines beside it, +#: because that block is where somebody looks when a shortcut is dead — and unlike a +#: taken combo, this one has a working answer to point at. +CHORD_UNAVAILABLE = ("chord unavailable (keyboard hook refused); " + "the toggle hotkey still works") + +#: Said when the `chord` value in profile.json could not be read. Same shape as +#: `IGNORED_LINE` and for the same reason: a setting that silently reverts is +#: indistinguishable from one that never saved (P2). +CHORD_IGNORED_LINE = "chord in profile.json ignored: {combo} - {reason}" + + +class _KBDLLHOOKSTRUCT(ctypes.Structure): + _fields_ = [ + ("vkCode", wintypes.DWORD), + ("scanCode", wintypes.DWORD), + ("flags", wintypes.DWORD), + ("time", wintypes.DWORD), + ("dwExtraInfo", ctypes.POINTER(wintypes.ULONG)), + ] + + +_HOOKPROC = ctypes.WINFUNCTYPE( + wintypes.LPARAM, ctypes.c_int, wintypes.WPARAM, wintypes.LPARAM +) + +user32.SetWindowsHookExW.argtypes = [ + ctypes.c_int, _HOOKPROC, wintypes.HINSTANCE, wintypes.DWORD +] +user32.SetWindowsHookExW.restype = wintypes.HHOOK +user32.UnhookWindowsHookEx.argtypes = [wintypes.HHOOK] +user32.UnhookWindowsHookEx.restype = wintypes.BOOL +user32.CallNextHookEx.argtypes = [ + wintypes.HHOOK, ctypes.c_int, wintypes.WPARAM, wintypes.LPARAM +] +user32.CallNextHookEx.restype = wintypes.LPARAM + + +def parse_chord(text) -> tuple[frozenset, str]: + """`{"ctrl", "win"}` for "ctrl+win", or `None` and why not. + + Two returns for the reason `parse` has two: the caller has to print what was wrong, + and a reason assembled at the call site is one that will eventually disagree with the + check that produced it. + + **One modifier is refused**, and it is the refusal worth explaining. A single-modifier + "chord" fires every time that key is tapped and released cleanly — and a bare Ctrl tap + is a thing hands do constantly while thinking. The result would be a dictation app + that starts recording at random, which is a defect nobody would attribute to a + setting they typed. + + **Four is refused for the opposite reason**: ctrl+alt+shift+win is not a shape, and a + chord that cannot be held is a chord that never fires. + """ + if not isinstance(text, str): + return None, "not a string" + parts = [p.strip().lower() for p in text.split("+") if p.strip()] + if not parts: + return None, "empty" + seen = set() + for part in parts: + if part not in CHORD_NAMES: + return None, f"{_echo(part)} is not a modifier" + seen.add(part) + if len(seen) < 2: + return None, "a chord needs two modifiers" + if len(seen) > 3: + return None, "a chord of four modifiers cannot be held" + return frozenset(seen), "" + + +def describe_chord(mods) -> str: + return "+".join(name for name in CHORD_NAMES if name in mods) + + +class Chord: + """Fires `action` when `mods` are held together and released with nothing else hit. + + Writes into a queue the caller owns — `Hotkeys.presses` in practice — so the session + drains one stream and cannot tell a chord from a registered combo. The chord is a + second *way in*, not a second thing to handle, and every consumer downstream already + knows what "toggle" means. + """ + + def __init__(self, presses, mods=frozenset({"ctrl", "win"}), + action: str = "talk", warm_action: str = "warm", + end_action: str = "talk-end", break_action: str = "talk-break", + gesture: str = None, toggle_action: str = "toggle") -> None: + self.presses = presses + self.mods = mods + #: Put when the chord forms: start capturing, the hold is the utterance. + self.action = action + #: Put immediately before `action`, so the models load during the hold rather + #: than inside the first sentence. `Session.warm` carries that argument in full. + self.warm_action = warm_action + #: Put when a chord modifier is released and the hold was clean — stop, and send + #: what was said. + self.end_action = end_action + #: Put when a third key lands *during* a hold that had already started. Windows + #: owns `ctrl+win+d` and `ctrl+win+arrow`, so this is the common case and not the + #: exotic one, and it has to stop the capture the press-down opened. Distinct + #: from `end_action` because what happens to the audio differs: this one does not + #: paste, and the session decides between dropping it and keeping it on screen + #: based on how much of it there is. + self.break_action = break_action + #: Which gesture this chord *is* — `"hold"` for push-to-talk, `"toggle"` for the + #: press-and-release that starts and stops hands-free listening. + #: + #: **A plain attribute, read inside the hook, so it can be changed at runtime.** + #: Rebuilding the `Chord` to switch would mean tearing down a `WH_KEYBOARD_LL` + #: hook and installing another one, which is the one operation in this file the + #: OS is entitled to refuse — and being refused *while changing a setting* would + #: leave somebody with no chord at all and no obvious way back. One string + #: assignment cannot fail. + #: + #: The two are genuinely different gestures rather than one with a flag, and the + #: reason they are both here is that they are good at different things: a hold is + #: better for a sentence, and a toggle is the only one of the two that works for + #: a paragraph, a phone call, or anybody who cannot hold two keys down. + self.gesture = gesture if gesture in GESTURES else GESTURE_DEFAULT + #: What a `"toggle"` chord puts, on the release and nowhere else. Kept separate + #: from `action` so the two gestures do not have to agree about a word: a hold + #: starts capture and a toggle flips it, and calling both "talk" would make the + #: dispatch table lie about one of them. + self.toggle_action = toggle_action + self.installed = False + #: Which of `mods` are down right now. A set is the honest shape and the wrong + #: one here — this is touched on the input path of every keystroke on the + #: machine, so it is a plain dict of name -> bool, written in place. + self._down = {name: False for name in mods} + #: True once a key outside `mods` went down while the chord was forming. The only + #: thing this file ever learns about that key. + self._other = False + #: The modifiers this chord does *not* want, and whether each is held right now. + #: + #: Tracked separately from `_other` because the two answer different questions, + #: and one flag answering both got it wrong in exactly one direction. `_other` is + #: "was something pressed *during* this hold", and it has to reset when the chord + #: forms — otherwise holding Ctrl to click a link, tapping a key, then adding Win + #: would be refused for a keystroke that had nothing to do with the chord. But a + #: modifier that is *still down* when the chord forms is not history, it is part + #: of the shape: ctrl+shift+win is a different chord from ctrl+win, and telling + #: them apart means asking what is held, not what was pressed. + #: + #: Only modifiers get this treatment, and that is the privacy line holding: a + #: modifier's identity is already in `_CHORD_VKS`, while every other key on the + #: board still collapses to the single boolean above. + self._extra = {name: False for name in CHORD_NAMES if name not in mods} + #: True while every modifier in `mods` is held. Latched rather than recomputed so + #: that releasing them one at a time still fires exactly once. + self._armed = False + #: True between the press-down that started capturing and whatever ends it — a + #: release, or a third key. Separate from `_armed` because a hold that formed + #: with an unwanted modifier already down arms without ever starting, and the end + #: has to know whether there is anything to end. Nothing about it is a keystroke: + #: it is the same single boolean `_other` is, and for the same reason. + self._talking = False + self._hook = None + self._tid = None + self._ready = threading.Event() + #: The callback is handed to the OS, which does not keep a Python reference to + #: it. Without this attribute it would be collected while still installed, and + #: the process would die inside a keystroke somewhere unrelated. + self._proc = _HOOKPROC(self._on_key) + self._thread = threading.Thread(target=self._run, daemon=True, name="chord") + + def start(self, timeout: float = 2.0) -> bool: + self._thread.start() + self._ready.wait(timeout) + return self.installed + + def _install(self): + """Ask for the hook, and treat every way of not getting one the same. + + `SetWindowsHookExW` answers NULL when the OS refuses — policy, another process, + a desktop this one cannot reach into. It is *also* the call that would raise if + this build ever ran somewhere the symbol is missing. Both are the same event to + everyone upstream: there is no chord, the registered toggle still works, and the + startup block says so on one line. + """ + try: + return user32.SetWindowsHookExW(WH_KEYBOARD_LL, self._proc, None, 0) + except (AttributeError, OSError, ValueError): + return None + + def stop(self) -> None: + if self._tid is not None: + user32.PostThreadMessageW(self._tid, WM_QUIT, 0, 0) + + def describe(self) -> str: + return describe_chord(self.mods) + + def _break(self) -> None: + """A third key landed mid-hold, so Windows meant something else. Stop capturing. + + Under the old toggle gesture this needed no code at all: nothing had started, so + refusing to fire was the whole behaviour. Push-to-talk opens the microphone on + the press-down, which means `ctrl+win+d` now has something to undo — and it must + be undone here, on the keystroke, rather than left for the release. Holding + `ctrl+win` through three desktop switches would otherwise record all of them. + + Once per hold. `_talking` is cleared first, so the arrow key that follows the + first arrow key does not put a second break on the queue. + """ + if self._talking: + self._talking = False + self.presses.put(self.break_action) + + def _on_key(self, code, wparam, lparam): + # Negative `code` means "pass it on without looking", and it is not advice. + if code >= 0: + vk = ctypes.cast(lparam, ctypes.POINTER(_KBDLLHOOKSTRUCT)).contents.vkCode + name = _CHORD_VKS.get(vk) + if wparam == WM_KEYDOWN or wparam == WM_SYSKEYDOWN: + if name is not None and name in self._down: + self._down[name] = True + if not self._armed and all(self._down.values()): + # A fresh hold starts a fresh verdict: whatever was *pressed* + # before the chord formed is not this chord's business. What is + # still *held* is — see `_extra`. + self._armed = True + self._other = any(self._extra.values()) + if not self._other and self.gesture == "hold": + # The hold has begun. Two puts and no other work — the rule + # about what this callback may do on the input path of every + # keystroke on the machine is unchanged. + # + # Nothing at all in the toggle gesture: it has no press-down + # half, and warming on one would load the models every time + # somebody reached for `ctrl+win+arrow`. + self._talking = True + self.presses.put(self.warm_action) + self.presses.put(self.action) + elif name is not None: + # A modifier this chord does not want. Held state, not history. + self._extra[name] = True + self._other = True + self._break() + else: + # Every other key on the keyboard. One boolean, and nothing else + # about it is read, kept or compared. + self._other = True + self._break() + elif wparam == WM_KEYUP or wparam == WM_SYSKEYUP: + if name is not None and name in self._down: + self._down[name] = False + if self._armed: + self._armed = False + if self._talking: + self._talking = False + self.presses.put(self.end_action) + elif self.gesture == "toggle" and not self._other: + # The original gesture, unchanged: a clean release — both + # held, nothing else touched — flips hands-free listening. + # `_other` is the same rule doing the same job it always + # did, which is why `ctrl+win+d` still makes a desktop and + # starts nothing. + self.presses.put(self.toggle_action) + elif name is not None: + self._extra[name] = False + return user32.CallNextHookEx(self._hook, code, wparam, lparam) + + def _run(self) -> None: + self._tid = kernel_thread_id() + try: + # `hMod` NULL with a thread id of 0 is the documented shape for a low-level + # hook: unlike the other WH_ hooks it is not injected into other processes, + # so it needs no module handle and the callback stays on this thread. + self._hook = self._install() + self.installed = bool(self._hook) + finally: + # Set in `finally` and not after, because `start()` is *waiting* on it. An + # exception on the line above would otherwise leave the launch blocked for + # the full timeout and then report "unavailable" for a reason nobody could + # find — the traceback goes to a daemon thread's stderr, which on a windowed + # build is nowhere at all. + self._ready.set() + if not self._hook: + return + msg = wintypes.MSG() + while user32.GetMessageW(ctypes.byref(msg), None, 0, 0) > 0: + pass + user32.UnhookWindowsHookEx(self._hook) + self._hook = None diff --git a/flow/profile.py b/flow/profile.py index 0a6054e..02ea0b2 100644 --- a/flow/profile.py +++ b/flow/profile.py @@ -133,6 +133,29 @@ def _hotkeys(value) -> dict: return dict(value) if isinstance(value, dict) else {} +def _apps(value) -> dict: + """The per-app instructions as written: a table, or nothing. + + Shape only, exactly like `_hotkeys`, and the line it stops at is the same one. What + counts as an application name is `flow/inject.py`'s knowledge — it is whatever + `Target.process` reports — and that module binds `user32` at import, so it cannot be + read on a Mac while this one is loaded on every launch including Lite's. + + But the second half of `_hotkeys`' argument does not apply here, and the difference + is worth being clear about: an action name has five right answers and a typo is + knowable, whereas an executable name has as many right answers as there are programs + on the machine. There is no table to check against and no such thing as a name Flow + can prove wrong — an entry for an app that is not installed is not a mistake, it is + somebody who has not opened it yet. So nothing here refuses an entry by name the way + registration does, and a key that never matches simply never fires. + + Values are carried through untouched, including ones that are not strings. The + instruction is judged where it is used, which is the only place that knows whether it + has anything to say. + """ + return dict(value) if isinstance(value, dict) else {} + + def _counter(value) -> Counter: """`{phrase: positive count}`. Entries that are not that shape are dropped. @@ -170,6 +193,36 @@ def _counter(value) -> Counter: MAX_WORKSPACES = 5 +#: The shipped modifier-only chord (see `flow/hotkey.py`, `Chord`). +#: +#: Canonical *here* rather than in `flow/hotkey.py`, and the direction is deliberate: +#: this module is imported on every launch including Lite and on a Mac, while +#: `flow.hotkey` binds `user32` at import and cannot be imported at all off Windows. A +#: default the profile could not read without Win32 would be a default Lite could not +#: write back, and a field that vanishes from the file on one platform is worse than one +#: nobody can edit. `flow.hotkey` owns what the string *means*; this owns what it is. +CHORD_DEFAULT = "ctrl+win" + +#: The shipped panel width, by name. Spelled here and not imported from `flow/ui.py` +#: for the reason `CHORD_DEFAULT` is not imported from `flow/hotkey.py`: this module is +#: loaded on every launch including Lite's, and a default it could not read without the +#: UI would be a default Lite could not write back. `ui.panel_width` maps the name to +#: pixels and treats an unknown one as this, so the two cannot disagree about anything +#: except the spelling of a word — which `tests/test_overlay.py` checks. +PANEL_DEFAULT = "regular" + +#: Where the stack sits, by name — bottom-centre of the monitor under the pointer, or +#: the bottom-right corner Flow shipped. Spelled here rather than imported from +#: `flow/ui.py` for `PANEL_DEFAULT`'s reason, and judged there for the same one. +PLACE_DEFAULT = "bottom" + +#: Which gesture the chord is: `"hold"` for push-to-talk, `"toggle"` for the original +#: press-and-release. Spelled here rather than imported from `flow/hotkey.py` for +#: `CHORD_DEFAULT`'s reason — that module binds user32 at import and cannot load on a +#: Mac, and this one is read on every launch including Lite's. +GESTURE_DEFAULT = "hold" + + def path_key(path: str | None) -> str | None: """One identity for one folder, however it was spelled. @@ -328,6 +381,27 @@ def __init__(self, path: Path | str | None = None) -> None: #: less than that: `_hotkeys` carries an entry whose value is not a string rather #: than dropping it, so that registration can refuse it by name. self.hotkeys: dict[str, str] = {} + #: The modifier-only chord, as written. `hotkey.parse_chord` judges what it + #: means, for the same reason `hotkeys` is judged there: this module is imported + #: on every launch including Lite, and `flow.hotkey` binds `user32` at import. + self.chord: str = CHORD_DEFAULT + #: exe name -> an extra instruction for rewrites made while that app is in front. + self.apps: dict[str, str] = {} + #: Which of `ui.PANEL_WIDTHS` the draft panel is drawn at. A name and not a + #: number: three widths that have each been drawn are worth more than an integer + #: nobody has rendered at, and the meaning is judged in `flow/ui.py` for the same + #: reason the hotkey table's is judged in `flow/hotkey.py` — this module is read + #: on every launch and must not need the module that knows what it means. + self.panel: str = PANEL_DEFAULT + #: Where the panels open. `"bottom"` is the shipped placement now — bottom-centre + #: of whichever monitor the pointer is on — and `"corner"` is the bottom-right + #: Flow used to use, kept for anybody who wants it back rather than removed. + self.place: str = PLACE_DEFAULT + #: How the chord behaves. Both gestures ship because neither replaces the other: + #: a hold is better for a sentence, a toggle is the only one that survives a + #: paragraph or a pair of hands that cannot hold two keys down. Judged in + #: `flow/hotkey.py`, which knows what the words mean. + self.gesture: str = GESTURE_DEFAULT #: Field names that were present in the file and unusable, so a caller can say so #: rather than leaving the user to notice their setting reverted. Empty on a first #: run and on any valid file. @@ -398,6 +472,16 @@ def take(key, validate, default=None, stored=None): # names live — see `_hotkeys`, and `hotkey.overridden` for what it says about # each entry it refuses. self.hotkeys = take("hotkeys", lambda v, _d: _hotkeys(v), {}) + # Absent means the shipped chord, and the empty string means "off" — somebody + # who does not want a global keyboard hook needs a way to say so that is not + # deleting the key, because the next save would write it straight back. + self.chord = take("chord", _text, CHORD_DEFAULT) + # Same bargain as `hotkeys`: a value that is not a table degrades to none and is + # named, and what is *in* the table is judged where it is used. + self.apps = take("apps", lambda v, _d: _apps(v), {}) + self.panel = take("panel", _text, PANEL_DEFAULT) + self.place = take("place", _text, PLACE_DEFAULT) + self.gesture = take("gesture", _text, GESTURE_DEFAULT) self.pairs = take("pairs", lambda v, _d: _counter(v), Counter()) self.misroutes = take("misroutes", lambda v, _d: _counter(v), Counter()) # `stored=[]` because JSON has no set: `save` writes this one as a sorted list, @@ -428,6 +512,14 @@ def save(self) -> bool: # the only advertisement this feature gets in a project with no settings # dialog to put it in. "hotkeys": dict(self.hotkeys), + "chord": self.chord, + # Written back as it was read, so a hand-edit survives every save Flow makes + # on its own — and an empty table lands in every profile, which is the only + # advertisement this feature gets in a project with no settings dialog. + "apps": dict(self.apps), + "panel": self.panel, + "place": self.place, + "gesture": self.gesture, "pairs": dict(self.pairs.most_common(MAX_PAIRS)), "misroutes": dict(self.misroutes.most_common(MAX_MISROUTES)), # Sorted so two saves of the same state produce the same file — a set's diff --git a/flow/refine.py b/flow/refine.py index 81bf8e1..a8b366f 100644 --- a/flow/refine.py +++ b/flow/refine.py @@ -136,6 +136,40 @@ def sane_timeout(value) -> float: _POLISH_SLACK = 600 +#: The per-app instruction, wrapped so the model can tell it from the user's own words. +#: +#: **Named as the destination rather than as a rule**, because that is what makes it +#: obey without over-obeying. "This text is going into Slack" is a fact the model can +#: weigh against the request; "always be informal" is a competing order, and a competing +#: order beats the instruction the user just spoke — which is the failure that would make +#: this feature worse than not having it. +#: +#: **Placed before the request, not after.** Both prompts end with the text, and the +#: sentence nearest the text is the one that wins ties. The user's actual instruction +#: has to be that sentence: a per-app note is a standing preference, and the whole point +#: of speaking is to override a standing preference when this one is different. +_APP_PROMPT = ( + "This text is going into {app}. Bear this in mind, without letting it override " + "anything asked for below: {note}" +) + + +def app_note(app: str, note) -> str: + """The `_APP_PROMPT` block for `app`, or "" when there is nothing to say. + + Everything that could be missing is treated as nothing to say, and deliberately so: + an unreadable entry in a hand-written table must cost that entry and not the rewrite. + The profile carries values through untouched (`profile._apps`), which is what leaves + the judging here — this is the only place that knows whether an instruction has any + content, and a non-string is simply an entry with none. + """ + if not isinstance(note, str) or not note.strip() or not app: + return "" + # `chr(10)` rather than an escape, the idiom this module already uses for every + # prompt it assembles — these strings are read far more often than they are edited. + return _APP_PROMPT.format(app=app, note=note.strip()) + chr(10) + chr(10) + + @dataclass(frozen=True) class Cli: name: str @@ -916,6 +950,7 @@ def refine( context: list[str] | None = None, cancel: threading.Event | None = None, skipped: list[str] | None = None, + app: str = "", ) -> tuple[str | None, str]: """Apply a semantic instruction to `text`. @@ -930,6 +965,10 @@ def refine( `cancel` abandons the call — the session sets it on close, so quitting does not wait out a rewrite nobody is going to read. + `app` is the per-app block from `app_note()` — already formatted, because the + profile table it comes from is the session's to read and this module has no business + knowing that `~/.flow/profile.json` exists. + `skipped` is an out-parameter rather than a third return value, and that is a judgement about blast radius: the two-tuple is unpacked at fifteen call sites across the tests and two scripts, none of which care which CLI was passed over on the way to @@ -947,6 +986,12 @@ def refine( if polish else _PROMPT.format(instruction=instruction, text=tail) ) + # Applied to both routes on purpose. A polish ignores the spoken instruction, which + # makes it the pass with the *most* to gain from knowing where the words are headed — + # a prompt bound for a terminal and one bound for a chat window differ in exactly the + # way this note is for. + if app: + prompt = app + prompt if context: prior = chr(10).join(f"- {turn}" for turn in context) prompt = ( diff --git a/flow/session.py b/flow/session.py index 8109eac..ea25246 100644 --- a/flow/session.py +++ b/flow/session.py @@ -52,7 +52,7 @@ from .profile import path_key from .refine import TIMEOUT_SEC as REFINE_TIMEOUT_SEC from .refine import MAX_CHARS as REFINE_MAX_CHARS -from .refine import ask, available, refine, tail_sent +from .refine import app_note, ask, available, refine, tail_sent from .thread import ASK_CONTEXT_CHARS, Thread # -- P4/P8: what a repair typed by hand teaches ------------------------------- @@ -295,11 +295,39 @@ def ask_framing(cwd: str | None) -> str: #: worker-idle check below, this is what bounds partial latency. PARTIAL_MIN_GROWTH_SEC = 0.7 -#: R8: drop the 141 MB model after a long quiet spell. The mic stays open — it is -#: cheap, and keeping it means speech still wakes the session with no keypress. This -#: is a deliberate narrowing of what docs/analysis.md §4 proposed (which released the -#: mic too): releasing it would make the app unable to hear its own wake-up. -IDLE_UNLOAD_SEC = 300.0 +#: R8: drop the models after a long quiet spell. The mic stays open — it is cheap, and +#: keeping it means speech still wakes the session with no keypress. This is a +#: deliberate narrowing of what docs/analysis.md §4 proposed (which released the mic +#: too): releasing it would make the app unable to hear its own wake-up. +#: +#: **"the 141 MB model" is what this comment used to say, and it was one tier out of +#: date.** Two are resident — `base.en` for partials at 141 MB and `small.en` for finals +#: at 464 MB — so what the idle path gives back is ~605 MB, not 141. The number is worth +#: correcting rather than rounding past: it is the entire case for unloading at all, and +#: it is four times better than the sentence defending it claimed. +#: +#: **Thirty minutes, not five.** Five was measuring the wrong thing — it asked how long +#: the session had been quiet, and answered as if quiet meant gone. It does not: the gaps +#: inside an ordinary working session run well past five minutes, so the common case was +#: not "reclaim memory from somebody who left", it was "pay a reload in the middle of +#: somebody's first sentence back". Half an hour rides out the gaps in a day and still +#: hands the memory back overnight, which is the case the unload was written for. +IDLE_UNLOAD_SEC = 1800.0 + +#: How long a warm request holds the models against the idle unload. +#: +#: Exists because the two clocks disagreed about what "idle" means. `_last_activity` is +#: only moved by Flow's own milestones, so a person who has just *reached for the chord* +#: is still idle by that measure — and the health pump, which runs every tick, could +#: unload the models between the press and the release that starts capture. That is the +#: one moment the warm exists to cover, and it is exactly the moment it would have lost. +#: +#: A grace window rather than a touch of `_last_activity`, because they answer different +#: questions and conflating them costs the unload its meaning: a chord press-down also +#: arrives from Windows' own `ctrl+win+arrow`, and letting that reset the idle clock +#: would mean anybody who switches virtual desktops through the day never unloads at +#: all. Sixty seconds covers press-hold-release-speak and then stops mattering. +WARM_GRACE_SEC = 60.0 #: How long to wait between attempts to reopen a microphone that went away. #: @@ -837,6 +865,16 @@ def __init__( #: None disables learning entirely — the tests and the benchmarks pass None so #: a harness run never writes to the user's real profile. self.profile = profile + #: The executable in front, as `inject.Target.process` spells it, or "". + #: + #: Written by the pill rather than read here, and that is the split that keeps + #: `OpenProcess` off a 30 fps path: `Pill._track_target` already asks who has the + #: foreground every frame, so it is the one place that knows when the answer + #: *changed* and a name only has to be resolved then. Lite leaves it empty — it + #: has no target-window awareness at all (product.md) — which reads here as an + #: app with nothing configured, and that is the correct behaviour rather than a + #: gap: a rewrite with no per-app note is what every launch did until now. + self.target_app = "" #: A content-free shadow of the event stream (see flow/diag.py). Off unless #: the caller passes one, for the same reason `profile=None` disables learning: #: the tests build sessions in their hundreds, and a default that wrote to @@ -923,6 +961,13 @@ def __init__( #: The one preload this session owns, or None before the first arm. Single, not #: one per arm: see `_warm`. self._preload_thread: threading.Thread | None = None + #: Deadline until which the idle unload stands down, set by `warm`. Zero means + #: nobody has asked, which is the state every session starts and mostly stays in. + self._warm_until = 0.0 + #: Whether the current push-to-talk hold is the thing that opened the mic. False + #: when the hold began against a session already capturing, which is what stops + #: a chord from closing a microphone the toggle hotkey opened. + self._ptt_opened = False # -- lifecycle --------------------------------------------------------- @@ -968,6 +1013,30 @@ def start(self) -> None: self._warm() self._set_state(State.IDLE) + def warm(self) -> None: + """Start loading the models now, before anybody asks them to decode anything. + + The chord's press-down calls this, and the release that follows is what actually + arms. That gap — a person holding two modifiers, about to let go and speak — is + free time the load used to spend nowhere: `start()` deliberately does not await + the preload (see the docstring there, and the first-run download it exists to + keep off the UI thread), so on a cold arm the reload landed *inside* the first + utterance instead of in front of it. Measured on the run that prompted this: + first partial 1 230 ms, the four behind it ~570 ms. + + Safe to call at any moment and as often as anybody likes. `_warm` is + single-flight and finds an already-loaded model instantly, so the cost of a + spurious call is a thread that starts and exits — which matters, because + `ctrl+win` is also the prefix Windows uses for `ctrl+win+d` and `ctrl+win+arrow` + and the hook cannot tell those from a real chord until the third key lands. + + The grace window is the half that is not just a preload: without it the health + pump, which runs every tick, is free to unload between this call and the arm it + is preparing for. See `WARM_GRACE_SEC`. + """ + self._warm_until = time.perf_counter() + WARM_GRACE_SEC + self._warm() + def _warm(self) -> None: """One preload at a time, however often this session is armed. @@ -1064,6 +1133,85 @@ def stop_speaking(self) -> bool: self._emit("note", "stopped reading the answer") return True + # -- push to talk ------------------------------------------------------ + # + # The chord's two halves, as session verbs. `Pill` owns the state machine between + # them — it is the thread that may touch Tk and the one that already knows what + # `armed` means — and these are the two things it cannot do from outside: opening + # capture for the length of a hold, and closing it *without* discarding what was + # said into it. + + @property + def busy(self) -> bool: + """True while a decode this session submitted is still in flight. + + Exists for the push-to-talk paste, which must wait for the *final* rather than + paste the partial that preceded it. Narrow on purpose: it answers "is the + decoder still working", not "is Flow doing anything" — a CLI refine has its own + state, and `send()` already refuses while one is running. + """ + return self.worker.busy + + def talk_start(self) -> bool: + """Open the microphone for a hold. True if capture is running when this returns. + + Idempotent against a session that is already capturing, and the return value is + the reason: somebody who armed with the toggle hotkey and then reaches for the + chord has a live microphone already, and re-opening it would cut the utterance + they are in the middle of. `talk_end` reads the same fact from `_ptt_opened` and + gives back only what this took. + + Raises what `start()` raises — no microphone, a device held exclusively by + something else. The caller renders that; a swallowed failure here would be a + hold that records nothing and says so nowhere. + """ + with self._lifecycle: + if self._closed: + return False + if self._mic_started: + self._ptt_opened = False + self.warm() + return True + self.start() + self._ptt_opened = True + return True + + def talk_end(self) -> bool: + """Close a hold. True if there is now a decode in flight worth waiting for. + + **`mic.stop()` and not `pause()`, and the difference is the whole gesture.** + `pause()` bumps the capture generation, which is precisely how a deliberate stop + refuses results decoded from before it — and under push-to-talk the words the + user just said are in flight *at this moment*. Pausing here would throw away the + utterance the release exists to send. `_give_up_on_device` reached the same + conclusion from the other direction and the comment there is the longer version. + + **What was said is always committed, including on the break path**, and there is + deliberately no minimum-length rule deciding otherwise. `_utter` is what the gate + let through, so a hold with nothing spoken into it is already empty and + `_finalise` already returns early — a `ctrl+win+d` costs nothing without a + threshold, and a threshold would be a number that eventually eats somebody's + one-word answer. Whether the words get *pasted* is the caller's decision and a + different question; whether they are kept is not up for debate (P2). + """ + pending = bool(self._utter) + self._finalise() + if self._ptt_opened: + try: + self.mic.stop() + except Exception: + pass # already gone; there is nothing left to close + self._mic_started = False + # The hygiene `pause()` does either side of the generation bump, minus the + # bump: a gate left open would resume the next hold mid-utterance with no + # onset, and blocks captured after the stop belong to nobody. + self.gate.reset() + self.mic.drain() + self._ptt_opened = False + self._settle_state() + self._emit("disarm", "push-to-talk") + return pending + def close(self) -> None: """Give back everything `start()` and the constructor took, in that order. @@ -1381,6 +1529,7 @@ def _pump_health(self) -> None: idle = now - self._last_activity if ( idle >= IDLE_UNLOAD_SEC + and now >= self._warm_until and not self.draft.text and not self.gate.speaking and not self.worker.busy @@ -2804,6 +2953,28 @@ def _pump_auto_ask(self) -> None: # -- semantic refine (off-thread: ~7 s measured) ------------------------ + def _app_note(self) -> str: + """The per-app block for whatever is in front, or "". + + Matched case-insensitively on the executable name, because `"Code.exe"` and + `"code.exe"` are the same program and a table that cared would be a table whose + entries silently stop matching after a vendor changes the capitalisation of a + shipped binary. + + Every way of having nothing to say lands on "" — no profile, no table, an app + with no entry, an entry that is blank or is not a string. A rewrite without a + note is exactly what Flow did before this existed, so the degraded path is the + old behaviour rather than an error. + """ + table = getattr(self.profile, "apps", None) if self.profile else None + if not table or not self.target_app: + return "" + wanted = self.target_app.lower() + for name, note in table.items(): + if isinstance(name, str) and name.strip().lower() == wanted: + return app_note(self.target_app, note) + return "" + def _start_refine(self, instruction: str, *, polish: bool = False) -> None: if self._refine_op is not None: # The refusal `send()` already makes, for the same reason. Two rewrites of @@ -2839,6 +3010,13 @@ def _start_refine(self, instruction: str, *, polish: bool = False) -> None: ) context = self.thread.tail() if self.following_up else [] + # Resolved here and not on the worker, because `target_app` is written by the UI + # thread every frame and the worker runs for the ~7 s the CLI takes. Reading it + # there would let the app the user tabbed to *during* the rewrite decide how the + # words they already spoke come out. + app = self._app_note() + if app: + self._emit("note", f"using your {self.target_app} note") def work() -> None: passed_over: list[str] = [] @@ -2846,7 +3024,7 @@ def work() -> None: before, instruction, cwd=self._refine_cwd, polish=polish, context=context, cancel=self._cancel, cli=self._cli, timeout=self._cli_timeout, - skipped=passed_over, + skipped=passed_over, app=app, ) with self._refine_lock: self._refine_result = (op, revision, result, tuple(passed_over)) diff --git a/flow/ui.py b/flow/ui.py index 3c37b2f..1d0b12f 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -13,6 +13,7 @@ from __future__ import annotations import ctypes +import math import os import sys import time @@ -53,6 +54,10 @@ class _RECT(ctypes.Structure): ("right", ctypes.c_long), ("bottom", ctypes.c_long)] +class _POINT(ctypes.Structure): + _fields_ = [("x", ctypes.c_long), ("y", ctypes.c_long)] + + #: SystemParametersInfo(SPI_GETWORKAREA) _SPI_GETWORKAREA = 0x0030 @@ -72,7 +77,7 @@ def __getattr__(self, _name): if sys.platform == "win32": - from .inject import foreground_hwnd, owned_by_flow, take_warnings + from .inject import classify, foreground_hwnd, owned_by_flow, take_warnings #: Its own handle rather than `ctypes.windll.user32`, which is a process-wide cached #: object: declaring `restype` on it would change the signature under `inject.py` @@ -93,12 +98,21 @@ def __getattr__(self, _name): _set_style.restype = ctypes.c_ssize_t else: # `inject.py` is not made portable and is not imported: 470 lines of Win32 with no - # meaning in a body that never types into another window. Its three exports are the + # meaning in a body that never types into another window. Its four exports are the # only ones this module needs, and each has an honest answer with no hands — there is - # no foreground to find, nothing of Flow's to recognise, and no paste to warn about. + # no foreground to find, nothing of Flow's to recognise, no window to name, and no + # paste to warn about. def foreground_hwnd() -> int: return 0 + def classify(_hwnd): + # An unnamed app, which reads downstream as one with no per-app note — the same + # answer `_track_target` gives on Lite, and the behaviour every launch had before + # per-app notes existed. + from .inject import Target + + return Target() + def owned_by_flow(_hwnd) -> bool: return False @@ -200,6 +214,158 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: return rect.left, rect.top, rect.right, rect.bottom +#: `MonitorFromPoint`'s "nearest monitor" flag, for a cursor that is briefly nowhere — +#: between two displays, or on a monitor that has just been unplugged. +_MONITOR_DEFAULTTONEAREST = 2 + + +class _MONITORINFO(ctypes.Structure): + _fields_ = [ + ("cbSize", ctypes.c_ulong), + ("rcMonitor", _RECT), + ("rcWork", _RECT), + ("dwFlags", ctypes.c_ulong), + ] + + +def _pointer_monitor(sw: int, sh: int) -> tuple[tuple, tuple]: + """`(full, work)` for the monitor under the mouse, each `(left, top, right, bottom)`. + + **Two rectangles, because FluidVoice places against two.** `positionWindow` centres + on `screen.frame` but sits the overlay on `screen.visibleFrame`, and the asymmetry is + deliberate: centred on the *physical* display, so it lands where the eye expects it, + but lifted clear of the Dock. Windows hands back exactly that pair — `rcMonitor` and + `rcWork` — from one call, so the rule ports without being reinterpreted. + + **The monitor under the pointer, not the primary one.** `_work_area` asks + `SystemParametersInfoW`, which only ever answers for the primary display, so on a + two-monitor desk everything Flow draws lands on the wrong one whenever the user is + working on the other. FluidVoice resolves this per presentation + (`OverlayScreenResolver.screenForCurrentPointer`, and `preferredPresentationScreen` + in the notch path does the same), and the pointer is the right proxy: it is where + the user's attention is, and it costs nothing to ask. + + Falls back to the primary work area, then to the whole screen, so a machine where + the call is unavailable places exactly where it placed before. + """ + pt = _POINT() + try: + user32 = ctypes.windll.user32 + if user32.GetCursorPos(ctypes.byref(pt)): + handle = user32.MonitorFromPoint(pt, _MONITOR_DEFAULTTONEAREST) + info = _MONITORINFO() + info.cbSize = ctypes.sizeof(_MONITORINFO) + if handle and user32.GetMonitorInfoW(handle, ctypes.byref(info)): + full = (info.rcMonitor.left, info.rcMonitor.top, + info.rcMonitor.right, info.rcMonitor.bottom) + work = (info.rcWork.left, info.rcWork.top, + info.rcWork.right, info.rcWork.bottom) + if full[2] > full[0] and work[2] > work[0]: + return full, work + except (AttributeError, OSError): + pass + work = _work_area(sw, sh) + return work, work + + +#: `GetSystemMetrics` indices for the bounding box of every monitor together. +_SM_XVIRTUALSCREEN, _SM_YVIRTUALSCREEN = 76, 77 +_SM_CXVIRTUALSCREEN, _SM_CYVIRTUALSCREEN = 78, 79 + +#: How far past the desktop a hidden panel is parked. FluidVoice's number +#: (`parkWindowOffscreen`), and generous on purpose: it has to clear whatever monitor +#: someone plugs in next, not merely the ones present when the window was hidden. +PARK_MARGIN = 1024 + + +def _virtual_desktop(sw: int, sh: int) -> tuple[int, int, int, int]: + """Every monitor's bounding box, which is what a parked window has to clear. + + The union rather than the current monitor, for the same reason FluidVoice unions + `NSScreen.screens`: a window parked past the right edge of the *left* display in a + two-monitor desk is parked in the middle of the right one, in full view. + """ + try: + metric = ctypes.windll.user32.GetSystemMetrics + x, y = metric(_SM_XVIRTUALSCREEN), metric(_SM_YVIRTUALSCREEN) + w, h = metric(_SM_CXVIRTUALSCREEN), metric(_SM_CYVIRTUALSCREEN) + if w > 0 and h > 0: + return x, y, x + w, y + h + except (AttributeError, OSError): + pass + return 0, 0, sw, sh + + +def park_spot(w: int, h: int, desktop) -> tuple[int, int]: + """Where a hidden panel waits: past the far corner of every monitor there is. + + **Parked rather than unmapped, which is the point.** FluidVoice never destroys or + hides its overlay — `prepare()` builds it at launch and parks it, and `show` pulls + it back — with the reason written on the method: it is paying the window-server + surface cost once, so that appearing costs a move and nothing else. Under + push-to-talk that is the difference that matters, because the overlay now has to be + up between a key going down and somebody starting to talk, which is a tenth of a + second on a fast day. + + Its own function so the arithmetic is testable without a window, and so the one + thing that must never be true — a parked panel landing on a monitor somebody is + looking at — is a property a test can state. + """ + _left, _top, right, bottom = desktop + return right + w + PARK_MARGIN, bottom + h + PARK_MARGIN + + +def park(win) -> None: + """Hide `win` by moving it off every monitor, rather than by unmapping it. + + `withdraw()` was what the panels did, and it is the honest thing for a window nobody + will want again soon. Push-to-talk made that untrue: a panel now has to be up between + a key going down and somebody starting to speak, and a remap is work done in exactly + that gap. Parking is FluidVoice's answer (`parkWindowOffscreen`, with `prepare()` + paying the surface cost at launch), and it makes appearing a move and nothing else. + + A function rather than a method because `Bubble` and `ConversationCard` share no base + class — they share a *job*, and this is the third thing they have both needed. The + window is re-placed by `reposition` on the way back, which `_render` already calls, + so there is no unparking step to forget. + """ + w = max(1, win.width) + h = max(1, getattr(win, "_h", 1)) + x, y = park_spot(w, h, _virtual_desktop( + win.winfo_screenwidth(), win.winfo_screenheight())) + win.geometry(f"{w}x{h}+{x}+{y}") + + +def bottom_centre(w: int, h: int, full, work, offset: int = 0) -> tuple[int, int]: + """Where a panel of `w`×`h` goes, by FluidVoice's `positionWindow` arithmetic. + + Centred horizontally on `full` and stood `offset` above the bottom of `work`, then + clamped into `work` with the same two buffers FluidVoice uses — 10 px off the bottom + and 40 px off the top. The clamp is what makes the offset safe to expose as a + setting: a number typed into a profile cannot push the panel off the screen, and a + panel taller than the display lands against the bottom rather than above the top. + + The y arithmetic is flipped from the original and means the same thing. macOS + measures up from the bottom, so `visibleFrame.minY + offset` is the panel's bottom + edge; Windows measures down from the top, so the same edge is + `work.bottom - h - offset`. + """ + left, top, right, bottom = work + x = (full[0] + full[2]) // 2 - w // 2 + y = bottom - h - int(offset) + lowest = bottom - h - 10 + highest = top + 40 + if highest > lowest: + # A panel too tall for the display it is on. The bottom is the edge worth + # keeping: the top of a draft can run under the taskbar and still be read, and + # the chips that act on it live at the bottom. + y = lowest + else: + y = max(highest, min(y, lowest)) + x = max(left, min(x, right - w)) + return x, y + + def _dpi_aware() -> float: """Tell Windows this process draws its own pixels, and return the scale factor. @@ -409,6 +575,34 @@ def _unload_fonts() -> None: BAR_W, BAR_GAP = 4, 2 DB_FLOOR, DB_CEIL = -58.0, -12.0 # level range mapped onto bar height +#: The meter's shape, taken from FluidVoice's `BottomWaveformView` (`visualizerPeakHeight` +#: and `updateBars`) rather than invented here. +#: +#: **This changed what the meter *is*.** Flow's bars used to be a scrolling history — a +#: level per frame, pushed through a deque, so the shape travelled right to left like a +#: seismograph. FluidVoice's are a symmetric bloom: every bar is driven by the *same* +#: current level and shaped by a fixed envelope that makes the middle ones tallest, so +#: the meter breathes in place instead of scrolling. +#: +#: The bloom is the better answer to the question R13 says this widget exists to answer — +#: *am I being heard right now*. A history answers "was I heard, recently", and the eye +#: has to read left to right to get at it; a bloom answers it in one glance with no +#: direction to follow. It is also what makes the meter read as one object rather than +#: twelve, which is the whole visual difference between the two apps. +#: +#: `_ENVELOPE_FLOOR`/`_ENVELOPE_SPAN`: `factor = max(0.18, 0.96 - distance * 0.78)`, +#: where distance is 0 at the centre bar and 1 at the ends. `_LEVEL_EXPONENT`: normal +#: speech should push the bars high, so the response is deliberately not linear. +#: `_BAR_VARIATION`: a per-bar wobble of ±8%, so a held tone is not twelve identical +#: rectangles. +_ENVELOPE_FLOOR, _ENVELOPE_SPAN, _ENVELOPE_MIN = 0.96, 0.78, 0.18 +_LEVEL_EXPONENT = 0.55 +_BAR_VARIATION_BASE, _BAR_VARIATION_SWING, _BAR_VARIATION_RATE = 0.92, 0.08, 1.45 +#: Half-heights, in pixels, at the ends of the response. The minimum is what silence +#: draws — a flat line of stubs rather than an empty box, which is the one thing Flow's +#: old meter and FluidVoice's agree on. +BAR_MIN_H, BAR_MAX_H = 1.5, 12.0 + #: Where the meter starts and how wide it ends up — named because the bar label has to #: begin after it, and two places computing `BARS * (BAR_W + BAR_GAP)` is how the label #: would come to be drawn through the twelfth bar the day one of them changed. @@ -524,6 +718,50 @@ def _unload_fonts() -> None: #: How long the lift back to full opacity takes once the pointer arrives. HOVER_LIFT_SEC = 0.4 +#: The ceiling on one push-to-talk hold, after which Flow stops capturing on its own. +#: +#: Not a limit on how long anybody may speak — it is a limit on how long a *missing +#: keystroke* may hold the microphone open. A keyup can genuinely fail to arrive: the OS +#: drops a low-level hook that overran `LowLevelHooksTimeout`, a lock screen or a UAC +#: prompt takes the input desktop mid-hold, an RDP session grabs the keyboard. Without +#: this the failure is a session recording a room until somebody notices. +#: +#: Two minutes because it has to sit clear of the longest hold anybody would make on +#: purpose without being so far out that the recording is a surprise. Whatever was said +#: is committed and kept, and the note says where it went — the one thing this must not +#: do is end a real dictation by discarding it. +PTT_MAX_HOLD_SEC = 120.0 + +#: How long the pill must be held before a press becomes a hold-to-talk rather than a +#: click, and how far the pointer may travel before it is a drag instead of either. +#: +#: **This is push-to-talk for everybody who has no chord**, which on a Mac is everybody: +#: `Chord` is a `WH_KEYBOARD_LL` hook and there is no such thing off Windows. The +#: gesture is the same one — press, speak, release, and the words are yours — but the +#: button is a window Flow already draws, so it costs no Accessibility permission, no +#: Input Monitoring, and no signed bundle to ask for them from. That is the one thing +#: Flow Lite can do that a native app driving a system hotkey cannot. +#: +#: 300 ms because three gestures now share one button and the other two are older: a +#: deliberate click is well under 200 ms, and a drag declares itself by moving. Slop is +#: 4 px rather than 0 because a hand resting on a mouse is not perfectly still, and a +#: hold that lost its nerve on one pixel of tremor would be a gesture nobody could rely +#: on. +PILL_HOLD_SEC = 0.30 +PILL_DRAG_SLOP = 4 + +#: How long the release waits for the decode it is going to paste. +#: +#: The paste cannot be synchronous. A final decode measured 0.7-7 s on the machine this +#: was built for, and the release has to return to the frame loop long before that. So +#: the wait is a state, and this is its ceiling. +#: +#: Fifteen seconds is past the worst final in that trace by a wide margin, and the +#: behaviour at the ceiling is not a discard: the words are in the draft, on screen, +#: and the note points at the Send chip. A paste that lands a minute late would arrive +#: in whatever window the user has since moved to, which is worse than not pasting. +PTT_PASTE_WAIT_SEC = 15.0 + #: Unified with `CARD_W` (decisions.md 2026-08-09, Phase 6): the widest state either #: panel reaches is the draft's full rescue row — Refine, Continue, Edit, "Was a #: command", Send, 345 px of chip width — and at the old 380 that left `chip_row_gap` @@ -603,23 +841,146 @@ def _unload_fonts() -> None: PARTIAL_GAP = 6 #: Characters a line of body text holds, measured on the real canvas at the body font and -#: `BUBBLE_W - 2 * PAD` = 352 px: 3 160 characters of ordinary prose wrapped to 56 lines, -#: so 56.4. Two things read it, and neither may cost a layout — how much draft is worth -#: handing the canvas, and how many lines are above what it shows. -BODY_CHARS_PER_LINE = 56 +#: the shipped 392 px column (`BUBBLE_W - 2 * PAD`): 3 160 characters of ordinary prose +#: wrapped to 51 lines, so 62.0. Two things read it, and neither may cost a layout — how +#: much draft is worth handing the canvas, and how many lines are above what it shows. +#: +#: **Rebound by `apply_panel_width` at launch**, because the column is a setting now. +#: This is the value at the shipped width, kept here so the module still reads straight +#: through and so a launch that never calls that function is the launch Flow always had. +#: The old 56 was measured at a 352 px column — `380 - 2 * PAD`, the bubble before +#: Phase 6 — and stayed one panel width behind until the measurement was re-taken; see +#: `_CHARS_PER_PX`. +BODY_CHARS_PER_LINE = 62 #: The window of draft actually laid out per event, and the reason render cost stops #: growing: `BODY_MAX_H` holds 20 lines, this is enough characters for about 28 of them, so #: the visible tail is always full even where the text wraps early — and a two-hour #: dictation is laid out at the same cost as a two-minute one (invariant 7, extended to -#: rendering). Measured before and after on the real canvas: 2.4 / 32.7 / 476.7 ms at -#: 1k / 10k / 50k characters, and flat afterwards. -BODY_TAIL_CHARS = 1600 +#: rendering). Lines, not characters, are what is held constant here: the re-measured +#: column holds more per line, so the character count moved with it and the 28 did not. +#: Measured before and after on the real canvas at the 392 px column: 0.8 / 14.1 / +#: 221.3 ms at 1k / 10k / 50k characters, and flat at ~1.4 ms afterwards. (The older +#: 2.4 / 32.7 / 476.7 in this comment were the same shape on a slower machine.) +BODY_TAIL_CHARS = 1750 #: How far past the cut to look for a space before giving up and cutting mid-word. #: Bounded, because a scan that can run the length of the draft is the cost being avoided. BODY_BOUNDARY_SCAN = 200 +#: The panel widths on offer, and why the list starts where it does rather than lower. +#: +#: 420 is a **floor**, not a default somebody liked. Two measured rows put it there: +#: the bubble's five-chip row runs to 345 px (`chip_row_gap`, which records that 380 +#: clipped Send by half a label), and the card's runs to 377 of the same 420. A "small" +#: option would have to either drop a chip or ship a row that clips, so there is not +#: one — this is a setting for people who want the draft easier to read, and every +#: direction that helps with that is up. +#: +#: Named rather than free-form for the reason `KEYS` is: three widths that have each +#: been drawn are worth more than an integer nobody has rendered at. +PANEL_WIDTHS: dict[str, int] = {"regular": 420, "large": 520, "larger": 640} +PANEL_DEFAULT = "regular" + +#: Where the stack sits. `"bottom"` is bottom-centre of the monitor under the pointer, +#: which is FluidVoice's placement (`BottomOverlayView.positionWindow`) and now Flow's +#: default; `"corner"` is the bottom-right Flow shipped. `Pill._placed` carries the +#: argument for the change. Named rather than free-form for `KEYS`' reason — a position +#: somebody has actually looked at beats a pair of coordinates nobody has rendered. +#: What each chord gesture is called in the menu. Phrased as what it *does* rather +#: than as its name, because "hold" and "toggle" are the words in the profile and this +#: is the row somebody reads once while deciding — the whole sentence is the label. +GESTURE_LABELS = { + "hold": "Hold to talk, release to send", + "toggle": "Press to start, press again to stop", +} + +PLACES = ("bottom", "corner") +PLACE_DEFAULT = "bottom" +PLACE = PLACE_DEFAULT + +#: How far above the work area's bottom edge the stack stands, in pixels. +#: +#: FluidVoice exposes this as `overlayBottomOffset` and so does Flow, for the reason +#: they do: the bottom of the screen is where a taskbar auto-hides, where a browser +#: puts its download shelf, and where some apps park a status bar, so the one number +#: that makes the overlay sit clear of all that is worth a setting. 24 rather than +#: their 0 because Flow's stack has a pill under the panel and the pill is the part +#: that would touch the edge. +PANEL_BOTTOM_OFFSET = 24 + + +def apply_place(name: str) -> None: + """Set where the stack sits. Call before the first draw, beside `apply_panel_width`. + + A module global for that function's reason and with the same concession behind it: + the alternative is threading a placement through every window that positions itself, + and rebinding one name before anything is drawn is the smaller change whose failure + mode is visible immediately. Unknown names fall back rather than raising — this + arrives from a hand-edited profile, and a typo should cost the setting, not the app. + """ + global PLACE + PLACE = name if name in PLACES else PLACE_DEFAULT + +#: Body characters per line, per pixel of column. Kept as a *ratio* because the column +#: is no longer a fixed number: a wider panel holds proportionally more, and a +#: `BODY_CHARS_PER_LINE` frozen at one width would under-feed the canvas at 640 px and +#: quietly put the bottom of the draft below the fold. +#: +#: **Anchored at 420, which is now also where the measurement was taken.** The ratio +#: behind the old 56 came from a 352 px column — `380 - 2 * PAD`, the bubble *before* +#: Phase 6 took it to 420 — so it was one panel width behind, and the setting deliberately +#: reproduced it rather than smuggle a behaviour change into a size option. The +#: measurement has since been re-taken by the same method at the shipped 392 px column +#: (3 160 characters of prose to 51 lines, 62.0 a line), and this is that figure. +#: +#: Still written as `62 / 392` rather than the raw 0.1581 so the shipped width comes back +#: exactly, and so the number a reader can check against the canvas is the one in the +#: source. +_CHARS_PER_PX = 62 / (420 - 2 * PAD) + +#: Lines of draft handed to the canvas per layout, which is what `BODY_TAIL_CHARS` +#: actually encodes: `BODY_MAX_H` shows 20, and about 28 are laid out so the visible +#: tail is full even where the text wraps early. Held constant across widths, so the +#: render-cost invariant (7) survives a wider panel instead of being re-measured. +_TAIL_LINES = 1750 / 62 + + +def apply_panel_width(width: int) -> None: + """Set the panel width and everything measured off it. Call before the first draw. + + Module globals rather than instance state, and that is a deliberate concession + rather than a preference. `BUBBLE_W` and `CARD_W` are read from roughly twenty + places across two window classes and the free functions that draw their chrome, and + threading a width through all of them would be a large refactor whose only new + behaviour is this one setting. Rebinding three derived numbers once, before anything + is drawn, is the smaller change and the one whose failure mode is visible + immediately. + + Clamped at the floor rather than trusted. The number can come from a hand-edited + profile, and a panel narrower than its own chip row is a window whose Send button is + half off the edge — the one control that must never be unreachable. + """ + global BUBBLE_W, CARD_W, BODY_CHARS_PER_LINE, BODY_TAIL_CHARS + BUBBLE_W = CARD_W = max(int(width), PANEL_WIDTHS[PANEL_DEFAULT]) + BODY_CHARS_PER_LINE = max(1, int((BUBBLE_W - 2 * PAD) * _CHARS_PER_PX)) + BODY_TAIL_CHARS = int(BODY_CHARS_PER_LINE * _TAIL_LINES) + + +def panel_width(name) -> int: + """A width for a profile value, falling back to the shipped one. + + Anything unknown is the default rather than a refusal, which is the opposite of how + `hotkey.parse` treats a bad combo — and the difference is what the two settings cost + when wrong. A hotkey that silently fell back would leave somebody pressing keys that + do nothing with no way to find out; a panel that falls back is a window that is + visibly not the size they asked for, and the evidence is on the screen. + """ + if isinstance(name, str): + return PANEL_WIDTHS.get(name.strip().lower(), PANEL_WIDTHS[PANEL_DEFAULT]) + return PANEL_WIDTHS[PANEL_DEFAULT] + + #: The line saying what is not in the window, at the note's font plus its gap. One number #: for both of them — `… N earlier lines` above a draft and `… N more lines` below an #: answer — because they are the same line in the same font pointing opposite ways. @@ -981,6 +1342,25 @@ class Pill(tk.Tk): _dots_frame = 0 _tint = 0.0 _flash = 0 + #: Same reason a fourth time, for push-to-talk's two clocks. These are read by + #: `_toggle`, `_clear` and the mode switch — three paths a fixture drives directly + #: without ever having held the chord — so the default has to be "no hold, nothing + #: waiting" rather than a recursion. `None` is that in both cases. + _ptt_since: float | None = None + _ptt_wait: float | None = None + #: And for the three gestures now sharing the left button: when it went down, where, + #: whether it has travelled since, whether the press turned into an utterance, and + #: the `after` id that would turn it into one. All idle here, which is the state a + #: fixture that never touched a mouse should read as. + _press_at: float | None = None + _press_xy = (0, 0) + _press_moved = False + _press_talking = False + _press_timer = None + #: And once more for the settings menu's newest row, which asks the live chord what + #: gesture it is. `--no-hotkeys` leaves this None for real, so the default is not a + #: fixture convenience — it is the shipped value on one of the supported launches. + hotkeys = None def __init__( self, session: Session, on_send=None, hotkeys=None, arm=False, @@ -1005,7 +1385,12 @@ def __init__( Path(settings_path) if settings_path is not None else LEXICON_PATH ) self._arm_on_start = arm - self.levels: deque[float] = deque([0.0] * BARS, maxlen=BARS) + #: The level every bar is drawn from this frame, 0…1. One number, where this + #: used to be a `deque` of `BARS` of them: the meter blooms from its own centre + #: now rather than scrolling a history past, so there is no past to keep. See + #: `_bar_half_height`. The eased value still lives in `_eased_level`; this is + #: what the last frame actually drew, which `_flatten` needs to fade from. + self._meter_level = 0.0 #: The level actually drawn, eased toward `session.level_db` rather than #: jumping to it — rise 60 ms, fall 160 ms (§07), so peaks fall slower than #: they rise. Only the newest sample eases; everything already in `levels` is @@ -1024,6 +1409,16 @@ def __init__( self._hover_since: float | None = None self.armed = False self._disarmed_since = time.perf_counter() # starts disarmed, so the clock does too + #: When the current push-to-talk hold began, or None when no chord is held. The + #: clock exists for `PTT_MAX_HOLD_SEC`: a release can be missed — a hook the OS + #: drops for taking too long, a lock screen, an RDP session taking the keyboard — + #: and a hold whose end never arrives is a microphone left open indefinitely. + self._ptt_since: float | None = None + #: When the release happened and Flow started waiting for the decode to land, or + #: None when nothing is waiting. The paste cannot be synchronous: a final decode + #: measured 0.7-7 s in this user's own trace, so the release arms a wait and the + #: frame loop finishes the gesture. + self._ptt_wait: float | None = None self._flash = 0 # frames remaining of the error flash, out of `FLASH_FRAMES` #: Where the three waiting dots are in their 1.2 s loop, and how far the pill has #: travelled toward converse's violet (0 = dictate, 1 = converse). Both advance @@ -1046,10 +1441,12 @@ def __init__( #: `attributes("-alpha", …)` when the target has actually changed. self._drawn_alpha = 0.94 - self.work = _work_area(self.winfo_screenwidth(), self.winfo_screenheight()) - left, top, right, bottom = self.work - self.x = right - PILL_W - 28 - self.y = bottom - PILL_H - 24 + #: The monitor the stack is placed against, as the two rectangles FluidVoice + #: places against — `full` for centring, `work` for standing on. Refreshed from + #: the pointer's monitor in `_sync_monitor`; see `_pointer_monitor`. + self.full, self.work = _pointer_monitor( + self.winfo_screenwidth(), self.winfo_screenheight()) + self.x, self.y = self._placed(PILL_W) self.geometry(f"{PILL_W}x{PILL_H}+{self.x}+{self.y}") #: The width last drawn, so `_sync_dock` can tell whether a panel appeared or #: went away since the last frame — and, holding the right edge fixed, by how @@ -1090,7 +1487,13 @@ def __init__( # list and threw away the press handler that records where in the pill it was # grabbed. The pill dragged — it just snapped its top-left corner to the cursor # first, every time. - self.canvas.bind("", self._toggle, add="+") + # Press, motion and release rather than one `` handler, because the + # button now carries three gestures — see `_on_press`. `add="+"` for the reason + # the comment above gives, which has not changed: the drag handler is bound to + # the same event and binding without it would replace the whole list. + self.canvas.bind("", self._on_press, add="+") + self.canvas.bind("", self._on_release, add="+") + self.canvas.bind("", self._on_motion, add="+") self.canvas.bind("", self._menu) # Nothing animates under the hand (§07) — the same rule `Bubble`/ # `ConversationCard` already keep, extended to the pill's own motion. @@ -1134,7 +1537,82 @@ def drag(e): self.canvas.bind("", drag) self.canvas.bind("", press, add="+") + # -- one button, three gestures ---------------------------------------- + + def _on_press(self, e) -> None: + """Start the clock. Which gesture this is cannot be known yet. + + `_toggle` used to be bound straight to ``, which in Tk is the *press* — + so arming happened before the button came back up, and **every drag of the pill + also toggled listening**. That was there before hold-to-talk and is fixed by the + same change: a click is judged on release, like a button anywhere else. + """ + self._press_at = time.perf_counter() + self._press_xy = (e.x_root, e.y_root) + self._press_moved = False + self._press_talking = False + self._press_timer = self.after(int(PILL_HOLD_SEC * 1000), self._press_held) + + def _press_held(self) -> None: + """`PILL_HOLD_SEC` has passed with the button down and the pointer still. + + Fired by a timer rather than checked on release, and that is the whole + difference between this and a long-click: capture has to start *while the user + is still holding*, because the hold is the utterance. Waiting for the release to + notice would record nothing at all. + """ + self._press_timer = None + if self._press_at is None or self._press_moved: + return + self._press_talking = True + self._talk_start() + + def _on_release(self, _e=None) -> None: + """Decide what the press was, now that it is over. + + Three outcomes and one rule each: a hold ends the utterance and sends it, a + still click toggles, and a drag has already done its own job and must not do a + second one on the way out. + """ + if self._press_timer is not None: + self.after_cancel(self._press_timer) + self._press_timer = None + talking, moved = self._press_talking, self._press_moved + self._press_at = None + self._press_talking = False + if talking: + self._talk_end(send=True) + elif not moved: + self._toggle() + + def _on_motion(self, e) -> None: + """Past the slop, this press is a drag — unless it is already an utterance. + + The order matters. Once capture is open the pointer is irrelevant: somebody + talking into a held pill may well move the mouse, and cancelling their sentence + for it would be the gesture betraying them. Before that, motion is the thing + that tells a drag from a hold. + """ + if self._press_at is None or self._press_talking: + return + x, y = self._press_xy + if abs(e.x_root - x) > PILL_DRAG_SLOP or abs(e.y_root - y) > PILL_DRAG_SLOP: + self._press_moved = True + def _toggle(self, _e=None) -> None: + if self._ptt_since is not None: + # The toggle hotkey pressed with the chord still held. `pause()` below would + # bump the capture generation, which is how a deliberate stop refuses a + # decode from before it — and the utterance being spoken *right now* is + # exactly what that would refuse. Ending the hold first commits it. + # + # Not sent, because a toggle is not a release: the user reached for the + # other control mid-sentence, and pasting on their behalf is not what + # either gesture asked for. The words land in the draft, where Send takes + # them. The `return` is the rest of it — the hold has already stopped + # capture, so falling through to `pause()` would do the damage anyway. + self._talk_end(send=False) + return if self.armed: self.armed = False self._disarmed_since = time.perf_counter() # starts the 8 s idle dim @@ -1153,6 +1631,118 @@ def _toggle(self, _e=None) -> None: self._disarmed_since = None # an actively-capturing pill never dims self._draw() + # -- push to talk ------------------------------------------------------ + + def _talk_start(self) -> None: + """The chord's press-down: open the microphone for as long as it is held. + + Re-entrant on purpose. A hold that is already running must not restart capture — + the OS repeats a held key in some configurations, and a second `talk` arriving + mid-utterance would be indistinguishable from the user having spoken into a + microphone that had just been reopened under them. + """ + if self._ptt_since is not None: + return + if getattr(self.session, "editing", False): + # The hand editor is open, and `_pump_audio` throws away every block while it + # is. A hold here would open the microphone, capture nothing, and end with no + # paste and no explanation — the silent deafness invariant 4 forbids. Said + # instead, on the surface the editor is on. + self.front.note("editing — close the editor to dictate") + return + # A hold that begins while a previous release is still waiting for its decode + # supersedes it. Not dropped — `_ptt_wait` is only the *paste*, and abandoning it + # leaves the earlier words in the draft where the next Send will take them. + self._ptt_wait = None + try: + self.session.talk_start() + except Exception as exc: + # No microphone, a device held exclusively elsewhere, a session already + # closing. Same refusal `_toggle` makes, for the same reason: a green pill + # over a dead capture is the one lie this surface must not tell. + self._flash = FLASH_FRAMES + self.bubble.surface(f"could not start capture: {exc}") + self._draw() + return + self._ptt_since = time.perf_counter() + self.armed = True + self._disarmed_since = None + self._draw() + + def _talk_end(self, *, send: bool) -> None: + """The release: stop capturing, and hand what was said to `send` if it was clean. + + `send=False` is the `ctrl+win+d` path. The words are still committed and still + land in the draft — nothing spoken is ever dropped for the user's convenience + (P2) — they simply do not paste themselves into whatever window a desktop switch + just moved to. In the overwhelmingly common case there are no words at all: the + gate never opened in the 50 ms before the third key, so the draft stays empty and + the whole thing is invisible. + + Idempotent against a release with no hold behind it. The OS can deliver a keyup + whose keydown this process never saw — a chord begun before Flow launched, or + while a UAC prompt owned the input desktop — and that must not stop a capture the + toggle hotkey started. + """ + if self._ptt_since is None: + return + self._ptt_since = None + pending = self.session.talk_end() + # `talk_end` emits `disarm` when the hold was what opened the microphone, and the + # event handler clears `armed`. It deliberately does not when the hold began + # against an already-capturing session, and this must not either: the chord gives + # back exactly what it took. + if not pending: + # Nothing was said into the hold. A tap, or a shortcut. Say nothing, do + # nothing — a note here would fire on every `ctrl+win+arrow` on the machine. + self._draw() + return + self._ptt_wait = time.perf_counter() if send else None + self._draw() + + def _pump_talk(self) -> None: + """Finish the two halves of the gesture that cannot finish on a keystroke. + + Called every frame, and both branches are timeouts. The first is the hold whose + release never came; the second is the paste whose decode never landed. Neither is + hypothetical — the second is the exact state the app was in when it wedged with a + `state -> idle` and no `final` behind it, and a wait with no ceiling is how that + turned into a microphone open on a session nobody could reach. + """ + now = time.perf_counter() + + if self._ptt_since is not None and now - self._ptt_since >= PTT_MAX_HOLD_SEC: + # Treated as a release and not as a break: the user was dictating, and the + # thing that went missing is a keystroke, not their intent. What they said is + # committed and the draft holds it; it is not pasted, because a paste this + # far from the gesture would land somewhere they are no longer looking. + self._talk_end(send=False) + self.front.note( + f"stopped after {PTT_MAX_HOLD_SEC / 60:.0f} min — the chord was still " + "held. What you said is here; press Send when you want it") + return + + if self._ptt_wait is None: + return + + # The decode has to be *finished*, not merely started. `busy` covers the worker + # queue and the partial in flight, so this waits out a final that is still being + # transcribed rather than pasting the partial that preceded it. + if not self.session.busy: + text = self.session.draft.text + self._ptt_wait = None + if text: + self._send() + return + + if now - self._ptt_wait >= PTT_PASTE_WAIT_SEC: + self._ptt_wait = None + # Never a silent give-up, and never a discard: the words are on screen and + # the chip that sends them is the one the user already knows. + self.front.note( + f"still decoding after {PTT_PASTE_WAIT_SEC:.0f}s — not pasting on my " + "own this late. Press Send when it lands") + def _enter(self, _e=None) -> None: self._pointer_in = True self._hover_since = time.perf_counter() @@ -1300,6 +1890,104 @@ def _draft_menu(self, parent: tk.Menu) -> None: sub.add_command(label="Clear", command=self._clear) parent.add_cascade(label="Draft", menu=sub) + def _panel_menu(self, parent: tk.Menu) -> None: + """How wide the draft panel draws, chosen from the widths that have been drawn. + + **Applied immediately rather than at next launch.** Every part of both windows + reads `BUBBLE_W` and `CARD_W` while drawing a frame rather than caching them at + construction, so rebinding them and forcing a redraw is a complete change — and + a size you cannot see until you restart is one nobody can choose between. + + Saved on the way through, because this is the kind of setting somebody sets once + and would be annoyed to set again. A save that fails is said out loud rather than + swallowed: the width is still applied, so the session honours the choice and the + note explains why the next one will not. + """ + sub = _dark_menu(parent) + here = BUBBLE_W + + def choose(name: str) -> None: + apply_panel_width(panel_width(name)) + # Both windows, because they are one window at two moments and only one of + # them is on screen to notice the change. `_frame` re-reads the width and + # re-lays every item it draws, so re-anchoring is the whole of the update. + for window in (self.bubble, self.card): + window.reposition() + profile = getattr(self.session, "profile", None) + if profile is None: + # `--no-profile`. The size is applied and lasts the session, which is + # exactly what that flag asks for, so there is nothing to report. + return + profile.panel = name + # The same shape `_set_trigger` uses, and for the same reason: this is a + # setting somebody chooses once, so a save that failed has to be visible now + # rather than discovered at the next launch. + if profile.save(): + self.bubble.note(f"panel size: {name}") + else: + self._flash = FLASH_FRAMES + self.bubble.note(f"could not save {profile.path}") + + for name, width in PANEL_WIDTHS.items(): + sub.add_command( + label=name.capitalize() + (" (current)" if width == here else ""), + command=lambda n=name: choose(n), + ) + parent.add_cascade(label="Panel size", menu=sub) + + def _gesture_menu(self, parent: tk.Menu) -> None: + """What the chord does, switchable while Flow is running. + + This is here because shipping push-to-talk *instead of* the toggle took a + working gesture away from everybody who had it, with no way back short of + editing a file — and the two are not a preference between equals, they are good + at different things. A hold needs no decision about when you are finished and + cannot leave a microphone running; a toggle is the only one of the two that + survives a paragraph, a long thought with pauses in it, or hands that cannot + hold two keys down for a minute. + + **Applied to the live hook rather than at next launch**, and that is the whole + reason `Chord.gesture` is a plain attribute the callback reads. Switching by + rebuilding the chord would mean unhooking and re-installing a `WH_KEYBOARD_LL` + hook, which is the one call in that file the OS is entitled to refuse — and + being refused *while changing a setting* would leave somebody with no chord at + all and no obvious way back. One string assignment cannot fail. + + Absent when there is no chord to describe: `--no-chord`, `"chord": ""`, or a + hook the OS refused. Same rule the help sheet follows — the menu says what works + on this machine this launch. + """ + chord = getattr(self.hotkeys, "chord", None) if self.hotkeys else None + if chord is None: + return + sub = _dark_menu(parent) + + def choose(name: str) -> None: + chord.gesture = name + # A gesture change mid-hold would leave the release with nothing to end: + # `_talking` is latched inside the hook and the pill is holding a `_ptt_since` + # for a capture the new gesture has no word for. Ending it here is the same + # tidy-up `_toggle` does, and for the same reason — the words are kept. + if self._ptt_since is not None: + self._talk_end(send=False) + profile = getattr(self.session, "profile", None) + if profile is None: + return # `--no-profile`: applied for this session, which is what it asks + profile.gesture = name + if profile.save(): + self.front.note(f"chord: {GESTURE_LABELS[name]}") + else: + self._flash = FLASH_FRAMES + self.front.note(f"could not save {profile.path}") + + for name in GESTURE_LABELS: + sub.add_command( + label=(GESTURE_LABELS[name] + + (" (current)" if name == chord.gesture else "")), + command=lambda n=name: choose(n), + ) + parent.add_cascade(label=f"Chord ({chord.describe()})", menu=sub) + def _settings_menu(self, parent: tk.Menu) -> None: """Everything somebody sets once, in one place they can find it twice. @@ -1309,7 +1997,9 @@ def _settings_menu(self, parent: tk.Menu) -> None: invites options to be added to it. """ sub = _dark_menu(parent) + self._gesture_menu(sub) self._trigger_menu(sub) + self._panel_menu(sub) # Also the CLI marker's refresh point: a CLI installed mid-session shows up here, # where a press is already paying for the PATH walk `_resolved` will not repeat. clis = self._clis = available() @@ -1776,7 +2466,22 @@ def _send(self, submit: bool = False) -> None: boom" yields "boom", so a refusing enter-variant would make the degraded decode the working case and the fuller utterance the broken one — the exact inversion the word order exists to prevent. + + **Every send goes through here, which is what makes one line enough to stop + push-to-talk sending twice.** There are four ways to send — this chip, the + `send` hotkey, the spoken trigger routed as a `send` event, and converse's + auto-ask countdown — and a release that has armed a paste is a fifth thing + waiting to do the same job. The collision is not hypothetical: hold the chord, + say "…and that's the plan, boom", let go, and the trigger fires a send when the + decode routes while the release is still waiting to fire its own. + + A hold owns *one* send, and whoever gets there first has it. Cancelling the wait + here covers all four collisions at the one point they have in common, rather + than four guards that would have to be kept in step — and it is the right way + round, because a send that has already happened is the one thing that proves the + wait has nothing left to do. """ + self._ptt_wait = None text = self.session.send() problem = "" if text and self.lite: @@ -1993,6 +2698,12 @@ def _clear(self) -> None: # Clear is the cheapest "stop" the user has, and with the microphone gated while # Flow talks it is one of the few ways left to cut a reply short. Doing that # first means one press does the obvious thing whichever is in progress. + # + # A pending push-to-talk paste is exactly such a thing in progress, and the + # nastiest one to leave running: the draft is cleared here, the decode lands a + # second later and refills it, and the wait pastes into the user's window the + # words they just pressed a key to stop. Clear means clear. + self._ptt_wait = None self.session.stop_speaking() self.session.draft.clear() self.bubble.hide() @@ -2063,6 +2774,57 @@ def _tick(self) -> None: if self._alive: self.after(30, self._tick) + def _placed(self, w: int) -> tuple[int, int]: + """Where a stack `w` wide belongs on the current monitor, per `PLACE`. + + Two answers, and the setting picks between them rather than one being a + degraded version of the other: + + `"bottom"` is FluidVoice's (`positionWindow`) — centred on the physical display, + stood on the work area. It is the default because the corner has a problem the + centre does not: the bottom-right of the screen is where Windows puts the tray, + every toast notification, and most apps' own status chrome, so the one place + Flow reserved for itself is the busiest real estate on the desktop. The centre + is empty, it is where the eye already is, and it is the same place on every + machine regardless of what is docked to which edge. + + `"corner"` is what Flow shipped, kept because somebody who has spent months + with the pill in the bottom right should not have it moved by an upgrade. + """ + if PLACE == "corner": + _left, _top, right, bottom = self.work + return right - w - 28, bottom - PILL_H - 24 + return bottom_centre(w, PILL_H, self.full, self.work, PANEL_BOTTOM_OFFSET) + + def _sync_monitor(self) -> None: + """Follow the pointer's monitor, and re-place the stack when it changes. + + Flow read the work area **once, in `__init__`, from `SystemParametersInfoW`** — + which only ever answers for the primary display. So on a two-monitor desk every + window Flow drew was placed against a screen the user might not be looking at, + and the clamps that keep panels on-screen were clamping to the wrong rectangle. + That is the placement problem, and it was never a rounding error: it is the + whole width of a monitor. + + Checked every frame and acted on only when the rectangle actually moves, which + is the same shape `_track_target` uses for `classify` and for the same reason — + the question is cheap, the answer changes a few times an hour, and doing the + work unconditionally would be a `geometry` call per frame forever. + """ + full, work = _pointer_monitor( + self.winfo_screenwidth(), self.winfo_screenheight()) + if (full, work) == (self.full, self.work): + return + self.full, self.work = full, work + self.x, self.y = self._placed(self.pill_w) + # The panels are placed *from* the pill, so moving it is the whole move — but + # only for a panel that is up. `reposition` on a withdrawn window would place it + # and leave it withdrawn, which is work nobody can see. + self._sync_dock() + for panel in (self.bubble, self.card): + if getattr(panel, "_visible", False): + panel.reposition() + def _track_target(self) -> None: """Remember the last window that had the foreground and was not Flow's own. @@ -2078,6 +2840,12 @@ def _track_target(self) -> None: return hwnd = foreground_hwnd() if hwnd and not owned_by_flow(hwnd): + if hwnd != self.paste_target: + # Only when the window actually changed. `classify` opens a process + # handle, and this runs every frame — at 30 fps an `OpenProcess` per + # frame is a cost paid forever to answer a question whose answer moves a + # few times an hour. Resolved on the edge, remembered in between. + self.session.target_app = classify(hwnd).process self.paste_target = hwnd @property @@ -2113,17 +2881,37 @@ def _swap_surfaces(self) -> None: def _frame(self) -> None: self._track_target() + self._sync_monitor() # Hotkeys arrive on their own thread; Tk is only ever touched from this one. if self.hotkeys is not None: for name in self.hotkeys.drain(): if name == "toggle": self._toggle() + elif name == "warm": + # The chord's press-down, one put ahead of `talk`, so the models load + # during the hold instead of inside the first sentence. + self.session.warm() + elif name == "talk": + self._talk_start() + elif name == "talk-end": + self._talk_end(send=True) + elif name == "talk-break": + # Windows meant `ctrl+win+d`. Stop, keep whatever was said, paste + # nothing — see `_talk_end`. + self._talk_end(send=False) elif name == "send": self._send() elif name == "cancel": self._clear() elif name == "mode": + # A pending paste belongs to the mode it was spoken in. Dictate + # pastes into a window and converse asks a CLI, so a wait armed in + # one and fired in the other does something the user never asked + # for — and the switch is one keypress away at all times. Dropped + # rather than translated: the words stay in the draft, and Send in + # the new mode does whatever it now means, deliberately. + self._ptt_wait = None self.session.toggle_mode() elif name == "quit": self.quit_app() @@ -2133,7 +2921,7 @@ def _frame(self) -> None: self.session.tick() if getattr(self.session, "hearing", True): self._deaf_frame = 0 - self.levels.append(self._eased(self._norm(self.session.level_db))) + self._meter_level = self._eased(self._norm(self.session.level_db)) else: self._flatten() else: @@ -2142,10 +2930,15 @@ def _frame(self) -> None: # arrived, because the code that collects a reply sat behind this check. self.session.pump_results() self._deaf_frame = 0 - self.levels.append(self._eased(0.0)) + self._meter_level = self._eased(0.0) self._pump_warnings() self._pump_events() + # After `_pump_events`, so a draft the final decode just produced is on + # `session.draft` by the time the wait looks for it — otherwise every paste + # would cost one extra frame, and a decode that landed in the same frame as the + # timeout would be reported as never having arrived. + self._pump_talk() if self.converse: self.card.tick_countdown() @@ -2296,9 +3089,13 @@ def _flatten(self) -> None: return self._deaf_frame = min(self._deaf_frame + 1, DEAF_COLLAPSE_FRAMES) self._eased_level = 0.0 - done = round(BARS * self._deaf_frame / DEAF_COLLAPSE_FRAMES) - for i in range(BARS - 1, BARS - 1 - done, -1): - self.levels[i] = 0.0 + # Collapsed as one level rather than bar by bar from the right. Emptying the + # right-hand bars first was the correct picture of a *scrolling* meter going + # quiet — the silence arrived at one end and travelled. A bloom has no ends to + # arrive at, so going deaf is the whole shape settling at once, which is also + # what FluidVoice does while it is processing rather than listening. + fade = 1.0 - self._deaf_frame / DEAF_COLLAPSE_FRAMES + self._meter_level = max(0.0, self._meter_level * fade) def _advance_motion(self) -> None: """Step the two §07 animations that have to remember where they were. @@ -2339,6 +3136,34 @@ def _eased(self, target: float) -> float: def _norm(db: float) -> float: return max(0.0, min(1.0, (db - DB_FLOOR) / (DB_CEIL - DB_FLOOR))) + @staticmethod + def _bar_half_height(index: int, level: float) -> float: + """Half the height of bar `index` at `level`, as FluidVoice shapes it. + + Three terms, and each is doing something the other two cannot: + + 1. **The envelope** puts the peak in the middle. A bar's ceiling falls off with + its distance from the centre, floored at 18% so the outermost bars still move + rather than sitting dead at the ends. + 2. **The exponent** bends the response so ordinary speech reaches most of the + way up. Linear is what made Flow's old meter look timid at conversational + volume — the top half of the widget was reserved for shouting. + 3. **The variation** breaks the symmetry very slightly, so a sustained note draws + a shape rather than a comb. + + Returns a half-height because the bars are mirrored about the pill's centre + line: `PILL_H` is 40 and the meter has 8 px of air, so 12 px each way. + """ + centre_distance = abs(index - (BARS - 1) / 2) + normalised = min(centre_distance / max((BARS - 1) / 2, 1), 1.0) + factor = max(_ENVELOPE_MIN, _ENVELOPE_FLOOR - normalised * _ENVELOPE_SPAN) + peak = BAR_MIN_H + (BAR_MAX_H - BAR_MIN_H) * factor + amplified = max(0.0, min(1.0, level)) ** _LEVEL_EXPONENT + variation = (_BAR_VARIATION_BASE + + _BAR_VARIATION_SWING * math.cos(index * _BAR_VARIATION_RATE)) + height = BAR_MIN_H + (peak - BAR_MIN_H) * amplified * variation + return max(BAR_MIN_H, min(BAR_MAX_H, height)) + # -- painting ---------------------------------------------------------- @property @@ -2450,8 +3275,20 @@ def _sync_dock(self) -> None: """ w = self.pill_w if w != self._docked_w: - left, _top, _right, _bottom = self.work - self.x = max(left, self.x + self._docked_w - w) + left, _top, right, _bottom = self.work + if PLACE == "corner": + # Hold the right edge and let the left one move out to meet the panel. + # That is the whole dock in corner placement: the pill is anchored to + # the screen edge it sits against, and growing away from it is the only + # direction there is. + self.x = max(left, self.x + self._docked_w - w) + else: + # Centred placement has no anchored edge, so holding one is what makes + # the stack lurch: the pill is 205 px and the panel 420, and keeping the + # right edge put the pair 107 px left of centre the moment a draft + # appeared — visibly, on every utterance. Re-centre on the new width. + self.x = self._placed(w)[0] + self.x = max(left, min(self.x, right - w)) self._docked_w = w self.canvas.configure(width=w) if self.window_geometry() != (w, self.x, self.y): @@ -2557,12 +3394,27 @@ def _draw(self) -> None: self._draw_dots(c, mid, accent) else: # Level bars (R13). Mirrored around the centre line so quiet reads as a - # flat line rather than an empty box. - for i, lvl in enumerate(self.levels): - h = max(1.5, lvl * (PILL_H - 16) / 2) + # flat line rather than an empty box, and blooming from the middle rather + # than scrolling — see `_bar_half_height` for what changed and why. + # + # `_meter_level` and not `self.levels[i]`: every bar reads the same level + # now, and the shape between them is the envelope rather than the past. + lvl = self._meter_level + shade = accent if lvl > 0.04 else MUTED + for i in range(BARS): + h = self._bar_half_height(i, lvl) x = METER_X + i * (BAR_W + BAR_GAP) - shade = accent if lvl > 0.04 else MUTED - c.create_rectangle(x, mid - h, x + BAR_W, mid + h, fill=shade, outline="") + # Rounded caps, radius half the bar width — FluidVoice draws its bars as + # `RoundedRectangle(cornerRadius: barWidth / 2)`, and at four pixels wide + # that is the difference between a meter and a bar chart. Squared off + # when the bar is shorter than its own cap, where a smoothed polygon + # would pinch into a lozenge. + if h * 2 > BAR_W: + _round_rect(c, x, mid - h, x + BAR_W, mid + h, BAR_W / 2, + fill=shade, outline="") + else: + c.create_rectangle(x, mid - h, x + BAR_W, mid + h, + fill=shade, outline="") self._draw_label(c, w, mid, accent) def _draw_dots(self, c: tk.Canvas, mid: int, accent: str) -> None: @@ -3021,7 +3873,7 @@ def show(self) -> None: def close(self) -> None: self._visible = False - self.withdraw() + park(self) @property def showing(self) -> bool: @@ -3592,7 +4444,7 @@ def hide(self) -> None: self._text = self._partial = self._note = self._sent = "" self._note_undo = False self._for_activity = False - self.withdraw() + park(self) # -- geometry ---------------------------------------------------------- diff --git a/tests/test_apps.py b/tests/test_apps.py new file mode 100644 index 0000000..d9e45a1 --- /dev/null +++ b/tests/test_apps.py @@ -0,0 +1,271 @@ +"""Per-app notes: a standing instruction that depends on which app is in front. + +The feature is one sentence — *"this text is going into slack.exe, bear it in mind"* — +appended to the rewrite prompt when the profile has something to say about the app that +holds the foreground. Almost none of it is new machinery. `Pill._track_target` already +polls the foreground every frame so Send can be aimed, `inject.classify` already names +the process behind a window because the terminal detection needed it, and `refine()` +already builds its call out of named prompt constants. What this adds is the table, the +lookup, and the two places that could get it wrong. + +Which is where the tests are aimed, because those two are the whole risk: + + **It must not out-shout what the user just said.** A per-app note is a *standing* + preference, and the entire point of speaking an instruction is to override a standing + preference on this one occasion. So the note is phrased as a destination rather than a + rule, and it is placed before the request rather than after it — the sentence nearest + the text is the one that wins ties, and that sentence has to be the user's. + + **It must cost nothing when there is nothing to say.** No profile, no table, an app + with no entry, a blank entry, an entry that is not a string, Lite, a window the OS + will not name: every one of those has to come out as a rewrite that behaves exactly + the way every rewrite behaved before this existed. + +The frame budget gets its own test in `tests/test_lite.py` (`classify` opens a process +handle, and this is polled at 30 fps), because that is where the polling lives. +""" + +import sys +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from flow.profile import Profile, _apps # noqa: E402 +from flow.refine import _POLISH_PROMPT, _PROMPT, Cli, app_note # noqa: E402 +from flow.session import Session # noqa: E402 + + +class Silent: + """The `asr`/`mic` surface `Session` needs and none of these tests exercise.""" + + def __getattr__(self, _name): + return lambda *_a, **_kw: None + + +class TestTheNoteIsBuiltOnlyWhenThereIsSomethingToSay(unittest.TestCase): + """`app_note`, on every shape a hand-written table can produce.""" + + def test_an_app_and_an_instruction_make_a_block(self): + block = app_note("slack.exe", "keep it informal") + self.assertIn("slack.exe", block) + self.assertIn("keep it informal", block) + + def test_it_names_the_destination_rather_than_giving_an_order(self): + # The phrasing is the safeguard, not decoration. "This text is going into Slack" + # is a fact the model weighs against the request; "always be informal" is a + # competing order, and a competing order beats the instruction the user just + # spoke — which would make the feature worse than not having it. + block = app_note("slack.exe", "keep it informal").lower() + self.assertIn("going into", block) + self.assertIn("without letting it override", block) + + def test_every_way_of_having_nothing_to_say_is_the_empty_string(self): + # Each of these is a rewrite that behaves exactly as it did before per-app notes + # existed, which is the only acceptable degraded path for a feature nobody asked + # to have switched on. + for app, note in ( + ("slack.exe", ""), + ("slack.exe", " "), + ("slack.exe", None), + ("slack.exe", 7), + ("slack.exe", ["informal"]), + ("", "keep it informal"), + ): + with self.subTest(app=app, note=note): + self.assertEqual(app_note(app, note), "") + + def test_the_instruction_is_trimmed_but_not_otherwise_touched(self): + # A `lexicon.txt`-shaped bargain: Flow does not reformat what somebody wrote in + # their own file. Whitespace goes because trailing space in JSON is invisible and + # never deliberate; nothing else does. + self.assertIn("Use British spelling.", + app_note("word.exe", " Use British spelling. ")) + + +class TestTheNoteGoesInFrontOfTheRequest(unittest.TestCase): + """Position is the second safeguard, and it is the one a refactor would lose.""" + + def _prompt(self, polish: bool) -> str: + from flow import refine as refine_mod + + seen = {} + + def capture(_cli, prompt, **_kw): + seen["prompt"] = prompt + # A real `Cli`, because `_clean` reads `.name` off whatever comes back to + # decide which CLI's furniture to strip. + return "REVISED", "", Cli("codex", ("codex", "exec")) + + with mock.patch.object(refine_mod, "_invoke_any", capture): + refine_mod.refine("shipping on friday", "make it formal", polish=polish, + app=app_note("slack.exe", "keep it informal")) + return seen["prompt"] + + def test_a_semantic_rewrite_carries_the_note(self): + self.assertIn("slack.exe", self._prompt(polish=False)) + + def test_a_polish_carries_it_too(self): + # The polish ignores the spoken instruction entirely, which makes it the pass + # with the *most* to gain from knowing where the words are headed: a prompt bound + # for a terminal and one bound for a chat window differ in exactly this way. + self.assertIn("slack.exe", self._prompt(polish=True)) + + def test_the_users_own_instruction_sits_nearer_the_text_than_the_note(self): + # The sentence nearest the text wins ties. A per-app note is a standing + # preference and speaking is how you override one on this occasion, so the + # spoken instruction has to be the closer of the two. + prompt = self._prompt(polish=False) + self.assertLess(prompt.index("slack.exe"), prompt.index("make it formal")) + + def test_neither_prompt_mentions_an_app_when_there_is_no_note(self): + # The default path for everybody who never writes an `apps` table, which is + # almost everybody: the prompt has to be byte-for-byte what it always was. + self.assertNotIn("going into", _PROMPT) + self.assertNotIn("going into", _POLISH_PROMPT) + + +class TestTheTableIsCarriedThroughUntouched(unittest.TestCase): + """`profile._apps`, which checks the shape and deliberately stops there.""" + + def test_a_table_survives(self): + self.assertEqual(_apps({"code.exe": "be terse"}), {"code.exe": "be terse"}) + + def test_anything_that_is_not_a_table_degrades_to_nothing(self): + for value in (None, "code.exe", 3, ["code.exe"], True): + with self.subTest(value=value): + self.assertEqual(_apps(value), {}) + + def test_an_entry_is_never_refused_by_name(self): + # The difference from the `hotkeys` table, and it is worth being explicit about. + # An action name has five right answers, so a typo is knowable. An executable + # name has as many right answers as there are programs in the world — an entry + # for an app that is not installed is not a mistake, it is somebody who has not + # opened it yet. So nothing is dropped and nothing is reported; a key that never + # matches simply never fires. + table = {"nothing-like-this.exe": "be terse", "": "x", "7": "y"} + self.assertEqual(_apps(table), table) + + def test_it_survives_a_save_and_a_load(self): + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + p = Profile(path) + p.apps = {"code.exe": "be terse"} + self.assertTrue(p.save()) + self.assertEqual(json.loads(path.read_text(encoding="utf-8"))["apps"], + {"code.exe": "be terse"}) + again = Profile(path) + self.assertTrue(again.load()) + self.assertEqual(again.apps, {"code.exe": "be terse"}) + self.assertNotIn("apps", again.faults) + + def test_an_empty_table_lands_in_every_saved_profile(self): + # The only advertisement this feature gets, in a project with no settings dialog + # to put it in — the same job `"hotkeys": {}` already does. + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + self.assertTrue(Profile(path).save()) + self.assertIn("apps", json.loads(path.read_text(encoding="utf-8"))) + + +class TestTheSessionLooksTheAppUp(unittest.TestCase): + """`Session._app_note`: the join between the window in front and the table.""" + + def session(self, apps=None, app="") -> Session: + s = Session(asr=Silent(), mic=Silent()) + self.addCleanup(s.close) + if apps is not None: + s.profile = type("P", (), {"apps": apps})() + s.target_app = app + return s + + def test_the_app_in_front_selects_its_entry(self): + s = self.session({"code.exe": "be terse", "slack.exe": "be informal"}, + app="slack.exe") + self.assertIn("be informal", s._app_note()) + self.assertNotIn("be terse", s._app_note()) + + def test_the_match_ignores_case(self): + # "Code.exe" and "code.exe" are the same program. A table that cared would be one + # whose entries silently stop matching the day a vendor changes the + # capitalisation of a shipped binary. + for written, running in (("Code.exe", "code.exe"), ("code.exe", "CODE.EXE"), + (" code.exe ", "Code.Exe")): + with self.subTest(written=written, running=running): + s = self.session({written: "be terse"}, app=running) + self.assertIn("be terse", s._app_note()) + + def test_an_app_with_no_entry_gets_nothing(self): + s = self.session({"code.exe": "be terse"}, app="notepad.exe") + self.assertEqual(s._app_note(), "") + + def test_every_way_of_having_no_table_gets_nothing(self): + for apps in ({}, None): + with self.subTest(apps=apps): + self.assertEqual(self.session(apps, app="code.exe")._app_note(), "") + + def test_no_profile_at_all_gets_nothing(self): + # `--no-profile`, and the launch before one has ever been saved. + s = self.session(app="code.exe") + s.profile = None + self.assertEqual(s._app_note(), "") + + def test_no_app_in_front_gets_nothing(self): + # Lite, which has no target-window awareness at all, and the moments when the OS + # declines to name the foreground. + s = self.session({"code.exe": "be terse"}, app="") + self.assertEqual(s._app_note(), "") + + def test_a_key_that_is_not_a_string_cannot_break_the_lookup(self): + # `_apps` carries entries through untouched by design, so this table is + # reachable — and a rewrite that raised because of one bad key would cost the + # user the words they had already spoken. + s = self.session({7: "be terse", "code.exe": "be brief"}, app="code.exe") + self.assertIn("be brief", s._app_note()) + + def test_a_blank_instruction_is_the_same_as_no_entry(self): + # How somebody switches one app off without deleting the line they wrote. + s = self.session({"code.exe": " "}, app="code.exe") + self.assertEqual(s._app_note(), "") + + +class TestItSaysWhenItUsedOne(unittest.TestCase): + """P2: a rewrite that quietly obeyed an invisible rule is one nobody can debug.""" + + def session(self, apps, app) -> Session: + s = Session(asr=Silent(), mic=Silent()) + self.addCleanup(s.close) + s.profile = type("P", (), {"apps": apps})() + s.target_app = app + return s + + def notes(self, s) -> str: + return " | ".join(e.text for e in s.events() if e.kind == "note") + + def test_the_note_is_named_when_it_applies(self): + s = self.session({"slack.exe": "keep it informal"}, "slack.exe") + s.draft.set("shipping on friday") + with mock.patch("flow.session.refine", return_value=("REVISED", "codex")): + s._start_refine("make it formal") + self.assertIn("slack.exe", self.notes(s)) + + def test_and_nothing_is_said_when_none_applies(self): + # The common case by far. A line about a note that did not exist would be noise + # in the one channel that has to stay believable. + s = self.session({"slack.exe": "keep it informal"}, "notepad.exe") + s.draft.set("shipping on friday") + with mock.patch("flow.session.refine", return_value=("REVISED", "codex")): + s._start_refine("make it formal") + self.assertNotIn("note", self.notes(s).replace("notes", "")) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_chord.py b/tests/test_chord.py new file mode 100644 index 0000000..4eb6eb4 --- /dev/null +++ b/tests/test_chord.py @@ -0,0 +1,680 @@ +"""The modifier-only chord: hold ctrl+win to dictate, let go to send. + +Push-to-talk. The press-down warms the models and opens the microphone, the hold is the +utterance, and the release stops capture and sends what was said. It used to be a +toggle that fired one word at the release, and the three moments here are the change. + + +`RegisterHotKey` cannot express this, which is the whole reason the code under test +exists — it takes a virtual key and there is no VK for "nothing". So the chord runs on a +`WH_KEYBOARD_LL` hook, and that is a decision (R16 said no global hooks) narrowed rather +than reversed. Three properties are what the narrowing rests on, and each is asserted +here rather than left to the comment that claims it: + + **It never learns which key you pressed.** A key outside the chord sets a boolean. The + suite proves the shape of that by driving keys through and checking that what changes + is *whether* the chord fires, never a record of what was typed — see + `TestDItLearnsNothingAboutTheKeysItRejects`. + + **It never swallows anything.** Every event reaches `CallNextHookEx`, including the one + that fires the chord. Ctrl and Win have real jobs and Flow does not get to keep them. + + **It sends nothing when Windows meant something else.** ctrl+win is a *prefix* in + Windows itself — ctrl+win+d makes a virtual desktop, ctrl+win+left and +right switch + between them. Every one presses a third key, and under push-to-talk that third key + *stops* a capture rather than merely declining to start one, because the press-down + already opened the microphone. `TestCWindowsOwnsCtrlWinToo` states what this costs and + what survives it; the promise that survives is that no desktop switch ever pastes. + +The hook is never installed. `Chord._on_key` is the callback the OS would call, so the +suite calls it directly with the structure Windows would pass — which tests the state +machine that is actually hard, and asks nothing of the developer's keyboard. +""" + +import ctypes +import sys +import unittest +from pathlib import Path +from unittest import mock + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +# Windows-only, and the guard is `tests/test_hotkey.py`'s word for word: `flow.hotkey` +# calls `ctypes.WinDLL("user32", use_last_error=True)` at module scope, so the failure +# would be the `import` below rather than anything a decorator could reach. +if sys.platform != "win32": # pragma: no cover - the CI legs that are not Windows + raise unittest.SkipTest("Windows-only: flow.hotkey binds user32 at import") + +import queue # noqa: E402 + +import flow.hotkey as hotkey # noqa: E402 +from flow.hotkey import ( # noqa: E402 + CHORD_NAMES, + VK_LCONTROL, + VK_LMENU, + VK_LSHIFT, + VK_LWIN, + VK_RCONTROL, + VK_RWIN, + WM_KEYDOWN, + WM_KEYUP, + WM_SYSKEYDOWN, + WM_SYSKEYUP, + Chord, + describe_chord, + parse_chord, +) +from flow.profile import CHORD_DEFAULT # noqa: E402 + +VK_D, VK_LEFT, VK_A = 0x44, 0x25, 0x41 + + +class _Keyboard: + """Drives `Chord._on_key` the way the OS would, and remembers what was passed on. + + `CallNextHookEx` is stubbed rather than called for real, for one reason and one + convenience. The reason: what this suite is checking is that *every* event reaches + it, and a real call returns a number that says nothing about whether it happened. The + convenience: the callback runs with `self._hook` still None, because nothing here + installs a hook, and a NULL hook handle is exactly the argument a stub should not + have to care about. + """ + + def __init__(self, chord): + self.chord = chord + self.passed = [] + + def _event(self, message, vk): + # The real `KBDLLHOOKSTRUCT`, by address, because `_on_key` casts the LPARAM and + # a fake that skipped the cast would not exercise the line most likely to be + # wrong on a 64-bit build. + block = hotkey._KBDLLHOOKSTRUCT(vkCode=vk, scanCode=0, flags=0, time=0, + dwExtraInfo=None) + # Held for the duration of the call: a struct that went out of scope here would + # be freed under the pointer the callback is about to read. + self._block = block + with mock.patch.object(hotkey, "user32") as fake: + fake.CallNextHookEx.return_value = 0 + self.chord._on_key(0, message, ctypes.addressof(block)) + self.passed.append((message, vk, fake.CallNextHookEx.called)) + + def down(self, *vks): + for vk in vks: + self._event(WM_KEYDOWN, vk) + return self + + def up(self, *vks): + for vk in vks: + self._event(WM_KEYUP, vk) + return self + + def sys_down(self, *vks): + for vk in vks: + self._event(WM_SYSKEYDOWN, vk) + return self + + def sys_up(self, *vks): + for vk in vks: + self._event(WM_SYSKEYUP, vk) + return self + + +def _chord(mods=("ctrl", "win")): + presses = queue.Queue() + return Chord(presses, frozenset(mods)), presses + + +def _drained(presses): + out = [] + while not presses.empty(): + out.append(presses.get_nowait()) + return out + + +#: One clean hold, start to finish, as the queue sees it. +HOLD = ["warm", "talk", "talk-end"] + + +def _fired(presses): + """Everything one gesture put on the queue, in order. + + Named `_fired` still, but it no longer means "the one word a release emits" — under + push-to-talk a hold is three moments, and the order between them *is* the feature. + Filtering any of them out would hide the two things most worth asserting: that the + warm arrives before the capture rather than with it, and that a hold Windows took + over is stopped on the third key rather than at the release. + """ + return _drained(presses) + + +class TestAWhatAChordMeansIsReadOffWhatSomebodyTyped(unittest.TestCase): + """`parse_chord`, on the string the guide prints and the shapes a hand-edit makes.""" + + def test_the_shipped_chord_parses_to_the_two_modifiers_it_names(self): + self.assertEqual(parse_chord("ctrl+win"), (frozenset({"ctrl", "win"}), "")) + + def test_the_shipped_default_is_a_chord_this_parser_accepts(self): + # The one assertion that would catch the two files disagreeing. `CHORD_DEFAULT` + # lives in `flow/profile.py` because that module imports on a Mac and + # `flow.hotkey` cannot — a split that buys platform reach and costs exactly this + # risk, so the risk is bought back here. + mods, reason = parse_chord(CHORD_DEFAULT) + self.assertIsNotNone(mods, reason) + + def test_case_and_spacing_are_the_writers_business(self): + # Same rule as `parse`, and for the same reason: this is JSON somebody typed by + # hand, and a setting that depends on where the spaces went is a bug report + # waiting to be filed. + canonical, _reason = parse_chord("ctrl+win") + for text in ("CTRL+WIN", "Ctrl+Win", " ctrl + win ", "cTrL+wIn", + "ctrl+win\n", "win+ctrl"): + with self.subTest(text=text): + self.assertEqual(parse_chord(text)[0], canonical) + + def test_one_modifier_is_refused_because_a_bare_tap_is_a_thing_hands_do(self): + # The refusal worth explaining. A single-modifier "chord" fires every time that + # key is tapped and released cleanly, and a bare Ctrl tap is a thing hands do + # constantly while thinking — so it would be a dictation app that starts + # recording at random, from a setting nobody would connect to the symptom. + for text in ("ctrl", "win", "shift", "ctrl+ctrl", "ctrl + CTRL"): + with self.subTest(text=text): + mods, reason = parse_chord(text) + self.assertIsNone(mods) + self.assertIn("two modifiers", reason) + + def test_four_modifiers_are_refused_because_that_is_not_a_shape(self): + mods, reason = parse_chord("ctrl+alt+shift+win") + self.assertIsNone(mods) + self.assertIn("cannot be held", reason) + + def test_a_key_is_not_a_modifier_and_is_named_as_the_thing_that_was_wrong(self): + # The likeliest hand-edit by far: somebody reads "chord" and writes the combo + # they already know. The reason has to point at the word that broke it, because + # the fix is deleting that word and nothing else. + mods, reason = parse_chord("ctrl+win+space") + self.assertIsNone(mods) + self.assertIn("space", reason) + self.assertIn("not a modifier", reason) + + def test_the_shapes_that_are_not_strings_at_all(self): + for value in (None, 3, ["ctrl", "win"], {"ctrl": "win"}, True): + with self.subTest(value=value): + self.assertEqual(parse_chord(value), (None, "not a string")) + + def test_an_empty_string_is_refused_here_and_meant_off_at_the_call_site(self): + # `_chord` in `flow/__main__.py` never reaches the parser with a blank, because + # blank is how somebody turns the chord off in the file. Refused rather than + # accepted anyway, so that the two places cannot drift into disagreeing about + # what an empty setting means. + self.assertEqual(parse_chord("")[1], "empty") + self.assertEqual(parse_chord(" + ")[1], "empty") + + def test_every_chord_that_parses_can_be_written_back_out(self): + # The round trip that keeps the startup line honest: the block says `chord + # toggle ctrl+win`, and a chord that could be asked for but not named would put + # something else there. + for a in CHORD_NAMES: + for b in CHORD_NAMES: + if a == b: + continue + with self.subTest(chord=f"{a}+{b}"): + mods, _ = parse_chord(f"{a}+{b}") + self.assertEqual(parse_chord(describe_chord(mods))[0], mods) + + def test_the_report_spells_a_chord_the_same_way_round_every_time(self): + # "win+ctrl" and "ctrl+win" are the same chord, and the startup block has to + # call them the same thing or the line stops being something to compare against + # the guide. + self.assertEqual(describe_chord(parse_chord("win+ctrl")[0]), "ctrl+win") + + +class TestBTheHoldIsTheUtterance(unittest.TestCase): + """The whole feature: press both, speak while they are down, let go to send. + + Three moments, and the order between them is the point. The press-down warms the + models and opens the microphone; the release stops it and sends what was said. The + old gesture put one word on this queue at the release and nothing at the press, and + the difference is a reload that used to land inside the user's first sentence. + """ + + def test_a_clean_hold_warms_then_captures_then_sends(self): + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_the_warm_lands_before_the_capture_and_not_beside_it(self): + # The whole reason the warm exists. A warm that arrived with `talk` would be the + # preload `Session.start` already does, and the hold would still be spent + # waiting. Asserted at the press, before any release exists. + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), ["warm", "talk"]) + + def test_nothing_starts_until_the_chord_is_complete(self): + # One modifier is not a hold. A bare Ctrl tap is a thing hands do constantly + # while thinking, and opening a microphone on it would be the defect + # `parse_chord` refuses single-modifier chords to avoid. + chord, presses = _chord() + _Keyboard(chord).down(VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_it_ends_on_the_first_release_and_not_again_on_the_second(self): + # The reason `_armed` is latched rather than recomputed. Two modifiers go up as + # two events, and a chord that asked "are they all up now?" would either end + # twice — sending the same utterance into the window twice — or need the + # releases in a particular order. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.up(VK_LWIN) + self.assertEqual(_fired(presses), HOLD) + keys.up(VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_the_order_the_two_go_down_in_does_not_matter(self): + for first, second in ((VK_LCONTROL, VK_LWIN), (VK_LWIN, VK_LCONTROL)): + with self.subTest(first=first): + chord, presses = _chord() + _Keyboard(chord).down(first, second).up(first, second) + self.assertEqual(_fired(presses), HOLD) + + def test_a_repeated_keydown_does_not_reopen_the_microphone(self): + # The OS repeats a held key in some configurations, and `_armed` is what stops a + # second `talk` arriving mid-utterance — which the UI could not tell apart from + # the user having spoken into a microphone reopened under them. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LCONTROL, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk"]) + + def test_holding_it_three_times_is_three_utterances(self): + # Obvious, and the one that would catch a flag that latches on and never clears. + chord, presses = _chord() + keys = _Keyboard(chord) + for _ in range(3): + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD * 3) + + def test_either_side_of_the_keyboard_works(self): + chord, presses = _chord() + _Keyboard(chord).down(VK_RCONTROL, VK_RWIN).up(VK_RWIN, VK_RCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_a_generic_modifier_code_works_because_injected_keys_carry_one(self): + # A physical press arrives sided (`VK_LCONTROL`); a synthesised one may carry the + # generic `VK_CONTROL`. `flow/inject.py` synthesises keys itself, so a hook that + # listened for only one of the two would behave differently depending on whether + # a human or a program pressed the chord. + chord, presses = _chord() + _Keyboard(chord).down(hotkey.VK_CONTROL, VK_LWIN).up(VK_LWIN, hotkey.VK_CONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_the_alt_bearing_chords_arrive_as_sys_keys_and_still_work(self): + # Windows sends WM_SYSKEYDOWN rather than WM_KEYDOWN while Alt is held. A chord + # containing alt that only watched the plain messages would never fire at all. + chord, presses = _chord(("ctrl", "alt")) + _Keyboard(chord).sys_down(VK_LCONTROL, VK_LMENU).sys_up(VK_LMENU, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_it_puts_the_words_it_was_built_with(self): + # The chord is a second way in, not a second thing to handle: it writes into + # `Hotkeys.presses`, so everything downstream drains one stream and cannot tell a + # chord from a registered combo. + presses = queue.Queue() + chord = Chord(presses, frozenset({"ctrl", "win"}), action="go", + warm_action="heat", end_action="stop", break_action="drop") + keys = _Keyboard(chord) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["heat", "go", "stop"]) + keys.down(VK_LCONTROL, VK_LWIN).down(VK_D).up(VK_D, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["heat", "go", "drop"]) + + def test_the_default_words_are_the_ones_the_ui_dispatches_on(self): + # A tripwire on four strings that live in two files: `Pill._frame` matches each + # by literal, and a rename here that missed it would land as a chord that opens + # a microphone nothing ever closes. + chord, _presses = _chord() + ui = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + for word in (chord.warm_action, chord.action, + chord.end_action, chord.break_action): + with self.subTest(word=word): + self.assertIn('name == "%s"' % word, ui) + + +class TestBBBothGesturesShipAndNeitherReplacesTheOther(unittest.TestCase): + """`Chord.gesture`, and the switch that should have been there from the start. + + Shipping push-to-talk *instead of* the toggle took a working gesture away from + everybody who had it. They are good at different things: a hold needs no decision + about when you are finished and cannot leave a microphone running, and a toggle is + the only one of the two that survives a paragraph, a long thought with pauses in it, + or hands that cannot hold two keys down for a minute. + """ + + def toggler(self): + chord, presses = _chord() + chord.gesture = "toggle" + return chord, presses + + def test_hold_is_what_ships(self): + self.assertEqual(hotkey.GESTURE_DEFAULT, "hold") + self.assertEqual(_chord()[0].gesture, "hold") + + def test_a_toggle_chord_fires_one_word_on_a_clean_release(self): + chord, presses = self.toggler() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["toggle"]) + + def test_a_toggle_chord_does_nothing_at_all_on_the_press(self): + # It has no press-down half. Warming here would load 605 MB of models every time + # somebody reached for `ctrl+win+arrow`, which is the cost the hold gesture + # accepts on purpose and this one has no reason to. + chord, presses = self.toggler() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), []) + + def test_a_toggle_chord_still_refuses_what_windows_meant(self): + # The original rule, unchanged and still doing its job: `ctrl+win+d` makes a + # desktop and starts nothing. Under `hold` this is a break; here it is silence. + chord, presses = self.toggler() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_D).up(VK_D).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), []) + + def test_it_never_emits_a_word_the_other_gesture_owns(self): + # The two vocabularies are disjoint, which is what lets one dispatch table serve + # both without a mode flag at the far end. + chord, presses = self.toggler() + keys = _Keyboard(chord) + for _ in range(3): + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(set(_fired(presses)) & set(HOLD), set()) + + def test_switching_gesture_is_one_assignment_and_takes_effect_at_once(self): + # The reason `gesture` is a plain attribute the callback reads rather than + # something baked in at construction: switching by rebuilding would mean + # unhooking and re-installing a `WH_KEYBOARD_LL` hook, which the OS may refuse — + # and being refused *while changing a setting* leaves somebody with no chord. + chord, presses = _chord() + keys = _Keyboard(chord) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + chord.gesture = "toggle" + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["toggle"]) + + def test_an_unknown_gesture_falls_back_rather_than_disabling_the_chord(self): + # It arrives from a hand-edited profile. A typo must cost the setting, not the + # shortcut — a chord that silently did nothing would be unattributable. + for name in ("Hold", "push-to-talk", "", None, 7): + with self.subTest(name=name): + presses = queue.Queue() + chord = Chord(presses, frozenset({"ctrl", "win"}), gesture=name) + self.assertEqual(chord.gesture, hotkey.GESTURE_DEFAULT) + + def test_the_menu_offers_exactly_the_gestures_that_exist(self): + # `flow/ui.py` cannot import `flow.hotkey` — that module binds user32 at import — + # so the menu keeps its own labels. This is the assertion that stops the two + # lists drifting into a row that selects a gesture the hook does not know. + import flow.ui as ui + + self.assertEqual(tuple(ui.GESTURE_LABELS), hotkey.GESTURES) + + +class TestCWindowsOwnsCtrlWinToo(unittest.TestCase): + """`ctrl+win` is a Windows prefix, and push-to-talk changed what that costs. + + Under the old toggle gesture this class asserted that `ctrl+win+d` did nothing at + all: nothing had started, so refusing to fire at the release was the whole + behaviour. Push-to-talk opens the microphone on the press-down, so a desktop switch + now genuinely starts capturing — and the promise has to be restated rather than + quietly kept: + + **A third key stops the capture on the keystroke, and sends nothing.** Not at the + release, which would record every desktop switch for as long as the user held the + keys; and never as a send, which is the half that would put words in a window. + + What is captured in the fifty milliseconds before the third key is whatever the gate + let through, which is nothing — nobody has begun speaking yet. `Session.talk_end` + commits it regardless rather than applying a minimum-length rule, on the grounds + that an empty utterance costs nothing and a threshold eventually eats a real word. + """ + + def test_ctrl_win_d_makes_a_virtual_desktop_and_sends_nothing(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_D).up(VK_D).up(VK_LWIN, VK_LCONTROL) + fired = _fired(presses) + self.assertEqual(fired, ["warm", "talk", "talk-break"]) + self.assertNotIn("talk-end", fired) + + def test_ctrl_win_left_switches_desktop_and_sends_nothing(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LEFT).up(VK_LEFT).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_the_break_happens_on_the_third_key_not_on_the_release(self): + # The timing is the whole difference. Waiting for the release would leave the + # microphone open across every desktop switch in a long hold. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertEqual(_fired(presses), ["warm", "talk"]) + keys.down(VK_LEFT) + self.assertEqual(_fired(presses), ["talk-break"]) + + def test_switching_desktop_twice_breaks_once(self): + # `_talking` is cleared by the first break, so the second arrow finds nothing to + # stop. A second break would ask the session to close a microphone it has + # already closed. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(VK_LEFT).up(VK_LEFT).down(VK_LEFT).up(VK_LEFT) + keys.up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_an_extra_modifier_is_a_different_chord_and_never_starts_one(self): + # ctrl+shift+win is not ctrl+win. Nothing starts, so nothing needs breaking. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LSHIFT, VK_LCONTROL, VK_LWIN) + keys.up(VK_LWIN, VK_LCONTROL, VK_LSHIFT) + self.assertEqual(_fired(presses), []) + + def test_the_extra_modifier_blocks_it_whenever_it_went_down(self): + # Both orders, because the first version of this got one of them wrong: an + # unwanted modifier pressed *before* the chord formed was forgotten when arming + # reset the verdict. Held state and press history are different questions, and + # this is the pair that tells them apart. The one order that is not here is + # shift arriving last — that is a break, and it has its own test below. + for order in ((VK_LSHIFT, VK_LCONTROL, VK_LWIN), + (VK_LCONTROL, VK_LSHIFT, VK_LWIN)): + with self.subTest(order=order): + chord, presses = _chord() + keys = _Keyboard(chord).down(*order) + keys.up(*reversed(order)) + self.assertEqual(_fired(presses), []) + + def test_shift_arriving_last_is_a_break_like_any_other_third_key(self): + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_LSHIFT) + keys.up(VK_LSHIFT, VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), ["warm", "talk", "talk-break"]) + + def test_releasing_the_extra_modifier_first_frees_the_chord_for_the_next_hold(self): + # `_extra` is held state, so it has to clear on release. If it latched, one + # accidental Shift would kill the chord for the life of the process — a defect + # whose only symptom is that the feature stops working and never comes back. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LSHIFT).up(VK_LSHIFT) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + def test_one_modifier_alone_never_starts_however_long_it_is_held(self): + for vk in (VK_LCONTROL, VK_LWIN): + with self.subTest(vk=vk): + chord, presses = _chord() + _Keyboard(chord).down(vk, vk, vk).up(vk) + self.assertEqual(_fired(presses), []) + + def test_ordinary_typing_never_starts_one(self): + chord, presses = _chord() + keys = _Keyboard(chord) + for vk in (VK_A, VK_D, VK_LEFT): + keys.down(vk).up(vk) + self.assertEqual(_fired(presses), []) + + def test_a_key_pressed_before_the_chord_formed_is_not_the_chords_business(self): + # Typing, then reaching for the chord, is somebody starting a new gesture — not + # a dirty hold. The history resets when the chord forms; only what is still + # *held* survives into the verdict. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_A).up(VK_A) + keys.down(VK_LCONTROL, VK_LWIN).up(VK_LWIN, VK_LCONTROL) + self.assertEqual(_fired(presses), HOLD) + + +class TestDItLearnsNothingAboutTheKeysItRejects(unittest.TestCase): + """The narrowing of R16, asserted rather than asserted *about*. + + The claim being defended is not "Flow is trustworthy" — it is that a key outside the + chord changes one boolean and leaves nothing behind. So the test is a state + comparison: type different things, and check the object cannot tell them apart. + """ + + def test_two_different_keys_leave_the_object_in_identical_states(self): + # If a virtual key were being recorded anywhere, typing `a` and typing `d` would + # have to produce different objects. They must not. + # Skipped by name, and the list is short on purpose: these are the per-instance + # objects that can only ever compare unequal — a queue, a thread, a C callback. + # *Everything else* is compared, which is what makes this a trap rather than a + # restatement: a field added tomorrow to hold a virtual key would be compared by + # default and would fail here. + identity = {"presses", "_proc", "_thread", "_ready", "_hook", "_tid"} + states = [] + for vk in (VK_A, VK_D, VK_LEFT): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + keys.down(vk).up(vk) + states.append(vars(chord).copy()) + for other in states[1:]: + for name, value in states[0].items(): + if name in identity: + continue + with self.subTest(field=name): + self.assertEqual(value, other[name]) + + def test_what_it_keeps_about_a_rejected_key_is_one_boolean(self): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN) + self.assertFalse(chord._other) + keys.down(VK_A) + self.assertIs(chord._other, True) + + def test_the_only_fields_it_has_are_the_ones_the_state_machine_needs(self): + # A guard on the shape rather than on today's code: a future field holding a + # keystroke would have to be added here first, which is the moment to argue + # about it. + chord, _presses = _chord() + _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_A).up(VK_A, VK_LWIN, VK_LCONTROL) + self.assertEqual( + set(vars(chord)), + {"presses", "mods", "action", "warm_action", "end_action", "break_action", + "toggle_action", "gesture", "installed", "_down", "_other", "_extra", + "_armed", "_talking", "_hook", "_tid", "_ready", "_proc", "_thread"}, + ) + + +class TestEItNeverSwallowsAKeystroke(unittest.TestCase): + """Ctrl and Win have real jobs, and Flow does not get to keep them.""" + + def test_every_event_reaches_the_next_hook(self): + chord, _presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN, VK_A) + keys.up(VK_A, VK_LWIN, VK_LCONTROL) + self.assertTrue(keys.passed) + for message, vk, passed_on in keys.passed: + with self.subTest(message=message, vk=vk): + self.assertTrue(passed_on) + + def test_including_the_release_that_fired_the_chord(self): + # The one most likely to be lost to a `return 1` added in a hurry — and losing it + # means the Ctrl release never reaches the app, which strands every ctrl- + # shortcut in whatever window had focus. + chord, presses = _chord() + keys = _Keyboard(chord).down(VK_LCONTROL, VK_LWIN).up(VK_LWIN) + self.assertEqual(_fired(presses), HOLD) + self.assertTrue(keys.passed[-1][2]) + + def test_a_negative_code_is_passed_on_without_being_looked_at(self): + # `nCode < 0` means "pass it on without looking", and it is not advice. A hook + # that inspected those events anyway is a hook that can fire on something the OS + # explicitly said not to read. + chord, presses = _chord() + chord._down["ctrl"] = chord._down["win"] = True + chord._armed = True + block = hotkey._KBDLLHOOKSTRUCT(vkCode=VK_LWIN, scanCode=0, flags=0, time=0, + dwExtraInfo=None) + with mock.patch.object(hotkey, "user32") as fake: + fake.CallNextHookEx.return_value = 0 + chord._on_key(-1, WM_KEYUP, ctypes.addressof(block)) + self.assertTrue(fake.CallNextHookEx.called) + self.assertEqual(_fired(presses), []) + self.assertTrue(chord._armed) + + +class TestFTheHookIsTornDownByWhoeverOwnsGlobalKeyInput(unittest.TestCase): + """A hook left installed is one the OS calls into a dead interpreter.""" + + def test_stopping_the_hotkeys_stops_the_chord(self): + # The pill already calls `hotkeys.stop()` on the way out, and there should be + # exactly one thing that owns the teardown of "global key input". Hanging the + # chord off `Hotkeys` is what makes that true without `flow/ui.py` learning a + # second name. + hotkeys = hotkey.Hotkeys(hotkey.DEFAULT_BINDINGS) + chord, _presses = _chord() + hotkeys.chord = chord + with mock.patch.object(chord, "stop") as stop: + with mock.patch.object(hotkey, "user32"): + hotkeys.stop() + self.assertTrue(stop.called) + + def test_hotkeys_without_a_chord_still_stop_cleanly(self): + # The `--no-chord` and hook-refused paths both leave `chord` as None, and the + # quit path must not care which one it is looking at. + hotkeys = hotkey.Hotkeys(hotkey.DEFAULT_BINDINGS) + self.assertIsNone(hotkeys.chord) + with mock.patch.object(hotkey, "user32"): + hotkeys.stop() + + def test_a_refused_hook_is_a_false_from_start_and_not_a_crash(self): + # `SetWindowsHookExW` answers NULL when the OS refuses — policy, another process, + # a desktop this one cannot reach into. Not fatal: the registered toggle is still + # there, and `flow/__main__.py` has a line to print about it. + chord, _presses = _chord() + with mock.patch.object(chord, "_install", return_value=None): + self.assertFalse(chord.start(timeout=5.0)) + self.assertFalse(chord.installed) + + def test_a_raising_install_is_the_same_event_as_a_refused_one(self): + # The failure that would otherwise surface as a traceback on a daemon thread's + # stderr — which on a windowed build is nowhere at all — while `start()` blocked + # for the whole timeout waiting for a flag nobody was going to set. + chord, _presses = _chord() + with mock.patch.object(hotkey, "user32") as fake: + fake.SetWindowsHookExW.side_effect = OSError("refused") + self.assertFalse(chord.start(timeout=5.0)) + self.assertFalse(chord.installed) + + def test_the_callback_is_held_so_the_os_cannot_call_a_collected_one(self): + # Without a reference on the instance the `WINFUNCTYPE` object is collected while + # still installed, and the process dies inside a keystroke somewhere unrelated — + # a crash with no connection to anything Flow was doing. + chord, _presses = _chord() + self.assertIsNotNone(chord._proc) + self.assertIn("_proc", vars(chord)) + + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_help.py b/tests/test_help.py index ac8f81c..a0d1ca5 100644 --- a/tests/test_help.py +++ b/tests/test_help.py @@ -29,9 +29,24 @@ class FakeHotkeys: """What `Hotkeys` looks like after `start()`: what registered, and what could not.""" - def __init__(self, chosen: dict, failed=()) -> None: + def __init__(self, chosen: dict, failed=(), chord=None) -> None: self.chosen = chosen self.failed = list(failed) + #: `None` on the launches that have no chord — `--no-chord`, `"chord": ""`, or a + #: hook the OS refused — which is most of the tests in this file and is why it + #: defaults that way. + self.chord = chord + + +class FakeChord: + """What `Chord` looks like once installed. It is not in `chosen` and never will be: + nothing registered it, because there is nothing `RegisterHotKey` could register.""" + + def __init__(self, spelling="ctrl+win", action="talk") -> None: + self.spelling, self.action = spelling, action + + def describe(self) -> str: + return self.spelling REGISTERED = {"toggle": "ctrl+shift+space", "send": "ctrl+alt+enter", @@ -62,6 +77,30 @@ def test_an_action_that_could_not_register_is_named_as_unavailable(self): line = next(ln for ln in text.splitlines() if "cancel" in ln) self.assertIn("owned by another app", line) + def test_the_chord_is_named_even_though_nothing_registered_it(self): + # The sheet is built from what `RegisterHotKey` accepted, and the chord is + # precisely the shortcut that call cannot express. A sheet built only from + # `chosen` would describe a machine on which the shape the user has been holding + # all week does not exist. + text = rendered(hotkeys=FakeHotkeys(REGISTERED, chord=FakeChord())) + self.assertIn("ctrl+win", text) + line = next(ln for ln in text.splitlines() if "ctrl+win" in ln) + self.assertIn("hold to talk, release to send", line) + self.assertIn("held", line) + + def test_a_launch_without_a_chord_does_not_mention_one(self): + # `--no-chord`, `"chord": ""` and a refused hook all land here, and they are the + # same rule the rest of the sheet follows: it says what works on this machine + # this launch, and nothing else. + self.assertNotIn("ctrl+win", rendered(hotkeys=FakeHotkeys(REGISTERED))) + + def test_the_chord_is_listed_before_the_hotkey_that_also_starts_dictation(self): + # They no longer do the same thing — the chord is a hold and the hotkey is a + # toggle — but they are the two ways to start talking, and the chord goes first + # because it is the one somebody opened this sheet to look up. + text = rendered(hotkeys=FakeHotkeys(REGISTERED, chord=FakeChord())) + self.assertLess(text.index("ctrl+win"), text.index("ctrl+shift+space")) + def test_no_hotkeys_at_all_is_a_sentence_rather_than_a_hole(self): # `--no-hotkeys` is a supported way to run, and an empty section reads as a bug # in the help rather than as a choice the user made at launch. diff --git a/tests/test_hotkey.py b/tests/test_hotkey.py index 3e51f95..8deb20d 100644 --- a/tests/test_hotkey.py +++ b/tests/test_hotkey.py @@ -313,6 +313,17 @@ def GetMessageW(self, *_args): def PostThreadMessageW(self, *_args): return 1 + #: The chord's hook, refused. This fake describes a machine with combos already + #: taken, and a hook is not a combo — nothing here is about `WH_KEYBOARD_LL`, and a + #: fake that granted one would have `main()` install a real global keyboard hook on + #: whoever is running the suite. NULL is a shape the app already handles: the + #: startup block says the chord is unavailable and the registered toggle stands. + def SetWindowsHookExW(self, *_args): + return None + + def UnhookWindowsHookEx(self, *_args): + return 1 + class Registered(unittest.TestCase): """One launch's worth of registration against a machine this test describes.""" diff --git a/tests/test_indicator.py b/tests/test_indicator.py index 73b3d63..9303353 100644 --- a/tests/test_indicator.py +++ b/tests/test_indicator.py @@ -363,7 +363,7 @@ def _pill(mode=CONVERSE, pinned=None): pill._flash = 0 pill.armed = True pill._clis = None - pill.levels = [0.0] * 18 + pill._meter_level = 0.0 pill.session = mock.Mock(mode=mode, state=State.IDLE, cli=pinned) return pill diff --git a/tests/test_lite.py b/tests/test_lite.py index 6bfb969..ab3f9e3 100644 --- a/tests/test_lite.py +++ b/tests/test_lite.py @@ -319,6 +319,12 @@ def _pill(self, lite: bool): pill = ui.Pill.__new__(ui.Pill) pill.lite = lite pill.paste_target = None + # `_track_target` writes the app name onto the session for `_app_note` to read + # later. A stand-in rather than a mock, because what the tests below check is the + # *value* that lands there — and on a `Pill` built by `__new__`, a missing + # attribute is not an AttributeError but a `tkinter` lookup that recurses until + # the interpreter gives up, which is a confusing way to learn this line exists. + pill.session = type("S", (), {"target_app": ""})() return pill def test_the_foreground_is_never_asked_about_in_lite(self): @@ -345,6 +351,45 @@ def test_full_mode_still_tracks_it(self): pill._track_target() self.assertEqual(pill.paste_target, 0x99) + def test_the_app_behind_the_window_is_named_for_the_per_app_note(self): + import flow.ui as ui + + pill = self._pill(lite=False) + with mock.patch.object(ui, "foreground_hwnd", return_value=0x99), mock.patch.object(ui, "owned_by_flow", return_value=False), mock.patch.object(ui, "classify") as named: + named.return_value = type("T", (), {"process": "code.exe"})() + pill._track_target() + self.assertEqual(pill.session.target_app, "code.exe") + + def test_it_is_resolved_on_the_edge_and_not_once_a_frame(self): + # `classify` opens a process handle and this runs at 30 fps. Paying that every + # frame is a cost paid forever to answer a question whose answer moves a few + # times an hour — so it is asked when the window changes and remembered between. + import flow.ui as ui + + pill = self._pill(lite=False) + with mock.patch.object(ui, "foreground_hwnd", return_value=0x99), mock.patch.object(ui, "owned_by_flow", return_value=False), mock.patch.object(ui, "classify") as named: + named.return_value = type("T", (), {"process": "code.exe"})() + for _ in range(10): + pill._track_target() + self.assertEqual(named.call_count, 1) + named.return_value = type("T", (), {"process": "slack.exe"})() + with mock.patch.object(ui, "foreground_hwnd", return_value=0xAB): + pill._track_target() + self.assertEqual(named.call_count, 2) + self.assertEqual(pill.session.target_app, "slack.exe") + + def test_lite_never_names_an_app_because_it_never_has_a_target(self): + # No target-window awareness at all (product.md), which reads downstream as an + # app with no note configured — the behaviour every launch had before per-app + # notes existed, rather than a gap. + import flow.ui as ui + + pill = self._pill(lite=True) + with mock.patch.object(ui, "classify") as named: + pill._track_target() + named.assert_not_called() + self.assertEqual(pill.session.target_app, "") + class TestTheWindowsOnlyTkAttributes(unittest.TestCase): """`-transparentcolor` and `-toolwindow` exist only on Windows. diff --git a/tests/test_longrun.py b/tests/test_longrun.py index 77b2ea0..74462f9 100644 --- a/tests/test_longrun.py +++ b/tests/test_longrun.py @@ -92,6 +92,52 @@ def test_model_is_dropped_after_idle(self): self.assertEqual(asr.unloads, 1) s.close() + def test_a_warm_holds_the_models_against_an_idle_that_is_already_due(self): + # The race the grace window exists for. `_last_activity` is moved by Flow's own + # milestones, so somebody who has just reached for the chord is still idle by + # that measure — and the health pump runs every tick, so without this it is free + # to unload between the press-down and the release that arms. + asr = TrackingAsr() + s = Session(asr=asr, mic=StubMic()) + s.start() + s.warm() + with mock.patch.object(session_mod, "IDLE_UNLOAD_SEC", 0.0): + s.tick() + self.assertTrue(asr.loaded) + self.assertEqual(asr.unloads, 0) + s.close() + + def test_and_lets_go_once_the_grace_is_spent(self): + # A window, not a veto. `ctrl+win` is also Windows' desktop-switch prefix, so a + # warm that reset the idle clock outright would mean anybody who switches + # desktops through the day never unloads at all — the setting would quietly stop + # existing for exactly the people using their machine most. + asr = TrackingAsr() + s = Session(asr=asr, mic=StubMic()) + s.start() + s.warm() + with mock.patch.object(session_mod, "IDLE_UNLOAD_SEC", 0.0), \ + mock.patch.object(session_mod, "WARM_GRACE_SEC", 0.0): + s.warm() + s.tick() + self.assertFalse(asr.loaded) + self.assertEqual(asr.unloads, 1) + s.close() + + def test_a_session_nobody_warmed_is_not_holding_anything_off(self): + # Zero is the state every session starts in and mostly stays in, so the guard + # must not be what decides the ordinary case. + s = Session(asr=TrackingAsr(), mic=StubMic()) + self.assertEqual(s._warm_until, 0.0) + s.close() + + def test_the_idle_threshold_is_the_gaps_in_a_day_and_not_five_minutes(self): + # Stated as a number because it is a judgement and not an accident: five minutes + # was inside the gaps of an ordinary working session, so the common case was not + # reclaiming memory from somebody who left, it was paying a reload in the middle + # of their first sentence back. + self.assertEqual(session_mod.IDLE_UNLOAD_SEC, 1800.0) + def test_model_is_kept_while_a_draft_is_held(self): # Unloading mid-draft would make the next correction pay a reload for nothing. asr = TrackingAsr() diff --git a/tests/test_overlay.py b/tests/test_overlay.py new file mode 100644 index 0000000..bbb2453 --- /dev/null +++ b/tests/test_overlay.py @@ -0,0 +1,205 @@ +"""Panel size: how wide the draft bubble and the conversation card draw. + +Three widths, chosen from a menu, remembered in the profile. The interesting part is +what the setting is *not* allowed to do, because the constants it moves were not chosen +by taste — each carries a measurement in its comment, and a size setting is exactly the +kind of change that quietly invalidates one. + + **It cannot go below 420.** That is a floor set by two measured rows: the bubble's + five-chip row runs to 345 px (`chip_row_gap` records that 380 clipped Send by half a + label) and the card's to 377. A panel narrower than its own chip row is a window whose + Send button is off the edge, and Send is the exit that must never be unreachable — so + the clamp is tested, not just written down. + + **It cannot change what today's users see.** `regular` has to reproduce the shipped + numbers exactly. A size setting that also silently re-flowed everybody's draft would + be two changes shipped as one, and only one of them was asked for. + + **The two windows cannot drift apart.** The bubble and the card are the same window at + two moments, docked to the same pill, and a width that moved one without the other + would be visible the first time somebody switched modes. + +`apply_panel_width` rebinds module globals, which is a concession to twenty-odd read +sites across two window classes rather than a preference — so these tests put the module +back the way they found it, and one of them checks that a launch which never calls it is +the launch Flow always had. +""" + +import sys +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +import flow.ui as ui # noqa: E402 +from flow.profile import PANEL_DEFAULT, Profile # noqa: E402 + +#: The shipped numbers, captured at import before any test moves them. Written as +#: literals as well, because a test that compared the module to itself would pass just +#: as happily after somebody changed all four. +SHIPPED = {"BUBBLE_W": 420, "CARD_W": 420, + "BODY_CHARS_PER_LINE": 62, "BODY_TAIL_CHARS": 1750} + + +class PanelWidth(unittest.TestCase): + """Restores the module afterwards: these globals outlive the test that moved them.""" + + def setUp(self): + self.addCleanup(ui.apply_panel_width, ui.PANEL_WIDTHS[ui.PANEL_DEFAULT]) + + def widths(self) -> dict: + return {name: getattr(ui, name) for name in SHIPPED} + + +class TestTheShippedWidthIsUntouched(PanelWidth): + def test_the_module_starts_where_it_always_did(self): + # Before anything calls `apply_panel_width` at all — the state of a launch that + # never got as far as reading a profile, and of every test module that imports + # `flow.ui` for some other reason. + self.assertEqual(self.widths(), SHIPPED) + + def test_and_asking_for_regular_puts_it_back_exactly(self): + # The promise to everybody who never opens this menu. A size setting that also + # re-flowed their draft would be two changes shipped as one. + ui.apply_panel_width(ui.PANEL_WIDTHS["large"]) + ui.apply_panel_width(ui.PANEL_WIDTHS["regular"]) + self.assertEqual(self.widths(), SHIPPED) + + def test_the_default_name_means_the_shipped_width(self): + self.assertEqual(ui.PANEL_WIDTHS[ui.PANEL_DEFAULT], SHIPPED["BUBBLE_W"]) + + +class TestTheFloorHoldsBecauseSendMustStayReachable(PanelWidth): + def test_a_width_below_the_floor_is_clamped_to_it(self): + # The measured rows are 345 px (bubble) and 377 px (card) of chips before gaps + # and padding. Below 420 the row either loses its gaps or loses a label, and the + # label it loses first is Send. + for asked in (0, 1, 100, 379, 419, -50): + with self.subTest(asked=asked): + ui.apply_panel_width(asked) + self.assertEqual(ui.BUBBLE_W, SHIPPED["BUBBLE_W"]) + + def test_no_offered_size_is_below_the_floor(self): + # A guard on the menu rather than on the clamp: an entry that had to be clamped + # would be a row somebody could pick that silently did nothing. + for name, width in ui.PANEL_WIDTHS.items(): + with self.subTest(name=name): + self.assertGreaterEqual(width, SHIPPED["BUBBLE_W"]) + + def test_the_sizes_only_go_up(self): + # Stated as a property because it is a design decision and not an accident: + # there is no "small", and the reason is the chip row rather than restraint. + self.assertEqual(min(ui.PANEL_WIDTHS.values()), SHIPPED["BUBBLE_W"]) + + +class TestTheTwoWindowsMoveTogether(PanelWidth): + def test_the_card_follows_the_bubble_at_every_size(self): + # One window at two moments, docked to the same pill. A width that moved one + # without the other is visible the first time somebody switches mode. + for width in ui.PANEL_WIDTHS.values(): + with self.subTest(width=width): + ui.apply_panel_width(width) + self.assertEqual(ui.CARD_W, ui.BUBBLE_W) + + +class TestTheMeasurementsScaleWithTheColumn(PanelWidth): + def test_a_wider_panel_lays_out_more_text(self): + # `BODY_CHARS_PER_LINE` frozen at one width would under-feed the canvas at 640 px + # put the bottom of the draft below the fold — the setting would make the thing + # it exists to improve worse. + seen = [] + for width in sorted(ui.PANEL_WIDTHS.values()): + ui.apply_panel_width(width) + seen.append((ui.BODY_CHARS_PER_LINE, ui.BODY_TAIL_CHARS)) + self.assertEqual(seen, sorted(seen)) + self.assertLess(seen[0][0], seen[-1][0]) + + def test_the_tail_stays_the_same_number_of_lines(self): + # What `BODY_TAIL_CHARS` really encodes is ~28 lines, which is what keeps render + # cost flat (invariant 7). Holding lines rather than characters is what carries + # that invariant across a width change instead of re-measuring it. + lines = [] + for width in ui.PANEL_WIDTHS.values(): + ui.apply_panel_width(width) + lines.append(round(ui.BODY_TAIL_CHARS / ui.BODY_CHARS_PER_LINE)) + self.assertEqual(len(set(lines)), 1, lines) + + def test_a_line_is_never_zero_characters(self): + # The clamp exists because this number is divided by. It cannot be reached + # through the menu; it can be reached by a future width and a smaller font. + ui.apply_panel_width(0) + self.assertGreaterEqual(ui.BODY_CHARS_PER_LINE, 1) + + +class TestANameIsTurnedIntoAWidth(PanelWidth): + def test_every_offered_name_resolves_to_its_own_width(self): + for name, width in ui.PANEL_WIDTHS.items(): + with self.subTest(name=name): + self.assertEqual(ui.panel_width(name), width) + + def test_case_and_spacing_are_the_writers_business(self): + # This is a value a hand-edit can put in `profile.json`, so it reads the way + # every other hand-written value in that file does. + for text in ("LARGE", "Large", " large ", "lArGe"): + with self.subTest(text=text): + self.assertEqual(ui.panel_width(text), ui.PANEL_WIDTHS["large"]) + + def test_anything_unknown_is_the_shipped_width_rather_than_a_refusal(self): + # Deliberately the opposite of how `hotkey.parse` treats a bad combo, and the + # difference is what each costs when wrong. A hotkey that silently fell back + # leaves somebody pressing keys that do nothing with no way to find out; a panel + # that falls back is a window visibly not the size they asked for, with the + # evidence on the screen. + for value in ("enormous", "", None, 7, ["large"], {"large": 1}, True): + with self.subTest(value=value): + self.assertEqual(ui.panel_width(value), SHIPPED["BUBBLE_W"]) + + +class TestTheProfileRemembersIt(PanelWidth): + def test_the_two_modules_agree_on_the_default_name(self): + # `flow/profile.py` spells the default itself rather than importing it, because + # it is read on every launch including Lite's and `flow.ui` is not something it + # may need. That buys platform reach and costs exactly this risk, so the risk is + # bought back here. + self.assertEqual(PANEL_DEFAULT, ui.PANEL_DEFAULT) + self.assertIn(PANEL_DEFAULT, ui.PANEL_WIDTHS) + + def test_it_survives_a_save_and_a_load(self): + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + p = Profile(path) + p.panel = "large" + self.assertTrue(p.save()) + again = Profile(path) + self.assertTrue(again.load()) + self.assertEqual(again.panel, "large") + self.assertNotIn("panel", again.faults) + + def test_a_fresh_profile_asks_for_the_shipped_width(self): + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + p = Profile(Path(tmp) / "profile.json") + self.assertEqual(ui.panel_width(p.panel), SHIPPED["BUBBLE_W"]) + + def test_a_nonsense_value_in_the_file_is_named_and_still_launches(self): + # Both halves matter. `faults` is how `--stats` and the startup block say a + # setting degraded, and the launch has to survive it: a profile is not a thing + # somebody can be locked out of the app by. + import json + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "profile.json" + path.write_text(json.dumps({"schema": 1, "panel": 7}), encoding="utf-8") + p = Profile(path) + self.assertTrue(p.load()) + self.assertIn("panel", p.faults) + self.assertEqual(ui.panel_width(p.panel), SHIPPED["BUBBLE_W"]) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) diff --git a/tests/test_pill.py b/tests/test_pill.py index afdda73..648254d 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -35,6 +35,11 @@ def __init__(self) -> None: def delete(self, *a, **kw) -> None: ... + #: `_sync_dock` resizes the canvas when a panel docks or goes away. Accepted and + #: recorded rather than ignored, so a test can tell a widened pill from a moved one. + def configure(self, **kw) -> None: + self.width = kw.get("width", getattr(self, "width", None)) + def create_polygon(self, *a, **kw) -> None: ... def create_arc(self, *a, **kw) -> None: ... @@ -63,7 +68,7 @@ def pill(state=State.IDLE, *, armed=True, mode=DICTATE, hearing=True, p = ui.Pill.__new__(ui.Pill) p.canvas = Canvas() p.armed = armed - p.levels = [0.0] * ui.BARS + p._meter_level = 0.0 p.session = mock.Mock( mode=mode, state=state, hearing=hearing, editing=editing, cli=None, mic=mock.Mock(active=mic_active), @@ -410,6 +415,328 @@ def test_the_glyph_and_the_label_travel_together(self): self.assertEqual({o[-1] for o in p.canvas.ovals if o[0] < ui.METER_X}, {half}) +class TestTheMeterBloomsFromItsCentre(unittest.TestCase): + """The shape taken from FluidVoice's `BottomWaveformView`, asserted as a shape. + + Flow's meter used to be a scrolling history — one level per frame through a deque, + travelling right to left. It is a symmetric bloom now: every bar reads the same + current level, and the envelope is what makes the middle ones tallest. These are + the properties that distinguish the two, so that a future edit which quietly + restored the old behaviour would fail here rather than merely look different. + """ + + def heights(self, level: float) -> list[float]: + return [ui.Pill._bar_half_height(i, level) for i in range(ui.BARS)] + + def test_the_middle_is_the_tallest_part_of_the_shape(self): + h = self.heights(1.0) + self.assertEqual(max(h), max(h[ui.BARS // 2 - 1:ui.BARS // 2 + 1])) + + def test_it_falls_away_toward_both_ends(self): + # Monotonic out from the centre in each direction. The per-bar variation is + # deliberately small enough not to break this — a wobble that reordered the + # envelope would be a comb, not a bloom. + h = self.heights(1.0) + mid = ui.BARS // 2 + self.assertEqual(h[:mid], sorted(h[:mid])) + self.assertEqual(h[mid:], sorted(h[mid:], reverse=True)) + + def test_the_ends_still_move_rather_than_sitting_dead(self): + # `_ENVELOPE_MIN` is 18% and not 0. An end bar pinned at the minimum would read + # as a broken meter rather than a shaped one. + self.assertGreater(self.heights(1.0)[0], self.heights(0.0)[0]) + + def test_silence_is_a_flat_line_of_stubs_and_not_an_empty_box(self): + # The one thing the old meter and this one agree on, and the reason `BAR_MIN_H` + # is not zero: an empty widget reads as "not working", not as "quiet". + self.assertEqual(self.heights(0.0), [ui.BAR_MIN_H] * ui.BARS) + + def test_nothing_ever_draws_outside_the_pill(self): + # Half-heights, mirrored, inside a 40 px pill with 8 px of air. + for level in (0.0, 0.25, 0.5, 0.75, 1.0, 2.0, -1.0): + with self.subTest(level=level): + for h in self.heights(level): + self.assertGreaterEqual(h, ui.BAR_MIN_H) + self.assertLessEqual(h, ui.BAR_MAX_H) + + def test_ordinary_speech_reaches_most_of_the_way_up(self): + # What the 0.55 exponent buys. At half level a linear meter would draw half + # height, which is what made the old one look timid at conversational volume — + # the top of the widget was reserved for shouting. + half = self.heights(0.5)[ui.BARS // 2] + full = self.heights(1.0)[ui.BARS // 2] + self.assertGreater(half, full * 0.6) + + def test_every_bar_answers_the_same_level(self): + # The bloom's defining property, and the one a reintroduced history would break: + # the shape between bars is the envelope, never the past. + first = self.heights(0.7) + self.assertEqual(first, self.heights(0.7)) + + +class TestWhereThePanelOpens(unittest.TestCase): + """FluidVoice's `positionWindow` arithmetic, ported and asserted. + + Their rule reads oddly until you notice it uses *two* rectangles on purpose: + centred on `screen.frame` — the physical display — but stood on + `screen.visibleFrame`, which excludes the Dock. Windows hands back the same pair as + `rcMonitor` and `rcWork`, so this ports without being reinterpreted. + """ + + #: A 1920×1080 display with a 48 px taskbar, offset on a virtual desktop so that a + #: test cannot pass by assuming the origin is (0, 0) — which is the bug a + #: second-monitor user gets. + FULL = (1920, 0, 3840, 1080) + WORK = (1920, 0, 3840, 1032) + + def test_it_centres_on_the_physical_display(self): + x, _y = ui.bottom_centre(400, 100, self.FULL, self.WORK) + self.assertEqual(x, 1920 + (1920 - 400) // 2) + + def test_it_stands_on_the_work_area_and_not_the_screen_edge(self): + # The asymmetry that matters: centred on `full`, but lifted clear of the + # taskbar. Standing on `full` would put the panel under it. + _x, y = ui.bottom_centre(400, 100, self.FULL, self.WORK) + self.assertLessEqual(y + 100, self.WORK[3]) + + def test_the_offset_lifts_it_and_is_measured_from_the_bottom(self): + # Both offsets clear of the 10 px floor, so this measures the offset rather than + # the clamp. FluidVoice floors at `visibleFrame.minY + 10` the same way, which + # is why an offset of 0 and an offset of 10 land in the same place. + _x, near = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=20) + _x, far = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=120) + self.assertEqual(near - far, 100) + + def test_the_last_ten_pixels_are_a_floor_rather_than_a_range(self): + flush = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=0) + floored = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset=10) + self.assertEqual(flush, floored) + + def test_an_offset_from_a_hand_edited_profile_cannot_push_it_off_screen(self): + # The clamp is what makes the offset safe to expose as a setting. Both ends, + # because a negative number is as easy to type as a huge one. + for offset in (-10_000, -1, 0, 900, 10_000): + with self.subTest(offset=offset): + x, y = ui.bottom_centre(400, 100, self.FULL, self.WORK, offset) + self.assertGreaterEqual(y, self.WORK[1] + 40) + self.assertLessEqual(y + 100, self.WORK[3] - 10) + self.assertGreaterEqual(x, self.WORK[0]) + self.assertLessEqual(x + 400, self.WORK[2]) + + def test_a_panel_taller_than_the_display_keeps_its_bottom(self): + # The one case where the clamp cannot satisfy both ends. The bottom is the edge + # worth keeping: the top of a draft can run under the taskbar and still be read, + # and the chips that act on it live at the bottom. + _x, y = ui.bottom_centre(400, 5000, self.FULL, self.WORK) + self.assertEqual(y + 5000, self.WORK[3] - 10) + + def test_a_panel_wider_than_the_display_starts_at_its_left_edge(self): + x, _y = ui.bottom_centre(4000, 100, self.FULL, self.WORK) + self.assertEqual(x, self.WORK[0]) + + def test_the_monitor_under_the_pointer_answers_with_two_rectangles(self): + # `_work_area` asks `SystemParametersInfoW`, which only ever answers for the + # primary display — so on a two-monitor desk everything Flow drew landed on the + # wrong one whenever the user was working on the other. + full, work = ui._pointer_monitor(1280, 720) + for rect in (full, work): + self.assertEqual(len(rect), 4) + self.assertGreater(rect[2], rect[0]) + self.assertGreater(rect[3], rect[1]) + # The work area is the one the taskbar comes out of, so it can only be smaller. + self.assertLessEqual(work[3] - work[1], full[3] - full[1]) + self.assertLessEqual(work[2] - work[0], full[2] - full[0]) + + def test_it_degrades_to_the_primary_work_area_rather_than_raising(self): + # A repaint is not a place to handle a Win32 failure. `_NoHands` is this + # module's own idea of "nothing happened", and the fallback has to land + # somewhere drawable rather than at (0, 0) with no size. + with mock.patch.object(ui.ctypes, "windll", ui._NoHands()): + full, work = ui._pointer_monitor(1280, 720) + self.assertEqual(full, work) + self.assertGreater(full[2], full[0]) + + +class TestWhichPlacementIsInForce(unittest.TestCase): + """`PLACE`, and the two answers `_placed` gives. + + Restores the module afterwards for `tests/test_overlay.py`'s reason: this is a + global, and a test that moved it would outlive itself. + """ + + FULL = (1920, 0, 3840, 1080) + WORK = (1920, 0, 3840, 1032) + + def setUp(self): + self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) + self.p = ui.Pill.__new__(ui.Pill) + self.p.full, self.p.work = self.FULL, self.WORK + + def test_bottom_is_what_ships(self): + # Stated as a fact rather than left to the constant, because this is the change + # of default: the corner is where Windows puts the tray, every toast, and most + # apps' own status chrome, and it was the one place Flow had reserved. + self.assertEqual(ui.PLACE_DEFAULT, "bottom") + self.assertEqual(ui.PLACE, "bottom") + + def test_bottom_centres_the_stack_on_the_display(self): + ui.apply_place("bottom") + x, _y = self.p._placed(ui.PILL_W) + self.assertEqual(x, ui.bottom_centre(ui.PILL_W, ui.PILL_H, self.FULL, + self.WORK, ui.PANEL_BOTTOM_OFFSET)[0]) + + def test_corner_still_puts_it_bottom_right_where_it_always_was(self): + # Kept rather than removed: somebody who has spent months with the pill in the + # bottom right should not have it moved by an upgrade they did not ask for. + ui.apply_place("corner") + x, y = self.p._placed(ui.PILL_W) + self.assertEqual(x, self.WORK[2] - ui.PILL_W - 28) + self.assertEqual(y, self.WORK[3] - ui.PILL_H - 24) + + def test_the_two_placements_are_actually_different(self): + # A guard against both branches collapsing to the same arithmetic, which is how + # a setting comes to look like it works while doing nothing. + ui.apply_place("bottom") + here = self.p._placed(ui.PILL_W) + ui.apply_place("corner") + self.assertNotEqual(here, self.p._placed(ui.PILL_W)) + + def test_a_typo_costs_the_setting_and_not_the_app(self): + # This arrives from a hand-edited profile. Raising here would be a launch that + # dies on a misspelled word. + for name in ("bottomm", "", "BOTTOM", "centre", "left"): + with self.subTest(name=name): + ui.apply_place(name) + self.assertEqual(ui.PLACE, ui.PLACE_DEFAULT) + + def test_the_widths_it_is_asked_about_are_the_ones_it_is_placed_at(self): + # `_placed` takes the width because the pill grows to the panel's when one is + # docked. Centring a 420-wide stack using the 152-wide pill's position is how + # the stack would sit off-centre exactly when it is most visible. + ui.apply_place("bottom") + narrow, _ = self.p._placed(ui.PILL_W) + wide, _ = self.p._placed(ui.BUBBLE_W) + self.assertGreater(narrow, wide) + + +class TestTheStackFollowsThePointersMonitor(unittest.TestCase): + """The bug the placement work turned up, asserted so it cannot come back. + + `self.work` was read once in `__init__` from `SystemParametersInfoW`, which only + ever answers for the primary display. On a two-monitor desk that put every window + Flow drew against a screen the user might not be looking at — and pointed the + on-screen clamps at the wrong rectangle too. + """ + + ONE = ((0, 0, 1920, 1080), (0, 0, 1920, 1032)) + TWO = ((1920, 0, 3840, 1080), (1920, 0, 3840, 1032)) + + def pill(self, at): + p = ui.Pill.__new__(ui.Pill) + p.full, p.work = at + p.x, p.y = p._placed(ui.PILL_W) + p._docked_w = ui.PILL_W + p.canvas = Canvas() + # `pill_w` reads `front`, which reads `session.mode`. Set for the reason + # `test_lite`'s harness sets everything: `tk.Misc.__getattr__` forwards an + # unknown attribute to `self.tk`, so a missing one recurses rather than defaults. + p.session = mock.Mock(mode=DICTATE) + p.bubble = mock.Mock(_visible=False) + p.card = mock.Mock(_visible=False) + p.winfo_screenwidth = lambda: 1920 + p.winfo_screenheight = lambda: 1080 + p.window_geometry = lambda: (ui.PILL_W, p.x, p.y) + p.geometry = mock.Mock() + return p + + def test_moving_the_pointer_to_the_other_monitor_moves_the_stack(self): + p = self.pill(self.ONE) + was = (p.x, p.y) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.TWO): + p._sync_monitor() + self.assertNotEqual((p.x, p.y), was) + self.assertGreaterEqual(p.x, self.TWO[1][0]) + self.assertLessEqual(p.x + ui.PILL_W, self.TWO[1][2]) + + def test_a_pointer_that_has_not_left_the_monitor_costs_nothing(self): + # Every frame asks. Acting unconditionally would be a `geometry` call per frame + # forever — the same shape `_track_target` uses for `classify`, for the reason. + p = self.pill(self.ONE) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.ONE): + p._sync_monitor() + p.geometry.assert_not_called() + + def test_a_panel_that_is_up_is_moved_with_the_pill(self): + # The panels are placed *from* the pill, so moving it is the whole move — but + # only for a window somebody can see. + p = self.pill(self.ONE) + p.bubble = mock.Mock(_visible=True, width=ui.BUBBLE_W) + with mock.patch.object(ui, "_pointer_monitor", return_value=self.TWO): + p._sync_monitor() + p.bubble.reposition.assert_called_once() + p.card.reposition.assert_not_called() + + +class TestAHiddenPanelIsParkedRatherThanUnmapped(unittest.TestCase): + """FluidVoice's `parkWindowOffscreen`, and the one property it must have. + + The panels used to `withdraw()`. Push-to-talk made that the wrong trade: a panel now + has to be up between a key going down and somebody starting to speak, and a remap is + work done in exactly that gap. + """ + + #: Two monitors side by side, the left one primary. The union is what matters. + DESKTOP = (0, 0, 3840, 1080) + + def test_a_parked_panel_is_clear_of_every_monitor(self): + # The one thing that must never happen. Parking off the right edge of the *left* + # display in a two-monitor desk parks it in the middle of the right one, in full + # view, which is why this is the union and not the current screen. + x, y = ui.park_spot(420, 300, self.DESKTOP) + self.assertGreater(x, self.DESKTOP[2]) + self.assertGreater(y, self.DESKTOP[3]) + + def test_the_panel_clears_the_edge_by_its_own_size_as_well_as_the_margin(self): + # Its top-left corner being past the edge is not enough — the window extends + # right and down from there. + for w, h in ((205, 40), (420, 300), (640, 900)): + with self.subTest(w=w, h=h): + x, y = ui.park_spot(w, h, self.DESKTOP) + self.assertGreaterEqual(x - self.DESKTOP[2], w + ui.PARK_MARGIN) + self.assertGreaterEqual(y - self.DESKTOP[3], h + ui.PARK_MARGIN) + + def test_the_desktop_is_every_monitor_and_not_the_primary_one(self): + left, top, right, bottom = ui._virtual_desktop(1280, 720) + self.assertGreater(right, left) + self.assertGreater(bottom, top) + + def test_it_degrades_to_the_screen_rather_than_raising(self): + with mock.patch.object(ui.ctypes, "windll", ui._NoHands()): + self.assertEqual(ui._virtual_desktop(1280, 720), (0, 0, 1280, 720)) + + def test_parking_moves_the_window_and_never_unmaps_it(self): + # The distinction the whole change rests on: a `withdraw` here would cost a + # remap on the next hold, which is the gap push-to-talk has to fit inside. + win = mock.Mock(width=420, _h=300) + win.winfo_screenwidth.return_value = 1280 + win.winfo_screenheight.return_value = 720 + ui.park(win) + win.withdraw.assert_not_called() + win.geometry.assert_called_once() + asked = win.geometry.call_args[0][0] + self.assertTrue(asked.startswith("420x300+"), asked) + + def test_a_panel_with_no_height_yet_is_still_parked_somewhere_legal(self): + # `_h` is not set until the first render, and a hide can beat it — a zero-sized + # geometry request is one a window manager is entitled to refuse. + win = mock.Mock(width=420, spec=["width", "geometry", "winfo_screenwidth", + "winfo_screenheight"]) + win.winfo_screenwidth.return_value = 1280 + win.winfo_screenheight.return_value = 720 + ui.park(win) + self.assertTrue(win.geometry.call_args[0][0].startswith("420x1+")) + + class TestMix(unittest.TestCase): def test_the_ends_are_exact(self): self.assertEqual(ui._mix(ui.HEARING, ui.CARD_ACCENT, 0.0), ui.HEARING) @@ -435,6 +762,7 @@ def docker(*, showing=True, panel_w=ui.BUBBLE_W, window=None, x=1047, docked_w=u p.bubble = mock.Mock(width=panel_w, _visible=showing) p.card = mock.Mock(width=panel_w, _visible=False) p.work = (0, 0, 1280, 720) + p.full = (0, 0, 1280, 720) p.x, p.y = x, 608 p._docked_w = docked_w p.geometry = mock.Mock() @@ -450,8 +778,18 @@ class TestTheDockIsCheckedRatherThanAssumed(unittest.TestCase): 205 px pill sits at, hanging 215 px off the screen and unjoined from the panel it is docked to, held there for five seconds because the width already matched and nothing looked again. + + Pinned to `"corner"`, because holding the right edge is *corner* placement's rule + and not the dock's: the pill is anchored to the screen edge it sits against, and + growing away from it is the only direction there is. Centred placement has no + anchored edge and re-centres instead — `TestTheDockRecentresWhenThereIsNoEdge` + below is the same class of check for it. """ + def setUp(self): + self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) + ui.apply_place("corner") + def test_a_panel_appearing_moves_the_left_edge_and_holds_the_right(self): p = docker() p._sync_dock() @@ -489,5 +827,61 @@ def test_the_panel_going_away_takes_the_width_back(self): p.geometry.assert_called_once_with(f"{ui.PILL_W}x{ui.PILL_H}+1047+608") +class TestTheDockRecentresWhenThereIsNoEdge(unittest.TestCase): + """The same dock under centred placement, where holding an edge is the bug. + + Found by running the shipped placement against the dock rather than by reading it: + the pill is 205 px and the panel 420, and holding the right edge put the pair + **107 px left of centre the moment a draft appeared** — visibly, on every utterance, + because a draft appearing is the single most common thing that happens. + + Corner placement is anchored to the screen edge it sits against, so growing away + from that edge is the only direction there is. Centred placement has no anchored + edge, and the width it should be centred on is the *new* one. + """ + + def setUp(self): + self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) + ui.apply_place("bottom") + + def centre_of(self, p, w): + return p._placed(w)[0] + + def test_a_panel_appearing_recentres_on_the_new_width(self): + p = docker(x=538, docked_w=ui.PILL_W) + p._sync_dock() + self.assertEqual(p.x, self.centre_of(p, ui.BUBBLE_W)) + + def test_the_panel_going_away_recentres_on_the_pill(self): + p = docker(showing=False, docked_w=ui.BUBBLE_W, x=430) + p._sync_dock() + self.assertEqual(p.x, self.centre_of(p, ui.PILL_W)) + + def test_the_stack_stays_centred_across_an_open_and_a_close(self): + # The round trip, because an off-by-one in either direction accumulates: a pill + # that came back a few pixels off would drift across a session. + p = docker(x=538, docked_w=ui.PILL_W) + p._sync_dock() + p.bubble = mock.Mock(width=ui.BUBBLE_W, _visible=False) + p.window_geometry = mock.Mock(return_value=(ui.BUBBLE_W, p.x, 608)) + p._sync_dock() + self.assertEqual(p.x, 538) + + def test_it_is_still_clamped_onto_the_screen(self): + # The clamp survives the branch. A panel wider than the display would otherwise + # be centred to a negative x. + p = docker(x=538, docked_w=ui.PILL_W, panel_w=4000) + p._sync_dock() + self.assertGreaterEqual(p.x, p.work[0]) + + def test_corner_placement_is_untouched_by_any_of_this(self): + # The two branches are different rules, not one rule with a flag, and the + # corner's is the one people have been using. + ui.apply_place("corner") + p = docker() + p._sync_dock() + self.assertEqual(p.x + ui.BUBBLE_W, 1047 + ui.PILL_W) + + if __name__ == "__main__": unittest.main() diff --git a/tests/test_talk.py b/tests/test_talk.py new file mode 100644 index 0000000..4ae6c27 --- /dev/null +++ b/tests/test_talk.py @@ -0,0 +1,737 @@ +"""Push-to-talk end to end: a hold on the keyboard, a paste out the other side. + +`tests/test_chord.py` covers the hook's state machine and stops at the queue. This picks +the same gesture up at the queue and follows it through `Pill`'s dispatch into a real +`Session`, because everything that can actually go wrong with push-to-talk lives in the +seam between them: + + **The release cannot paste.** A final decode measured 0.7-7 s on the machine this was + built for, so the release arms a wait and the frame loop finishes the gesture. Every + edge case below is some way that wait can be wrong — the decode that never lands, the + release that never arrives, the hold that starts while another is still waiting. + + **What was said is never dropped to make the code simpler.** A hold Windows took over + with `ctrl+win+d` still commits its audio; it just does not paste. The draft is where + words wait, and the Send chip is what the user already knows. + +The microphone and the decoder are fakes, and the decode is driven by hand, so a test +can sit exactly in the half-second between a release and the final that answers it. +""" + +import queue +import sys +import time +import unittest +from pathlib import Path +from unittest import mock + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +if sys.platform != "win32": # pragma: no cover - the CI legs that are not Windows + raise unittest.SkipTest("Windows-only: flow.hotkey binds user32 at import") + +import flow.ui as ui # noqa: E402 +from flow.hotkey import Chord # noqa: E402 +from flow.session import BLOCK, Session # noqa: E402 + + +class FakeMic: + """Counts opens and closes, because the gesture is judged on them. + + `start`/`stop` are the pair push-to-talk has to get exactly right: a hold that opens + a microphone it does not close is the failure mode the whole `PTT_MAX_HOLD_SEC` + branch exists for, and one that closes a microphone it did not open is the chord + walking off with the toggle hotkey's session. + """ + + def __init__(self) -> None: + self.starts = 0 + self.stops = 0 + self.active = False + self.dropped = 0 + self.device_name = "fake" + self._blocks: list[np.ndarray] = [] + + def start(self) -> None: + self.starts += 1 + self.active = True + + def stop(self) -> None: + self.stops += 1 + self.active = False + + def drain(self) -> list: + out, self._blocks = self._blocks, [] + return out + + def alive(self) -> bool: + return self.active + + +class FakeAsr: + loading = False + loaded = False + + def load(self, final=None) -> None: + self.loaded = True + + def unload(self) -> None: + self.loaded = False + + def text(self, audio, *, final=False, hotwords="") -> str: + return "" + + +class Harness: + """A `Pill` with the four attributes push-to-talk touches, and nothing drawn. + + Built with `__new__` for `tests/test_lite.py`'s reason, word for word: `tk.Misc` + forwards an unknown attribute to `self.tk`, so on an instance whose `__init__` never + ran a missing attribute recurses instead of defaulting. Every attribute the paths + under test read is set here on purpose. + """ + + def __init__(self, *, lite: bool = False) -> None: + self.mic = FakeMic() + # `Pill.converse` is a property reading `session.mode`, so the surface follows + # the session here exactly as it does in the app — there is nothing to set. + self.session = Session(asr=FakeAsr(), mic=self.mic) + self.pasted: list[tuple[str, bool]] = [] + self.notes: list[str] = [] + + p = ui.Pill.__new__(ui.Pill) + p.session = self.session + p.lite = lite + + p.on_send = self._on_send + p.paste_target = 0x22 + p.bubble = mock.Mock() + p.card = mock.Mock() + p.bubble.note.side_effect = self.notes.append + p.card.note.side_effect = self.notes.append + p.armed = False + p._flash = 0 + p._disarmed_since = 0.0 + p._ptt_since = None + p._ptt_wait = None + p._draw = lambda: None + p.clipboard_clear = lambda: None + p.clipboard_append = lambda _t: None + p.update_idletasks = lambda: None + self.pill = p + + self.presses: queue.Queue[str] = queue.Queue() + self.chord = Chord(self.presses, frozenset({"ctrl", "win"})) + + def _on_send(self, text: str, target=None, submit: bool = False) -> str: + self.pasted.append((text, submit)) + return "" + + # -- driving ----------------------------------------------------------- + + def dispatch(self) -> None: + """Drain the chord's queue through `Pill`'s own dispatch table. + + The words are re-listed here rather than calling `_frame`, which would want a + Tk canvas. The tripwire in `test_chord.py` is what keeps this list honest: it + asserts each of the four words appears in `ui.py` as a dispatch literal. + """ + while not self.presses.empty(): + name = self.presses.get_nowait() + if name == "warm": + self.session.warm() + elif name == "talk": + self.pill._talk_start() + elif name == "talk-end": + self.pill._talk_end(send=True) + elif name == "talk-break": + self.pill._talk_end(send=False) + elif name == "send": + self.pill._send() + elif name == "cancel": + self.pill._clear() + elif name == "mode": + self.pill._ptt_wait = None + self.session.toggle_mode() + elif name == "toggle": + self.pill._toggle() + self.pump_events() + + def key(self, name: str) -> "Harness": + """Press one of the registered hotkeys, through the same dispatch the chord uses. + + The five combos and the chord's four words land on one queue by design — the + session cannot tell them apart, and neither should a test of what happens when + two of them mean overlapping things. + """ + self.presses.put(name) + self.dispatch() + return self + + def pump_events(self) -> None: + """The one event this gesture depends on, handled the way `_pump_events` does. + + `talk_end` closes the microphone and *asks* the pill to disarm rather than + reaching into it: `armed` belongs to the UI thread. In the app the real + `_pump_events` runs in the same frame as the dispatch, so the pill is never seen + claiming to listen with a closed device — reproducing that ordering here is what + makes this harness worth trusting. + """ + for ev in self.session.events(): + if ev.kind == "disarm": + self.pill.armed = False + + def settle(self, timeout: float = 5.0) -> "Harness": + """Wait for the real decode worker to finish what the release handed it. + + The worker is a live thread even with a fake transcriber, so `session.busy` is + genuinely true for a moment after a release. Waiting rather than patching keeps + the ordering under test real: the paste must follow the final, and a test that + stubbed `busy` to False could not tell the difference. + """ + end = time.perf_counter() + timeout + while self.session.busy and time.perf_counter() < end: + time.sleep(0.005) + self.session.pump_results() + self.pump_events() + return self + + def press(self) -> "Harness": + self.chord._talking = True + self.presses.put("warm") + self.presses.put("talk") + self.dispatch() + return self + + def release(self, *, clean: bool = True) -> "Harness": + self.presses.put("talk-end" if clean else "talk-break") + self.dispatch() + return self + + def speak(self, seconds: float = 1.0) -> "Harness": + """Put audio in the utterance buffer the way `_pump_audio` would.""" + blocks = max(1, int(seconds * 16000 / BLOCK)) + self.session._utter = [np.zeros(BLOCK, dtype=np.float32)] * blocks + return self + + def decode(self, text: str) -> "Harness": + """Land a final decode: let the worker drain, then put the words on the draft.""" + self.settle() + self.session.draft.set(text) + return self + + def frame(self) -> "Harness": + self.pill._pump_talk() + return self + + +class TestTheHoldOpensAndTheReleaseCloses(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_press_opens_the_microphone_and_the_release_closes_it(self): + self.h.press() + self.assertTrue(self.h.mic.active) + self.assertTrue(self.h.pill.armed) + self.h.speak().release() + self.assertFalse(self.h.mic.active) + + def test_the_press_warms_before_it_captures(self): + # The reason the warm is a separate word. By the time capture is open the models + # have been asked for, so the load overlaps the utterance instead of following + # it — which is what the 1 230 ms first partial in the trace was. + self.assertFalse(self.h.session.asr.loaded) + self.h.press() + self.assertTrue(self.h.session.asr.loaded) + + def test_a_second_press_does_not_reopen_a_microphone_already_open(self): + # Key repeat, or a `talk` that raced a slow frame. Reopening mid-utterance would + # be indistinguishable to the user from Flow having lost what they just said. + self.h.press() + self.h.press() + self.assertEqual(self.h.mic.starts, 1) + + def test_a_release_with_no_hold_behind_it_does_nothing(self): + # The OS can deliver a keyup whose keydown this process never saw — a chord + # begun before Flow launched, or while a UAC prompt owned the input desktop. + # It must not close a microphone the toggle hotkey opened. + self.h.session.start() + self.assertTrue(self.h.mic.active) + self.h.release() + self.assertTrue(self.h.mic.active) + + +class TestTheChordGivesBackExactlyWhatItTook(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_a_hold_over_an_already_armed_session_leaves_it_armed(self): + # Somebody armed with the toggle hotkey for long-form dictation, then reached + # for the chord. The release sends what they said; it does not switch off the + # microphone they turned on by another route. + self.h.session.start() + opens = self.h.mic.starts + self.h.press() + self.assertEqual(self.h.mic.starts, opens) + self.h.speak().release() + self.assertTrue(self.h.mic.active) + self.assertEqual(self.h.mic.stops, 0) + + def test_and_a_hold_that_opened_it_closes_it(self): + self.h.press().speak().release() + self.assertEqual(self.h.mic.stops, 1) + self.assertFalse(self.h.pill.armed) + + +class TestWhatWasSaidSurvivesEveryPath(unittest.TestCase): + """P2, on the one path with the most reason to cut a corner. + + `pause()` bumps the capture generation, which is precisely how a deliberate stop + refuses a decode from before it — so a `talk_end` written with `pause()` would throw + away the utterance the release exists to send. `_give_up_on_device` learned this + first and the comment there is the longer version. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_release_does_not_refuse_the_decode_it_just_asked_for(self): + before = self.h.session._capture_generation + self.h.press().speak().release() + self.assertEqual(self.h.session._capture_generation, before) + + def test_the_utterance_reaches_the_decoder(self): + self.h.press().speak(2.0) + self.assertTrue(self.h.session._utter) + self.h.release() + self.assertFalse(self.h.session._utter) # committed, not abandoned + self.assertTrue(self.h.session._sent) + + def test_a_broken_hold_keeps_the_words_and_pastes_nothing(self): + # ctrl+win+d after somebody had already started talking. The desktop switch is + # not a reason to lose a sentence, and it is not a reason to paste one into the + # window the switch just moved to either. + self.h.press().speak(2.0).release(clean=False) + self.assertTrue(self.h.session._sent) + self.h.decode("what they said").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "what they said") + + def test_a_desktop_switch_nobody_spoke_into_is_invisible(self): + # The common case, and the one that must not produce a note. `_utter` is what + # the gate let through, and in the 50 ms before the third key that is nothing — + # so there is no minimum-length rule here, and nothing to say. + self.h.press().release(clean=False) + self.assertFalse(self.h.session._sent) + self.assertEqual(self.h.notes, []) + self.assertEqual(self.h.pasted, []) + + +class TestThePasteWaitsForTheDecode(unittest.TestCase): + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_nothing_is_pasted_while_the_decoder_is_still_working(self): + # The bug this ordering prevents is pasting the *partial* — the dimmed, italic, + # possibly-hallucinated text that precedes a final by design. + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.decode("half a sentence").frame() + self.assertEqual(self.h.pasted, []) + + def test_and_it_pastes_the_moment_the_final_lands(self): + self.h.press().speak().release() + self.h.decode("the whole sentence").frame() + self.assertEqual(self.h.pasted, [("the whole sentence", False)]) + + def test_a_hold_nobody_spoke_into_never_arms_a_wait(self): + # An accidental tap. Without this the next decode from any source would be + # pasted by a wait that had been sitting open since the tap. + self.h.press().release() + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("something said later").frame() + self.assertEqual(self.h.pasted, []) + + def test_an_empty_decode_pastes_nothing(self): + # Silence, noise, or a hallucination `clean.py` rejected. Pasting "" would clear + # a selection in the user's window for no reason. + self.h.press().speak().release() + self.h.decode("").frame() + self.assertEqual(self.h.pasted, []) + + def test_the_wait_is_disarmed_once_it_fires(self): + # Otherwise the next thing decoded — a partial from a later utterance, an + # answer — would be pasted again by a wait nobody rearmed. + self.h.press().speak().release() + self.h.decode("first").frame() + self.h.decode("second").frame() + self.assertEqual(self.h.pasted, [("first", False)]) + + def test_a_new_hold_supersedes_a_wait_that_is_still_open(self): + # Somebody released, got impatient, and pressed again. The old wait must not + # fire into the new utterance — and the words it was waiting for stay in the + # draft rather than being discarded. + self.h.press().speak().release() + self.h.press() + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("from the first hold").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "from the first hold") + + +class TestTheTwoTimeoutsAreNotHypothetical(unittest.TestCase): + """Both branches of `_pump_talk`, which exist because this app has already wedged. + + The trace from the night this was written ends with `state -> idle` and no `final` + behind it. A paste wait with no ceiling turns that into a gesture that never + completes; a hold with no ceiling turns a dropped keyup into a microphone left open. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_a_decode_that_never_lands_gives_up_and_says_where_the_words_are(self): + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.pill._ptt_wait -= ui.PTT_PASTE_WAIT_SEC + 1 + self.h.frame() + self.assertEqual(self.h.pasted, []) + self.assertIsNone(self.h.pill._ptt_wait) + self.assertIn("Press Send", " ".join(self.h.notes)) + + def test_and_does_not_paste_it_late_when_it_finally_arrives(self): + # A paste a minute after the gesture lands in whatever window the user has moved + # to since, which is worse than not pasting at all. + self.h.press().speak().release() + with mock.patch.object(type(self.h.session), "busy", + property(lambda _s: True)): + self.h.pill._ptt_wait -= ui.PTT_PASTE_WAIT_SEC + 1 + self.h.frame() + self.h.decode("very late").frame() + self.assertEqual(self.h.pasted, []) + + def test_a_hold_whose_release_never_arrives_stops_on_its_own(self): + # A hook the OS dropped for overrunning `LowLevelHooksTimeout`, a lock screen, an + # RDP session taking the keyboard. Without this the microphone stays open. + self.h.press().speak(2.0) + self.h.pill._ptt_since -= ui.PTT_MAX_HOLD_SEC + 1 + self.h.frame() + self.assertFalse(self.h.mic.active) + self.assertIsNone(self.h.pill._ptt_since) + + def test_and_keeps_what_was_said_rather_than_pasting_it(self): + self.h.press().speak(2.0) + self.h.pill._ptt_since -= ui.PTT_MAX_HOLD_SEC + 1 + self.h.frame() + self.assertTrue(self.h.session._sent) + self.h.decode("a long dictation").frame() + self.assertEqual(self.h.pasted, []) + self.assertIn("press Send", " ".join(self.h.notes)) + + def test_an_ordinary_hold_is_nowhere_near_the_ceiling(self): + # A guard on the number rather than on the branch: two minutes has to sit clear + # of the longest hold anybody would make on purpose. + self.h.press().speak(2.0) + self.h.frame() + self.assertTrue(self.h.mic.active) + self.assertIsNotNone(self.h.pill._ptt_since) + + +class TestOneHoldOwnsOneSend(unittest.TestCase): + """The collisions, which are the part of push-to-talk that is genuinely hard. + + There are four ways to send — the Send chip, the `send` hotkey, the spoken trigger + routed as a `send` event, and converse's auto-ask countdown — and a release that has + armed a paste is a fifth thing waiting to do the same job. Any two of them firing for + one utterance is a double paste into somebody's editor. + + The rule is that a hold owns *one* send and whoever gets there first has it, and it + is enforced at the single point all five have in common rather than at five guards + that would have to be kept in step. These are the tests that say so. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + + def test_the_spoken_send_word_does_not_paste_twice(self): + # The one the whole rule exists for. Hold the chord, say "…and that's the plan, + # boom", let go: the trigger fires a send when the decode routes, while the + # release is still waiting to fire its own. + self.h.press().speak().release() + self.h.decode("that's the plan") + self.h.pill._send() # the trigger, arriving as a `send` event + self.h.frame() # the wait, arriving one line later + self.assertEqual(len(self.h.pasted), 1) + + def test_the_send_hotkey_during_the_wait_does_not_paste_twice(self): + # Somebody who does not yet trust the release and reaches for ctrl+alt+enter. + self.h.press().speak().release() + self.h.decode("said once") + self.h.pill._send() + self.h.frame() + self.assertEqual(self.h.pasted, [("said once", False)]) + + def test_whoever_gets_there_first_has_it_and_the_wait_stands_down(self): + self.h.press().speak().release() + self.assertIsNotNone(self.h.pill._ptt_wait) + self.h.decode("first past the post") + self.h.pill._send() + self.assertIsNone(self.h.pill._ptt_wait) + + def test_clearing_the_draft_cancels_a_paste_that_has_not_landed(self): + # The nastiest one to leave running: the draft is cleared, the decode lands a + # second later and refills it, and the wait pastes into the user's window the + # words they just pressed a key to stop. Clear means clear. + self.h.press().speak().release() + self.h.pill._clear() + self.h.decode("what they cancelled").frame() + self.assertEqual(self.h.pasted, []) + + def test_switching_mode_cancels_it_rather_than_translating_it(self): + # A wait armed in dictate pastes into a window; fired in converse it would ask a + # CLI. The switch is one keypress away at all times. Driven through the `mode` + # dispatch rather than by clearing the flag here, which would be a test asserting + # what it had just done itself. + self.h.press().speak().release() + self.assertIsNotNone(self.h.pill._ptt_wait) + self.h.key("mode") + self.assertIsNone(self.h.pill._ptt_wait) + self.h.decode("asked, not pasted").frame() + self.assertEqual(self.h.pasted, []) + + def test_the_toggle_hotkey_mid_hold_keeps_the_words(self): + # `pause()` bumps the capture generation, which is how a deliberate stop refuses + # a decode from before it — and the utterance being spoken *right now* is exactly + # what that would refuse. The hold is ended first, so it is committed. + self.h.press().speak(2.0) + before = self.h.session._capture_generation + self.h.pill._toggle() + self.assertEqual(self.h.session._capture_generation, before) + self.assertTrue(self.h.session._sent) + + def test_and_does_not_paste_them_because_a_toggle_is_not_a_release(self): + # The user reached for the other control mid-sentence. Pasting on their behalf + # is not what either gesture asked for. + self.h.press().speak(2.0) + self.h.pill._toggle() + self.h.decode("mid sentence").frame() + self.assertEqual(self.h.pasted, []) + self.assertEqual(self.h.session.draft.text, "mid sentence") + + def test_the_toggle_does_not_then_pause_the_session_it_just_stopped(self): + # The `return` in `_toggle`. Falling through would do the generation bump the + # branch above exists to avoid. + self.h.press().speak(2.0) + self.h.pill._toggle() + self.assertIsNone(self.h.pill._ptt_since) + self.assertFalse(self.h.mic.active) + + def test_the_mode_switch_really_does_clear_it_in_the_app(self): + # `Harness.dispatch` mirrors `Pill._frame`, so the test above could pass on the + # mirror alone. This is the tripwire on the original — the same shape + # `test_chord.py` uses for the four chord words, and for the same reason. + ui_src = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + branch = ui_src[ui_src.index('elif name == "mode":'):] + branch = branch[:branch.index('elif name == "quit"')] + self.assertIn("_ptt_wait = None", branch) + + def test_send_and_clear_clear_it_at_their_own_single_choke_point(self): + # Same tripwire, for the two methods every other send and stop path funnels + # through. One line in each is what makes four collisions impossible rather + # than four guards that would have to be kept in step. + ui_src = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + for name in ("_send", "_clear"): + with self.subTest(method=name): + body = ui_src[ui_src.index(f" def {name}(self"):] + body = body[:body.index("\n def ", 10)] + self.assertIn("self._ptt_wait = None", body) + + def test_a_hold_is_refused_while_the_hand_editor_is_open(self): + # `_pump_audio` throws away every block while `editing` is true, so a hold here + # would open the microphone, capture nothing, and end with no paste and no + # explanation — the silent deafness invariant 4 forbids. + self.h.session.editing = True + self.h.press() + self.assertIsNone(self.h.pill._ptt_since) + self.assertEqual(self.h.mic.starts, 0) + self.assertIn("editing", " ".join(self.h.notes)) + + +class TestHoldingThePillIsPushToTalkWithoutAHotkey(unittest.TestCase): + """The gesture for every platform that has no chord — which is every platform but one. + + `Chord` is a `WH_KEYBOARD_LL` hook and there is no such thing off Windows, so Flow + Lite has never had push-to-talk. It does not need a hotkey to: the button can be a + window Flow already draws, which costs no Accessibility permission, no Input + Monitoring, and no signed bundle to ask for them from. That is the one thing Lite + can do that a native app driving a system hotkey cannot. + + Three gestures now share the left button, and each test below is one of them, plus + the bug that fell out of fixing the arrangement. + """ + + def setUp(self): + self.h = Harness() + self.addCleanup(self.h.session.close) + self.p = self.h.pill + self.timers = [] + self.p.after = lambda ms, fn: self.timers.append(fn) or len(self.timers) + self.p.after_cancel = lambda tid: self.timers.__setitem__(tid - 1, None) + self.p._toggle = mock.Mock(name="toggle") + + def at(self, x=0, y=0): + return mock.Mock(x_root=x, y_root=y) + + def hold(self): + """Let the pending hold timer fire, as Tk would after `PILL_HOLD_SEC`.""" + for fn in self.timers: + if fn is not None: + fn() + + def test_a_quick_click_still_toggles(self): + # The gesture that was there first, and the one muscle memory depends on. + self.p._on_press(self.at()) + self.p._on_release() + self.p._toggle.assert_called_once() + self.assertEqual(self.h.mic.starts, 0) + + def test_holding_it_starts_capturing_before_the_button_comes_up(self): + # The whole difference between this and a long-click: capture has to start while + # the user is still holding, because the hold *is* the utterance. Waiting for + # the release to notice would record nothing at all. + self.p._on_press(self.at()) + self.hold() + self.assertTrue(self.h.mic.active) + self.assertIsNotNone(self.p._ptt_since) + + def test_and_releasing_sends_what_was_said(self): + self.p._on_press(self.at()) + self.hold() + self.h.speak() + self.p._on_release() + self.h.decode("held the pill and spoke").frame() + self.assertEqual(self.h.pasted, [("held the pill and spoke", False)]) + self.p._toggle.assert_not_called() + + def test_a_hold_never_also_toggles(self): + # Both gestures on one button, so the release has to pick exactly one. + self.p._on_press(self.at()) + self.hold() + self.h.speak() + self.p._on_release() + self.p._toggle.assert_not_called() + + def test_dragging_the_pill_no_longer_toggles_listening(self): + # The bug this arrangement fixes, and it predates hold-to-talk: `_toggle` was + # bound to ``, which in Tk is the *press* — so every drag of the pill + # armed or disarmed capture on the way past. + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(60, 0)) + self.p._on_release() + self.p._toggle.assert_not_called() + + def test_a_drag_does_not_start_an_utterance_either(self): + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(60, 0)) + self.hold() + self.assertEqual(self.h.mic.starts, 0) + self.assertIsNone(self.p._ptt_since) + + def test_a_hand_that_is_merely_not_still_is_not_a_drag(self): + # `PILL_DRAG_SLOP` exists because a hand resting on a mouse trembles, and a hold + # that lost its nerve on one pixel would be a gesture nobody could rely on. + self.p._on_press(self.at(0, 0)) + self.p._on_motion(self.at(ui.PILL_DRAG_SLOP, ui.PILL_DRAG_SLOP)) + self.hold() + self.assertTrue(self.h.mic.active) + + def test_moving_while_talking_does_not_cancel_the_utterance(self): + # Once capture is open the pointer is irrelevant: somebody talking into a held + # pill may well move the mouse, and cancelling their sentence for it would be + # the gesture betraying them. + self.p._on_press(self.at(0, 0)) + self.hold() + self.p._on_motion(self.at(400, 400)) + self.assertTrue(self.h.mic.active) + self.h.speak() + self.p._on_release() + self.h.decode("still mine").frame() + self.assertEqual(self.h.pasted, [("still mine", False)]) + + def test_a_click_cancels_the_pending_hold_rather_than_leaving_it_armed(self): + # Otherwise the timer fires after the button is already up and opens a + # microphone nobody is holding — the exact failure `PTT_MAX_HOLD_SEC` exists to + # catch, arriving by a route it should never have to. + self.p._on_press(self.at()) + self.p._on_release() + self.hold() + self.assertEqual(self.h.mic.starts, 0) + + def test_it_works_in_lite_where_there_is_no_chord_at_all(self): + # The point of the whole gesture. Lite copies instead of pasting — `_send` + # already knows — so the hold ends on the clipboard with nothing granted but + # the microphone. + h = Harness(lite=True) + self.addCleanup(h.session.close) + timers = [] + h.pill.after = lambda ms, fn: timers.append(fn) or len(timers) + h.pill.after_cancel = lambda tid: timers.__setitem__(tid - 1, None) + h.pill._on_press(mock.Mock(x_root=0, y_root=0)) + for fn in timers: + if fn is not None: + fn() + self.assertTrue(h.mic.active) + h.speak() + h.pill._on_release() + h.decode("onto the clipboard").frame() + self.assertEqual(h.pasted, []) # copied, not pasted + + +class TestItFailsTheWayTheRestOfTheSurfaceDoes(unittest.TestCase): + def test_a_microphone_that_will_not_open_leaves_the_pill_disarmed(self): + # `_toggle` already refuses to show a green pill over a dead capture, and the + # chord must tell the same truth — with the added stake that under push-to-talk + # the user is about to speak into it. + h = Harness() + self.addCleanup(h.session.close) + with mock.patch.object(h.mic, "start", side_effect=OSError("device in use")): + h.press() + self.assertFalse(h.pill.armed) + self.assertIsNone(h.pill._ptt_since) + self.assertTrue(h.pill._flash) + + def test_and_the_release_afterwards_is_a_no_op(self): + # There is no hold to end. Without the `_ptt_since` guard this would commit an + # empty utterance and stop a microphone that never started. + h = Harness() + self.addCleanup(h.session.close) + with mock.patch.object(h.mic, "start", side_effect=OSError("device in use")): + h.press() + h.release() + self.assertEqual(h.mic.stops, 0) + + def test_lite_copies_instead_of_pasting(self): + # The gesture is the same; where the words go is `_send`'s business, and it + # already knows. Asserted so the chord does not grow its own idea of Lite. + h = Harness(lite=True) + self.addCleanup(h.session.close) + h.press().speak().release() + h.decode("into the clipboard").frame() + self.assertEqual(h.pasted, []) + + +if __name__ == "__main__": # pragma: no cover + unittest.main(verbosity=2) From 7d6f07563bff75ef03ca093b74f2a480739efad9 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 20:59:34 -0500 Subject: [PATCH 02/41] Give Aqua a window class, and measure the work area instead of assuming it Two things a Mac reported in one breath: the pill had close and maximize buttons on it, and it sat under the Dock. The frame is Aqua's window class, not the redirect flag. `overrideredirect(True)` strips the title bar on Windows and X11, but Tk maps a Toplevel to an NSWindow whose style mask comes from the window class, so on a Mac it did nothing visible. `MacWindowStyle plain noActivates` is the supported way to ask, and it carries the second half too: `noActivates` is Aqua's WS_EX_NOACTIVATE, without which clicking the pill pulls focus off whatever is being dictated into - the defect that made every paste land in the wrong window on Windows, already fixed once and no reason to ship again. The Dock is a work area Flow was not asking for. `_work_area` degrades to the whole screen off Windows, so bottom-centre placement stood the stack on the very bottom edge - behind the Dock on macOS, behind the panel on a bottom-taskbar Linux. `wm_maxsize` looked like the answer and is useless: on Windows it reports the full screen even with a taskbar present, wrong in exactly the way this is meant to fix. What works is maximising a transparent probe window and reading back where the window manager put it, since it has to honour its own panels to do that - exact against SystemParametersInfoW on left, right and bottom. Measured once and cached, because `_sync_monitor` asks every frame. `top` is taken as reported and is a title bar low. `winfo_rooty` is the client area and the frame inset differs between a normal window and a maximised one - measuring the inset and subtracting it made the answer worse, not better (-8 against a true 0). It feeds one thing, the ceiling in `bottom_centre`, where being conservative costs nothing. scripts/mac_probe.py prints every number Tk will give for this, next to what Flow computes from them, so the next platform question is a measurement rather than a third guess. It is what found the `wm_maxsize` dead end. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 110 +++++++++++++++++++++++++++++++++++++-- scripts/mac_probe.py | 118 +++++++++++++++++++++++++++++++++++++++++ tests/test_pill.py | 121 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 345 insertions(+), 4 deletions(-) create mode 100644 scripts/mac_probe.py diff --git a/flow/ui.py b/flow/ui.py index 1d0b12f..57b5edd 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -154,6 +154,35 @@ def toplevel_hwnd(win) -> int: return _user32.GetParent(win.winfo_id()) or 0 +def _mac_window_style(win) -> bool: + """Aqua's answer to `overrideredirect` + `WS_EX_NOACTIVATE`, in one call. + + `overrideredirect(True)` is what strips the title bar on Windows and X11. On Aqua it + is not enough on its own — reported from a real Mac as a pill sitting there with its + close and maximize buttons showing — because Tk maps a Toplevel to a real + `NSWindow` whose style mask is set from the *window class*, not from the redirect + flag. `MacWindowStyle` is the supported way to choose that class, and `plain` is the + one with no frame at all. + + `noActivates` is the second half and the more important one: it is Aqua's + `WS_EX_NOACTIVATE`. Without it, clicking the pill pulls focus away from whatever the + user is dictating into — which on Windows is the defect that made every paste land + in the wrong window, and there is no reason to ship it on a Mac having fixed it once + already. + + Named `::tk::unsupported::` by Tk itself, which is why this is wrapped and reported + rather than trusted: it is the interface Aqua actually offers and it may not be + there. A False here costs a title bar, not a session. + """ + try: + win.update_idletasks() + win.tk.call("::tk::unsupported::MacWindowStyle", "style", + win._w, "plain", "noActivates") + return True + except tk.TclError: + return False + + def _no_activate(win) -> bool: """Take `win` out of the activation chain, and report whether it took. @@ -161,7 +190,14 @@ def _no_activate(win) -> bool: so a call that did nothing and a call that worked hand back the same plausible number, and there is no other way to tell them apart. The one thing this window style has to be is true. + + Off Windows this is `_mac_window_style`'s job, which does both halves at once + because Aqua sets the frame and the activation policy from the same window class. """ + if sys.platform == "darwin": + return _mac_window_style(win) + if sys.platform != "win32": + return False try: # The wrapper has to exist before it can be styled, and this is what creates it. win.update_idletasks() @@ -214,6 +250,69 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: return rect.left, rect.top, rect.right, rect.bottom +#: `_tk_work_area`'s answer, measured once. A module global because the measurement +#: costs a window and `_sync_monitor` asks every frame — measuring per frame would open +#: and destroy a Toplevel thirty times a second. +_TK_WORK: tuple | None = None + + +def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: + """The usable area, measured by asking the window manager to maximise something. + + Reported from a Mac: the pill sat under the Dock. `_work_area` degrades to the whole + screen off Windows, so bottom-centre placement stood the stack on the very bottom + edge — behind the Dock on a default macOS desktop, behind the panel on a + bottom-taskbar Linux. + + **Measured rather than asked, because asking does not work.** `wm_maxsize` is the + obvious call and it is useless: on Windows it answers with the whole screen even + with a taskbar present, so a fallback built on it would have been wrong in exactly + the way it was meant to fix. What *is* reliable is maximising a window and looking + at where the window manager put it — it has to honour its own panels to do that. + Checked against `SystemParametersInfoW` on Windows, where the two agree exactly on + left, right and bottom. + + The probe is transparent while it is measured, so nothing flashes on screen. + + **`top` is taken as reported and is a title bar too low.** `winfo_rooty` is the + client area, and the frame inset differs between a normal window and a maximised one + — measuring the inset first and subtracting it made the answer worse, not better + (−8 against a true 0). It is left alone because `top` feeds one thing, the ceiling + in `bottom_centre`, where being conservative by a title bar costs nothing. The three + edges that place the stack are exact. + """ + global _TK_WORK + if _TK_WORK is not None: + return _TK_WORK + fallback = (0, 0, sw, sh) + probe = None + try: + probe = tk.Toplevel(win) + probe.attributes("-alpha", 0.0) + probe.geometry("200x120+80+80") + probe.state("zoomed") + probe.update_idletasks() + x, y = probe.winfo_rootx(), probe.winfo_rooty() + w, h = probe.winfo_width(), probe.winfo_height() + found = (x, y, x + w, y + h) + except (tk.TclError, AttributeError, ValueError): + # `zoomed` is documented for Windows and X11 and may not exist on this build. + # The whole screen is the honest answer then: wrong by a Dock, rather than + # wrong by whatever a broken measurement returned. + found = fallback + finally: + if probe is not None: + try: + probe.destroy() + except tk.TclError: + pass + if not (found[2] > found[0] and found[3] > found[1] + and found[2] - found[0] <= sw and found[3] - found[1] <= sh): + found = fallback + _TK_WORK = found + return found + + #: `MonitorFromPoint`'s "nearest monitor" flag, for a cursor that is briefly nowhere — #: between two displays, or on a monitor that has just been unplugged. _MONITOR_DEFAULTTONEAREST = 2 @@ -228,7 +327,7 @@ class _MONITORINFO(ctypes.Structure): ] -def _pointer_monitor(sw: int, sh: int) -> tuple[tuple, tuple]: +def _pointer_monitor(sw: int, sh: int, win=None) -> tuple[tuple, tuple]: """`(full, work)` for the monitor under the mouse, each `(left, top, right, bottom)`. **Two rectangles, because FluidVoice places against two.** `positionWindow` centres @@ -264,7 +363,10 @@ def _pointer_monitor(sw: int, sh: int) -> tuple[tuple, tuple]: return full, work except (AttributeError, OSError): pass - work = _work_area(sw, sh) + # Off Windows there is no `MonitorFromPoint` and no `rcWork`, so the two rectangles + # collapse into the one Tk can answer for. `win` is optional because the fallback + # has to keep working for the callers that have no window yet. + work = _tk_work_area(win, sw, sh) if win is not None else _work_area(sw, sh) return work, work @@ -1445,7 +1547,7 @@ def __init__( #: places against — `full` for centring, `work` for standing on. Refreshed from #: the pointer's monitor in `_sync_monitor`; see `_pointer_monitor`. self.full, self.work = _pointer_monitor( - self.winfo_screenwidth(), self.winfo_screenheight()) + self.winfo_screenwidth(), self.winfo_screenheight(), self) self.x, self.y = self._placed(PILL_W) self.geometry(f"{PILL_W}x{PILL_H}+{self.x}+{self.y}") #: The width last drawn, so `_sync_dock` can tell whether a panel appeared or @@ -2812,7 +2914,7 @@ def _sync_monitor(self) -> None: work unconditionally would be a `geometry` call per frame forever. """ full, work = _pointer_monitor( - self.winfo_screenwidth(), self.winfo_screenheight()) + self.winfo_screenwidth(), self.winfo_screenheight(), self) if (full, work) == (self.full, self.work): return self.full, self.work = full, work diff --git a/scripts/mac_probe.py b/scripts/mac_probe.py new file mode 100644 index 0000000..4ee5ded --- /dev/null +++ b/scripts/mac_probe.py @@ -0,0 +1,118 @@ +"""Ask a Mac the two questions Windows answers with one API call, and print what it says. + +Flow places its windows against a *work area* — the desktop minus whatever the OS keeps +for itself. On Windows that is one call (`SystemParametersInfoW(SPI_GETWORKAREA)`) and +one more for the monitor under the pointer. Off Windows there is neither, so `ui.py` +falls back to what Tk will admit to, and the first Mac to run it put the pill under the +Dock. + +This exists so the fix is a measurement rather than a third guess. It prints every +number Tk can be asked for, next to what Flow currently computes from them, and it also +checks the two window attributes the pill depends on — the borderless frame and the +no-activate policy — because a Mac reported those wrong in the same breath. + + uv run python scripts/mac_probe.py + +Nothing is opened that the user has to close: the probe window is withdrawn before it +would be seen, except for the two seconds the visible check needs. +""" + +import sys +import tkinter as tk +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +import flow.ui as ui # noqa: E402 + + +def row(label: str, value) -> None: + print(f" {label:<34} {value}") + + +def main() -> None: + root = tk.Tk() + root.withdraw() + sw, sh = root.winfo_screenwidth(), root.winfo_screenheight() + + print(f"platform: {sys.platform} tk {tk.TkVersion}") + print("\nwhat Tk says about the screen") + row("winfo_screenwidth/height", f"{sw} x {sh}") + try: + row("wm_maxsize()", root.wm_maxsize()) + except Exception as exc: + row("wm_maxsize()", f"FAILED: {exc}") + row("winfo_vrootwidth/height", + f"{root.winfo_vrootwidth()} x {root.winfo_vrootheight()}") + row("winfo_vrootx/y", f"{root.winfo_vrootx()}, {root.winfo_vrooty()}") + row("winfo_fpixels('1i') (dpi)", root.winfo_fpixels("1i")) + + # The measurement that would settle it: zoom a window and read where it landed. + # Supported on Windows and, depending on the build, on Aqua. If it works, its + # numbers are the work area *including* the origin, which `wm_maxsize` cannot give. + print("\nthe measurement that would settle it") + probe = tk.Toplevel(root) + probe.geometry("200x120+80+80") + try: + probe.state("zoomed") + probe.update_idletasks() + x, y = probe.winfo_rootx(), probe.winfo_rooty() + w, h = probe.winfo_width(), probe.winfo_height() + row("zoomed geometry (l,t,r,b)", (x, y, x + w, y + h)) + except Exception as exc: + row("state('zoomed')", f"UNSUPPORTED: {exc}") + # Asking for a position past the bottom edge: a window manager that clamps reveals + # the bottom of the usable area by where it puts the window instead. + try: + probe.state("normal") + probe.geometry(f"200x120+40+{sh + 500}") + probe.update_idletasks() + row("asked y=%d, landed at" % (sh + 500), probe.winfo_rooty()) + except Exception as exc: + row("clamp test", f"FAILED: {exc}") + probe.destroy() + + print("\nwhat Flow computes from the above") + row("_work_area()", ui._work_area(sw, sh)) + row("_tk_work_area()", ui._tk_work_area(root, sw, sh)) + full, work = ui._pointer_monitor(sw, sh, root) + row("_pointer_monitor() full", full) + row("_pointer_monitor() work", work) + row("pill would be placed at", + ui.bottom_centre(ui.PILL_W, ui.PILL_H, full, work, ui.PANEL_BOTTOM_OFFSET)) + row("a 420x300 panel at", + ui.bottom_centre(ui.BUBBLE_W, 300, full, work, ui.PANEL_BOTTOM_OFFSET)) + row("hidden panels park at", ui.park_spot(ui.BUBBLE_W, 300, + ui._virtual_desktop(sw, sh))) + + print("\nthe two window attributes the pill depends on") + win = tk.Toplevel(root) + win.geometry("260x60+60+60") + ok = [] + for label, call in ( + ("overrideredirect(True)", lambda: win.overrideredirect(True)), + ("-topmost", lambda: win.attributes("-topmost", True)), + ("-alpha 0.94", lambda: win.attributes("-alpha", 0.94)), + ): + try: + call() + win.update() + row(label, "OK") + except tk.TclError as exc: + row(label, f"FAILED: {exc}") + row("MacWindowStyle plain/noActivates", + "OK" if ui._mac_window_style(win) else "FAILED (or not a Mac)") + + tk.Label(win, text="Any title bar on this? (2s)", bg="#101216", + fg="#e6e8ee").pack(fill="both", expand=True) + win.update() + row("visible for 2s at", (win.winfo_rootx(), win.winfo_rooty())) + win.after(2000, root.destroy) + root.mainloop() + print("\nPaste all of the above back. The line that matters most is whether the") + print("window above had a title bar, and what `zoomed geometry` says.") + print(f"(unused: {ok})" if ok else "") + + +if __name__ == "__main__": + main() diff --git a/tests/test_pill.py b/tests/test_pill.py index 648254d..efa5538 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -737,6 +737,127 @@ def test_a_panel_with_no_height_yet_is_still_parked_somewhere_legal(self): self.assertTrue(win.geometry.call_args[0][0].startswith("420x1+")) +class TestTheWorkAreaOffWindows(unittest.TestCase): + """`_tk_work_area`, which exists because a Mac put the pill under the Dock. + + `_work_area` degrades to the whole screen off Windows, so bottom-centre placement + stood the stack on the very bottom edge — behind the Dock on macOS, behind the panel + on a bottom-taskbar Linux. + + The fix is a *measurement*: maximise a window and look at where the window manager + put it, since it has to honour its own panels to do that. The obvious call, + `wm_maxsize`, was tried first and is useless — on Windows it answers with the whole + screen even with a taskbar present, which is wrong in exactly the way this is meant + to fix. + """ + + def setUp(self): + # Module-level cache, so a test that measured would outlive itself. + self._was = ui._TK_WORK + ui._TK_WORK = None + self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) + + def fake_win(self, zoomed=(0, 23, 1280, 672)): + """A Tk stand-in whose `Toplevel` reports a maximised geometry.""" + x, y, r, b = zoomed + probe = mock.Mock() + probe.winfo_rootx.return_value = x + probe.winfo_rooty.return_value = y + probe.winfo_width.return_value = r - x + probe.winfo_height.return_value = b - y + return probe + + def test_it_reads_back_where_the_window_manager_put_a_maximised_window(self): + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 23, 1280, 672)) + + def test_the_probe_is_invisible_and_cleaned_up(self): + # Nothing may flash on screen, and a probe left alive is a stray window. + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + ui._tk_work_area(mock.Mock(), 1280, 720) + probe.attributes.assert_any_call("-alpha", 0.0) + probe.destroy.assert_called_once() + + def test_it_is_measured_once_and_then_remembered(self): + # `_sync_monitor` asks every frame. Measuring per frame would open and destroy a + # Toplevel thirty times a second. + probe = self.fake_win() + with mock.patch.object(ui.tk, "Toplevel", return_value=probe) as made: + for _ in range(50): + ui._tk_work_area(mock.Mock(), 1280, 720) + made.assert_called_once() + + def test_a_build_without_zoomed_falls_back_to_the_whole_screen(self): + # `state("zoomed")` is documented for Windows and X11 and may not exist here. + # The whole screen is the honest answer then: wrong by a Dock, rather than wrong + # by whatever a broken measurement returned. + probe = self.fake_win() + probe.state.side_effect = ui.tk.TclError("bad state") + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 0, 1280, 720)) + probe.destroy.assert_called_once() + + def test_a_nonsense_measurement_is_refused(self): + # A window manager that hands back something larger than the screen, or inside + # out, has not answered the question. + for bad in ((0, 0, 4000, 672), (0, 0, 0, 0), (500, 0, 100, 672)): + with self.subTest(bad=bad): + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win(bad)): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), + (0, 0, 1280, 720)) + + def test_the_bottom_edge_is_the_one_that_has_to_be_right(self): + # It is what bottom-centre placement stands on, and getting it wrong is the + # whole bug: 672 here against a screen of 720 is a 48 px Dock found. + with mock.patch.object(ui.tk, "Toplevel", return_value=self.fake_win()): + work = ui._tk_work_area(mock.Mock(), 1280, 720) + _x, y = ui.bottom_centre(ui.PILL_W, ui.PILL_H, (0, 0, 1280, 720), work, + ui.PANEL_BOTTOM_OFFSET) + self.assertLessEqual(y + ui.PILL_H, 672) + + +class TestTheMacWindowStyle(unittest.TestCase): + """Aqua's `overrideredirect` + `WS_EX_NOACTIVATE`, which is one call and unsupported. + + Reported from a real Mac: the pill sat there with its close and maximize buttons + showing. `overrideredirect(True)` is not enough on Aqua, because Tk maps a Toplevel + to an `NSWindow` whose style comes from the window *class*. + """ + + def test_it_asks_for_a_plain_window_that_does_not_activate(self): + # Both halves matter. `plain` is the frameless class; `noActivates` is what + # stops a click on the pill pulling focus off whatever is being dictated into — + # the defect that made every paste land in the wrong window on Windows. + win = mock.Mock() + self.assertTrue(ui._mac_window_style(win)) + args = win.tk.call.call_args[0] + self.assertEqual(args[0], "::tk::unsupported::MacWindowStyle") + self.assertIn("plain", args) + self.assertIn("noActivates", args) + + def test_an_unsupported_build_costs_a_title_bar_and_not_a_session(self): + # Tk calls the interface `::tk::unsupported::` itself, so it may not be there. + win = mock.Mock() + win.tk.call.side_effect = ui.tk.TclError("no such command") + self.assertFalse(ui._mac_window_style(win)) + + def test_no_activate_routes_to_it_on_a_mac_and_refuses_elsewhere(self): + # One name for "take this window out of the activation chain", answered by + # whichever platform API can actually do it. + win = mock.Mock() + with mock.patch.object(ui.sys, "platform", "darwin"): + self.assertTrue(ui._no_activate(win)) + win.tk.call.assert_called_once() + with mock.patch.object(ui.sys, "platform", "linux"): + self.assertFalse(ui._no_activate(mock.Mock())) + + class TestMix(unittest.TestCase): def test_the_ends_are_exact(self): self.assertEqual(ui._mix(ui.HEARING, ui.CARD_ACCENT, 0.0), ui.HEARING) From 74a66827dfae2f205fe2cc4888bae06d2b0a91d6 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 22:44:33 -0500 Subject: [PATCH 03/41] Give Flow a second engine, and reach for it only when the first one has nothing to run faster-whisper's weights are on HuggingFace and nowhere else official - SYSTRAN's GitHub ships the library, not the models - so a network that blocks that one host leaves copying 138 MB by hand onto every machine. macOS already has an on-device recogniser whose models belong to the OS. Using it ends the transport problem rather than routing around it. A subprocess, not a binding. The dependency budget is three and PyObjC is not one of them, and Flow already shells out to codex and claude, so a process that reads audio and writes text is a shape this app has. It also puts every Objective-C API behind a pipe, where a crash is an exit code instead of a dead interpreter. The helper is compiled on first use rather than shipped: an unsigned prebuilt binary is a Gatekeeper prompt with a worse story than the source it came from. The seam was already there. `Transcriber` is one method wide, `load`/`unload`/ `take_confidence`/`take_drops` are read through getattr, and clean.py has a documented narrow path for an engine that cannot report `no_speech_prob`. Somebody planned for this. --engine auto will not switch a machine that is working, and that is the whole rule. Apple's recogniser is a *different* engine, not a spare one: no `no_speech_prob`, so the hallucination filter drops to the narrow check; one quality tier where Whisper has two; no biasing, so the re-listen that rescues a mis-heard command comes back unbiased. Changing what Flow hears without being asked would be the wrong default, so auto reaches for it in exactly one case - the Whisper models are not on this machine and cannot be fetched. The startup line always says which engine ran and why. The presence check asks with the hub forced offline and puts the variable back, because it is asked precisely when the network may be unusable and must not become the download it is testing for. Not compiled here - this machine has no Swift. flow_stt.swift is written against SFSpeechRecognizer with requiresOnDeviceRecognition, and `--file` transcribes one recording so Apple's quality can be judged before any of it is wired into a dictation loop. The Python half is tested: the wire format is asserted against the bytes actually written, since the far side reads them with load(as: UInt32.self).littleEndian and a disagreement there is a silent mis-decode rather than an error. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 62 +++++++++ flow/__main__.py | 101 +++++++++++++- flow/native.py | 227 ++++++++++++++++++++++++++++++ native/flow_stt.swift | 191 ++++++++++++++++++++++++++ tests/test_native.py | 312 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 888 insertions(+), 5 deletions(-) create mode 100644 flow/native.py create mode 100644 native/flow_stt.swift create mode 100644 tests/test_native.py diff --git a/docs/guide.md b/docs/guide.md index 88ce824..9b897e1 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -222,6 +222,7 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--final-model X` | stronger model for the pasted text (default `small.en`) | | `--model X` | pin BOTH tiers to one model, for a low-memory machine | | `--decode-device {auto,cuda,cpu}` | where decoding runs (default `auto`: the GPU when there is a working one) | +| `--engine {auto,whisper,native}` | which decoder. `whisper` is faster-whisper and needs model files; `native` is macOS on-device speech, which needs no download at all. Default `auto`: whisper unless its models are not on the machine and the native engine is ready — see [Without HuggingFace](#without-huggingface) | | `--lexicon PATH` | personal terms file (default `~/.flow/lexicon.txt`) | | `--no-lexicon` | ignore that file without deleting it | | `--device N` | input device index; list them with `scripts/devices.py`. **Pinned**: if it goes away mid-session Flow retries *this* index and never substitutes another — see [When the microphone goes away](#if-the-microphone-goes-away-mid-session) | @@ -537,6 +538,67 @@ block says so on one line and Flow carries on with the registered combos: chord unavailable (keyboard hook refused); the toggle hotkey still works ``` +### Without HuggingFace + +faster-whisper's model files are published on HuggingFace and nowhere else official — +SYSTRAN's GitHub ships the library, not the weights. On a network that blocks +`huggingface.co` there are three ways through, and they are in this order for a reason. + +**An internal proxy, if your organisation runs one.** Nothing to move, nothing to host: + +```bash +export HF_ENDPOINT=https:///huggingface +``` + +**Any local directory.** `--model`, `--partial-model` and `--final-model` take a path as +happily as a name, so a folder holding `config.json`, `model.bin`, `tokenizer.json` and +`vocabulary.txt` is all Flow needs: + +```bash +uv run python -m flow --model ~/flow-models/base.en +``` + +`--model` pins **both** decoder tiers to one model — 138 MB for `base.en` instead of 599 +for the usual pair. Finals come out a little weaker, since `small.en` is what that tier +exists for, and everything else is unchanged. + +Copying a HuggingFace cache between machines needs one flag: the cache stores every file +as a symlink into `blobs/`, so a plain copy arrives as four broken links. Dereference +them: + +```bash +cp -RL ~/.cache/huggingface/hub/models--Systran--faster-whisper-base.en +``` + +**The macOS engine, which needs no model files at all.** + +```bash +uv run python -m flow --engine native +``` + +macOS has an on-device recogniser with models the OS downloads through **System Settings +▸ Keyboard ▸ Dictation** — enable it once and there is nothing else to fetch, ever. Flow +talks to it through a small Swift helper it compiles on first use, which needs Xcode +Command Line Tools (`xcode-select --install`) and one grant under **Privacy & Security ▸ +Speech Recognition**. + +**`--engine auto`, the default, will not switch a machine that is working.** It reaches +for the native engine in exactly one case: the Whisper models are not on the machine and +cannot be fetched. That is deliberate — Apple's recogniser is a *different* engine rather +than a spare one. It reports no `no_speech_prob`, so Flow's hallucination filter falls +back to a much narrower check; it has one quality tier where Whisper has two; and it +cannot be biased toward your lexicon, so the re-listen that rescues a mis-heard command +comes back unbiased. Those are real differences, and changing what Flow hears without +being asked would be the wrong default. The startup line always says which engine you +got and why. + +Before wiring it into anything, you can judge it on your own voice: + +```bash +swiftc -O -o flow-stt native/flow_stt.swift +./flow-stt --file some-recording.wav +``` + ### Where the panels open **Bottom centre of whichever monitor your mouse is on.** That is the shipped placement, diff --git a/flow/__main__.py b/flow/__main__.py index e268ccc..eee7d69 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -137,6 +137,87 @@ def _chord(profile, hotkeys, Chord, parse_chord, echo, ignored_line, return chord +def _native_transcriber(): + """Import late, so a Windows launch never touches the macOS-only module.""" + from .native import NativeTranscriber + + return NativeTranscriber() + + +def _engine(args, partial_name: str, final_name: str) -> tuple[str, str]: + """Which decoder this launch gets, and the clause explaining why. + + `--engine` decides when it is asked to. `auto` is the interesting one, and its rule + is deliberately conservative: **Whisper unless it cannot run.** Apple's recogniser + is a real engine but a different one — no `no_speech_prob`, so `clean.py` drops to + the narrow filler check it documents for a non-Whisper engine, one quality tier + where Whisper has two, and no hotword biasing for the rescue path. Switching to it + silently, on a machine where Whisper was working, would change what Flow hears for + reasons the user never asked about. + + So `auto` reaches for it in exactly one situation: the Whisper models are not on + this machine and cannot be fetched. That is the situation it was written for — a + network that blocks huggingface.co, where the alternative is not a worse engine but + no dictation at all. + + Every path returns a clause for the startup line, because the engine decides what + Flow can hear and a silent choice would be the one thing nobody could check. + """ + if args.engine == "whisper": + return "whisper", "" + if sys.platform != "darwin": + if args.engine == "native": + say("--engine native is macOS only; using whisper") + return "whisper", "" + + from .native import available as native_available + + if args.engine == "native": + ok, why = native_available() + if ok: + return "native", "" + say(f"--engine native unavailable: {why}") + return "whisper", "" + + # auto. Ask the cheap question first: are the models here? + if _models_present(partial_name, final_name): + return "whisper", "" + ok, why = native_available() + if ok: + return "native", " (whisper models not found on this machine)" + return "whisper", f" (not found locally, and no native engine: {why})" + + +def _models_present(*names: str) -> bool: + """Whether every named model is already on disk, without reaching the network. + + A directory holding `model.bin` is what `--model /some/path` gives, and a name like + `base.en` is present when huggingface_hub has it cached. Asked with the hub in + offline mode so this cannot become the download it is testing for. + """ + import os + from pathlib import Path + + for name in names: + path = Path(name) + if (path / "model.bin").exists(): + continue + was = os.environ.get("HF_HUB_OFFLINE") + os.environ["HF_HUB_OFFLINE"] = "1" + try: + from huggingface_hub import snapshot_download + + snapshot_download(f"Systran/faster-whisper-{name}", local_files_only=True) + except Exception: + return False + finally: + if was is None: + os.environ.pop("HF_HUB_OFFLINE", None) + else: + os.environ["HF_HUB_OFFLINE"] = was + return True + + def main(argv: list[str] | None = None) -> int: # First, before anything resolves anything. Windows searches the current directory # ahead of PATH for a bare executable name, and Flow is launched *inside* project @@ -172,6 +253,12 @@ def main(argv: list[str] | None = None) -> int: "--decode-device", default=DEVICE, choices=("auto", "cuda", "cpu"), help="where decoding runs (default auto: the GPU when there is a working one)", ) + ap.add_argument( + "--engine", default="auto", choices=("auto", "whisper", "native"), + help="which decoder: whisper (faster-whisper, needs model files) or native " + "(macOS on-device speech, no download at all). Default auto: whisper " + "unless its models are missing and the native engine is ready", + ) ap.add_argument( "--lexicon", default=None, help=f"personal terms to bias decoding toward (default {DEFAULT_PATH})", @@ -381,10 +468,14 @@ def main(argv: list[str] | None = None) -> int: default_partial, default_final = default_models(decode_device) partial_name = args.model or args.partial_model or default_partial final_name = args.model or args.final_model or default_final - if partial_name == final_name: - say(f"model: {final_name}, for partials and finals both") + engine, engine_why = _engine(args, partial_name, final_name) + if engine == "native": + say(f"engine: macOS on-device speech{engine_why}") + elif partial_name == final_name: + say(f"model: {final_name}, for partials and finals both{engine_why}") else: - say(f"models: {partial_name} for partials, {final_name} for finals") + say(f"models: {partial_name} for partials, " + f"{final_name} for finals{engine_why}") from .diag import Diag from .profile import Profile, resolve_workspace @@ -469,11 +560,11 @@ def main(argv: list[str] | None = None) -> int: profile.save() session = Session( - asr=WhisperTranscriber( + asr=(_native_transcriber() if engine == "native" else WhisperTranscriber( partial_name, final_name, lexicon=lexicon, baseline=profile.confidence if profile is not None else None, device=args.decode_device, - ), + )), device=args.device, speaker=speaker, profile=profile, diff --git a/flow/native.py b/flow/native.py new file mode 100644 index 0000000..e9a0b92 --- /dev/null +++ b/flow/native.py @@ -0,0 +1,227 @@ +"""Apple's on-device recogniser, as a `Transcriber` Flow can use instead of Whisper. + +**Why this exists.** faster-whisper's CT2 weights are published on HuggingFace and +nowhere else official — SYSTRAN's GitHub ships the library, not the models. On a network +that blocks `huggingface.co` that leaves copying 138 MB by hand onto every machine. +macOS has a speech engine already installed, with models the OS downloads through +System Settings, and using it ends the transport problem rather than routing around it. + +**Why a subprocess.** The dependency budget is three (R16) and PyObjC is not one of them. +Flow already shells out to `codex` and `claude`, so a process that reads audio and writes +text is a shape this app has. It also puts every Objective-C API behind a pipe, where a +crash is an exit code instead of a dead interpreter. + +**What it costs, stated rather than discovered.** Apple's recogniser reports no +`no_speech_prob`, so `clean.py` falls to the narrow whole-utterance filler check it +documents for exactly this case — hallucination filtering is weaker here than with +Whisper. `hotwords` has no equivalent on this path and is accepted and ignored, which +makes the constrained re-decode of a suspected mis-heard command a no-op rather than an +error. And the permission prompt is attributed to whatever launched Flow, usually a +terminal, because this is not a signed bundle. +""" + +from __future__ import annotations + +import struct +import subprocess +import sys +import threading +from pathlib import Path + +import numpy as np + +#: The helper's source, and where its build lands. Built into the user's cache rather +#: than into the checkout: a repo on a read-only share still has to work, and a binary +#: in a source tree is a thing that gets committed by accident. +SOURCE = Path(__file__).resolve().parent.parent / "native" / "flow_stt.swift" +BUILD_DIR = Path.home() / ".flow" / "bin" +BINARY = BUILD_DIR / "flow-stt" + +#: How long a build may take before it is called a failure. `swiftc` on a cold toolchain +#: is slow; a launch is not the place to find out how slow. +BUILD_TIMEOUT_SEC = 120.0 +#: How long one utterance may take. The helper has its own 30 s bound inside; this is the +#: outer one for a process that has stopped answering at all. +DECODE_TIMEOUT_SEC = 40.0 + + +class NotAvailable(RuntimeError): + """This machine cannot run the native engine, with the reason in the message. + + One exception for every way of not having it — wrong platform, no toolchain, a build + that failed, a permission the user declined — because every one of them means the + same thing to the caller: use Whisper, and say this sentence to explain why. + """ + + +def _run(argv: list[str], timeout: float) -> subprocess.CompletedProcess: + return subprocess.run(argv, capture_output=True, text=True, timeout=timeout, + check=False) + + +def build(force: bool = False) -> Path: + """Compile the helper if it is not already built. Returns the binary's path. + + Built on first use rather than shipped, because a prebuilt binary would have to be + signed and notarised to be worth anything on a Mac, and an unsigned one is a + Gatekeeper prompt with a worse story than the source it came from. `swiftc` is + present wherever Xcode Command Line Tools are, which is most machines that have + ever run `git`. + """ + if sys.platform != "darwin": + raise NotAvailable("the native engine is macOS only") + if BINARY.exists() and not force: + return BINARY + if not SOURCE.exists(): + raise NotAvailable(f"helper source missing: {SOURCE}") + try: + which = _run(["xcrun", "--find", "swiftc"], 20.0) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"no Swift toolchain: {exc}") from exc + if which.returncode != 0: + raise NotAvailable("no Swift toolchain — run: xcode-select --install") + BUILD_DIR.mkdir(parents=True, exist_ok=True) + try: + built = _run(["xcrun", "swiftc", "-O", "-o", str(BINARY), str(SOURCE)], + BUILD_TIMEOUT_SEC) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"build failed: {exc}") from exc + if built.returncode != 0 or not BINARY.exists(): + # The compiler's own words, trimmed. A build error the user cannot see is a + # feature that is missing for no stated reason. + detail = (built.stderr or built.stdout or "").strip().splitlines() + raise NotAvailable("build failed: " + (detail[-1] if detail else "no output")) + return BINARY + + +def available() -> tuple[bool, str]: + """`(usable, why not)` for this machine, without starting a session on it. + + Runs the helper's own `--probe`, which is the only honest check: the engine exists + when the OS says it does, the locale resolves, on-device recognition is supported — + which needs Dictation enabled so macOS has downloaded the offline model — and the + user has granted the permission. Anything less is a guess that fails later, in the + middle of somebody's first sentence. + """ + try: + binary = build() + except NotAvailable as exc: + return False, str(exc) + try: + probe = _run([str(binary), "--probe"], 60.0) + except (OSError, subprocess.SubprocessError) as exc: + return False, f"probe failed: {exc}" + if probe.returncode != 0: + return False, (probe.stderr or "probe refused").strip().splitlines()[-1] + return True, "" + + +class NativeTranscriber: + """Satisfies `asr.Transcriber` by talking to one long-lived helper process. + + Long-lived rather than one process per utterance, because starting a process and + authorising a recogniser costs more than the decode does. The process is started + lazily on the first decode — `load()` is what a caller uses to pay that cost at a + moment of its choosing, which is exactly what `Session._warm` already does for + Whisper's tiers. + """ + + #: Set so `Session._pump_health`'s idle-unload check reads something true. There is + #: no 605 MB to give back here — the models belong to the OS — but the session asks. + loading = False + + def __init__(self, binary: Path | None = None) -> None: + self._binary = Path(binary) if binary else None + self._proc: subprocess.Popen | None = None + #: One decode at a time over one pipe. `DecodeWorker` is single-threaded today, + #: and this makes that a guarantee of this class rather than a fact about its + #: caller. + self._lock = threading.Lock() + + # -- lifecycle --------------------------------------------------------- + + @property + def loaded(self) -> bool: + return self._proc is not None and self._proc.poll() is None + + def load(self, final=None) -> None: + """Start the helper. Idempotent, and the signature matches Whisper's tier load.""" + with self._lock: + self._ensure() + + def unload(self) -> None: + """Close the helper down. The idle path calls this; there is little to reclaim. + + Closing stdin is the documented way out — the helper returns from its read loop + and exits — so this asks before it insists. + """ + with self._lock: + proc, self._proc = self._proc, None + if proc is None: + return + try: + if proc.stdin: + proc.stdin.close() + proc.wait(timeout=2.0) + except (OSError, subprocess.SubprocessError): + pass + if proc.poll() is None: + proc.kill() + + def _ensure(self) -> None: + if self.loaded: + return + binary = self._binary or build() + try: + self._proc = subprocess.Popen( + [str(binary)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + except (OSError, subprocess.SubprocessError) as exc: + raise NotAvailable(f"could not start {binary}: {exc}") from exc + + # -- the one method the protocol asks for ------------------------------ + + def text(self, audio: np.ndarray, *, final: bool = False, + hotwords: str = "") -> str: + """Transcribe mono float32 at 16 kHz. + + `final` is accepted and ignored: Apple's recogniser has one quality, where + Whisper has a fast tier and a strong one. That is a real difference and not a + stub — there is no second model to reach for, so a partial and a final of the + same audio return the same words. + + `hotwords` is accepted and ignored too. Biasing has no equivalent here, so the + constrained re-decode that `Session.submit_rescue` performs comes back + unbiased rather than failing. + """ + block = np.ascontiguousarray(audio, dtype=np.float32) + if block.size == 0: + return "" + with self._lock: + self._ensure() + proc = self._proc + if proc is None or proc.stdin is None or proc.stdout is None: + raise NotAvailable("helper is not running") + try: + proc.stdin.write(struct.pack(" Never { + FileHandle.standardError.write(("flow-stt: " + message + "\n").data(using: .utf8)!) + exit(code) +} + +/// Ask once, block until the user has answered, and treat every non-authorized answer +/// the same. The prompt is attributed to whatever launched this — a terminal, usually — +/// which is the known wart of running unbundled and is documented on the Python side. +func authorize() { + let gate = DispatchSemaphore(value: 0) + var status: SFSpeechRecognizerAuthorizationStatus = .notDetermined + SFSpeechRecognizer.requestAuthorization { got in + status = got + gate.signal() + } + gate.wait() + guard status == .authorized else { + die("speech recognition not authorized (\(status.rawValue)). " + + "System Settings > Privacy & Security > Speech Recognition.", 2) + } +} + +func recognizer() -> SFSpeechRecognizer { + guard let rec = SFSpeechRecognizer(locale: Locale(identifier: "en-US")) else { + die("no recognizer for en-US on this machine", 3) + } + guard rec.isAvailable else { die("recognizer exists but is not available", 4) } + // The whole point. Without this the audio goes to Apple's servers, which is a + // different product from the one Flow is: local by construction. + guard rec.supportsOnDeviceRecognition else { + die("on-device recognition unavailable — enable Dictation in System Settings " + + "so macOS downloads the offline model", 5) + } + return rec +} + +/// Transcribe one finished buffer. Synchronous on purpose: the caller has already +/// decided this audio is complete, and Flow's decode worker is the thing that owns +/// concurrency. +func transcribe(_ rec: SFSpeechRecognizer, _ samples: [Float]) -> String { + guard !samples.isEmpty else { return "" } + let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, channels: 1, interleaved: false)! + guard let buffer = AVAudioPCMBuffer(pcmFormat: format, + frameCapacity: AVAudioFrameCount(samples.count)) + else { return "" } + buffer.frameLength = AVAudioFrameCount(samples.count) + samples.withUnsafeBufferPointer { src in + buffer.floatChannelData!.pointee.update(from: src.baseAddress!, + count: samples.count) + } + + let request = SFSpeechAudioBufferRecognitionRequest() + request.requiresOnDeviceRecognition = true + request.shouldReportPartialResults = false + // Punctuation is the difference between dictation and a transcript. Flow's own + // cleaner assumes sentences. + if #available(macOS 13.0, *) { request.addsPunctuation = true } + request.append(buffer) + request.endAudio() + + let gate = DispatchSemaphore(value: 0) + var text = "" + let task = rec.recognitionTask(with: request) { result, error in + if let result = result, result.isFinal { + text = result.bestTranscription.formattedString + gate.signal() + } else if error != nil { + // A recogniser that heard nothing reports an error rather than an empty + // result. Silence is not a failure here — Flow's gate already decided this + // block was speech, and an empty line is how "nothing said" is spelled. + gate.signal() + } + } + // Bounded, because a recogniser that never calls back would otherwise hang the + // decode worker and, through it, the draft the user is waiting for. + if gate.wait(timeout: .now() + 30) == .timedOut { task.cancel() } + return text +} + +func serve(_ rec: SFSpeechRecognizer) { + let input = FileHandle.standardInput + let output = FileHandle.standardOutput + while true { + let header = input.readData(ofLength: 4) + if header.count < 4 { return } // stdin closed: Flow is gone, so are we + let count = header.withUnsafeBytes { $0.load(as: UInt32.self).littleEndian } + var payload = Data() + while payload.count < Int(count) * 4 { + let chunk = input.readData(ofLength: Int(count) * 4 - payload.count) + if chunk.isEmpty { return } + payload.append(chunk) + } + let samples = payload.withUnsafeBytes { raw -> [Float] in + Array(raw.bindMemory(to: Float.self)) + } + let line = transcribe(rec, samples).replacingOccurrences(of: "\n", with: " ") + output.write((line + "\n").data(using: .utf8)!) + } +} + +func fromFile(_ path: String, _ rec: SFSpeechRecognizer) { + guard let file = try? AVAudioFile(forReading: URL(fileURLWithPath: path)) else { + die("cannot read \(path)") + } + let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, channels: 1, interleaved: false)! + guard let converter = AVAudioConverter(from: file.processingFormat, to: format), + let source = AVAudioPCMBuffer(pcmFormat: file.processingFormat, + frameCapacity: AVAudioFrameCount(file.length)) + else { die("cannot convert \(path) to 16 kHz mono") } + try? file.read(into: source) + let frames = AVAudioFrameCount(Double(source.frameLength) + * SAMPLE_RATE / file.processingFormat.sampleRate) + 1024 + guard let out = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frames) else { + die("cannot allocate output buffer") + } + var done = false + var error: NSError? + converter.convert(to: out, error: &error) { _, status in + if done { status.pointee = .noDataNow; return nil } + done = true + status.pointee = .haveData + return source + } + if let error = error { die("resample failed: \(error.localizedDescription)") } + let samples = Array(UnsafeBufferPointer(start: out.floatChannelData![0], + count: Int(out.frameLength))) + print(transcribe(rec, samples)) +} + +// -- main ------------------------------------------------------------------ + +let args = Array(CommandLine.arguments.dropFirst()) +if args.first == "--probe" { + // What the Python side calls to decide whether this engine exists at all, before + // it commits a session to it. Authorization included, because an engine the user + // has not allowed is not an engine that works. + authorize() + _ = recognizer() + print("ok") + exit(0) +} +authorize() +let rec = recognizer() +if args.count == 2 && args[0] == "--file" { + fromFile(args[1], rec) +} else if args.isEmpty { + serve(rec) +} else { + die("usage: flow-stt [--probe | --file AUDIO]") +} diff --git a/tests/test_native.py b/tests/test_native.py new file mode 100644 index 0000000..dd3e6df --- /dev/null +++ b/tests/test_native.py @@ -0,0 +1,312 @@ +"""The macOS on-device decoder, and the rule that decides when Flow reaches for it. + +The engine decides what Flow can *hear*, so the interesting tests here are not about +Swift — none of this compiles a helper — they are about the choice. Two properties carry +it: + + **`auto` never switches a working machine.** Apple's recogniser is a different engine, + not a spare one: no `no_speech_prob`, so `clean.py` falls to the narrow filler check it + documents for exactly that; one quality tier where Whisper has two; no hotword biasing + for the rescue path. Reaching for it on a machine where Whisper was fine would change + what Flow hears for a reason nobody asked about. + + **Every refusal says why.** "Not available" is four different sentences — wrong + platform, no toolchain, a build that failed, a permission declined — and a user who + cannot see which one has a feature that is missing for no stated reason. + +The subprocess is faked at `Popen`, so the framing this file cares about — a +length-prefixed block of float32 out, one line of text back — is asserted against the +bytes actually written rather than against a Mac nobody in CI has. +""" + +import struct +import subprocess +import sys +import unittest +from pathlib import Path +from unittest import mock + +import numpy as np + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +import flow.native as native # noqa: E402 +from flow.__main__ import _engine, _models_present # noqa: E402 + + +def args(engine="auto"): + return mock.Mock(engine=engine) + + +def _only(present): + """A `Path.exists` that is true for exactly one path. + + `mock.patch.object` cannot patch an attribute on a `Path` *instance* — they are + read-only — so the class method is replaced and told which path is meant to be + there. Which is the distinction under test: the helper source exists, the built + binary does not, so a build is attempted. + """ + return lambda self: self == present + + +class TestTheEngineChoice(unittest.TestCase): + """`_engine`, which is the whole feature — the rest is plumbing.""" + + def test_asking_for_whisper_gets_whisper_and_asks_nothing(self): + # No probe, no build, no subprocess: somebody who named the engine has already + # answered the question this function exists to ask. + with mock.patch.object(native, "available") as probe: + self.assertEqual(_engine(args("whisper"), "base.en", "small.en"), + ("whisper", "")) + probe.assert_not_called() + + def test_auto_keeps_whisper_when_the_models_are_on_the_machine(self): + # The property that matters most. A working machine is never switched. + with mock.patch("flow.__main__._models_present", return_value=True), \ + mock.patch.object(native, "available") as probe: + self.assertEqual(_engine(args(), "base.en", "small.en"), ("whisper", "")) + probe.assert_not_called() + + def test_auto_reaches_for_native_only_when_whisper_has_nothing_to_run(self): + # The situation this was written for: a network that blocks huggingface.co, + # where the alternative is not a worse engine but no dictation at all. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", return_value=(True, "")): + engine, why = _engine(args(), "base.en", "small.en") + self.assertEqual(engine, "native") + self.assertIn("models not found", why) + + def test_and_says_so_on_the_startup_line(self): + # A silent engine change would be the one thing nobody could check. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", return_value=(True, "")): + _engine, why = _engine_result() + self.assertTrue(why.strip()) + + def test_neither_engine_available_still_returns_whisper_and_names_the_reason(self): + # Whisper will fail its own way, with its own message. What this must not do is + # return an engine that is not there, or fail silently between the two. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", + return_value=(False, "Dictation is off")): + engine, why = _engine(args(), "base.en", "small.en") + self.assertEqual(engine, "whisper") + self.assertIn("Dictation is off", why) + + def test_asking_for_native_off_a_mac_is_refused_out_loud(self): + said = [] + with mock.patch.object(sys, "platform", "win32"), \ + mock.patch("flow.__main__.say", said.append): + self.assertEqual(_engine(args("native"), "base.en", "small.en"), + ("whisper", "")) + self.assertIn("macOS only", " ".join(said)) + + def test_asking_for_native_on_a_mac_that_cannot_reports_the_reason(self): + said = [] + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__.say", said.append), \ + mock.patch.object(native, "available", + return_value=(False, "no Swift toolchain")): + self.assertEqual(_engine(args("native"), "base.en", "small.en"), + ("whisper", "")) + self.assertIn("no Swift toolchain", " ".join(said)) + + +def _engine_result(): + return _engine(args(), "base.en", "small.en") + + +class TestTheModelPresenceCheckNeverDownloads(unittest.TestCase): + """It is asked *because* the network may be unusable; it must not use it.""" + + def test_a_directory_holding_a_model_counts_without_asking_the_hub(self): + # What `--model /some/path` gives. Nothing to look up. + with mock.patch.object(Path, "exists", return_value=True): + with mock.patch.dict(sys.modules, {"huggingface_hub": None}): + self.assertTrue(_models_present("/somewhere/base.en")) + + def test_a_hub_lookup_is_made_offline(self): + seen = {} + fake = mock.Mock() + + def snapshot(repo, local_files_only=False): + import os + seen["offline"] = os.environ.get("HF_HUB_OFFLINE") + seen["local_only"] = local_files_only + return "/cache" + + fake.snapshot_download = snapshot + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + self.assertTrue(_models_present("base.en")) + self.assertEqual(seen["offline"], "1") + self.assertTrue(seen["local_only"]) + + def test_a_missing_model_is_absent_rather_than_an_exception(self): + fake = mock.Mock() + fake.snapshot_download = mock.Mock(side_effect=OSError("not cached")) + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + self.assertFalse(_models_present("base.en")) + + def test_the_environment_is_left_as_it_was_found(self): + # It sets HF_HUB_OFFLINE to ask its question. A launch that then downloaded + # nothing for the rest of the session would be this function's fault. + import os + + fake = mock.Mock() + fake.snapshot_download = mock.Mock(return_value="/cache") + was = os.environ.get("HF_HUB_OFFLINE") + with mock.patch.object(Path, "exists", return_value=False), \ + mock.patch.dict(sys.modules, {"huggingface_hub": fake}): + _models_present("base.en") + self.assertEqual(os.environ.get("HF_HUB_OFFLINE"), was) + + +class TestItRefusesOffAMacWithAReason(unittest.TestCase): + def test_build_names_the_platform(self): + with self.assertRaises(native.NotAvailable) as caught: + native.build() + self.assertIn("macOS", str(caught.exception)) + + def test_available_answers_false_and_why_rather_than_raising(self): + # `available()` is called during startup. A raise there is a launch that dies + # over a feature the machine was never going to have. + ok, why = native.available() + self.assertFalse(ok) + self.assertTrue(why) + + def test_a_missing_toolchain_is_named_with_the_command_that_fixes_it(self): + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", _only(native.SOURCE)), \ + mock.patch.object(native, "_run", + return_value=mock.Mock(returncode=1)): + ok, why = native.available() + self.assertFalse(ok) + self.assertIn("xcode-select --install", why) + + def test_a_build_error_carries_the_compilers_last_word(self): + fail = mock.Mock(returncode=1, stderr="flow_stt.swift:9: error: no such module", + stdout="") + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", _only(native.SOURCE)), \ + mock.patch.object(Path, "mkdir", lambda *a, **k: None), \ + mock.patch.object(native, "_run", + side_effect=[mock.Mock(returncode=0), fail]): + ok, why = native.available() + self.assertFalse(ok) + self.assertIn("no such module", why) + + +class TestTheWireFormat(unittest.TestCase): + """A length-prefixed block of float32 out, one line of text back. + + Asserted against the bytes actually written, because the Swift on the other end is + reading them with `load(as: UInt32.self).littleEndian` and a disagreement about + endianness or element size is a silent mis-decode rather than an error. + """ + + def transcriber(self, reply=b"hello there\n"): + proc = mock.Mock() + proc.poll.return_value = None + proc.stdout.readline.return_value = reply + proc.stdin = mock.Mock() + a = native.NativeTranscriber(binary=Path("/fake/flow-stt")) + a._proc = proc + return a, proc + + def test_it_writes_a_little_endian_count_then_the_samples(self): + a, proc = self.transcriber() + audio = np.arange(4, dtype=np.float32) + a.text(audio) + count, payload = [c[0][0] for c in proc.stdin.write.call_args_list] + self.assertEqual(count, struct.pack(" Date: Sun, 30 Aug 2026 22:59:10 -0500 Subject: [PATCH 04/41] Compile the Swift helper in CI, and check the arguments before asking for consent The macOS decoder is written on a machine with no Swift toolchain, so until now the compiler was a person on a laptop pasting errors back. The first version shipped with a hard one in it: Swift allows top-level statements only in a file called `main.swift`, and this is not one. A macos-latest leg is the fix - the same runner the suite already uses, doing the one thing that machine can do and this one cannot. Three more found by reading it again with that in mind, two of which would have been runtime rather than a build failure: * The recogniser delivers its callbacks to the main queue by default, and `transcribe` blocks the calling thread waiting for one. On the main thread that is a deadlock, not a slow decode. It gets its own OperationQueue. * Every read out of `Data` is unaligned now. `Data` carries no alignment guarantee and `load(as:)` requires one, so the aligned form traps on some buffers and not others - the worst way to find out. * `update(from:count:)` is recent; `memcpy` is not, and this has to build on whatever toolchain a machine happens to have. Arguments are parsed before `authorize()`. A typo should not raise a permission prompt, and CI has to reach the usage line on a runner where no permission could ever be granted - which is what lets the leg prove the binary links and runs at all rather than merely compiles. Compile-and-link only, and the comment says why that is the honest limit: actually running it needs a granted Speech Recognition permission and an on-device model that only arrives when a human enables Dictation. Faking either would make the leg green about something it never checked. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 43 ++++++++++ native/flow_stt.swift | 170 ++++++++++++++++++++++++++------------- 2 files changed, 157 insertions(+), 56 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f51a9cf..c3d3476 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -88,3 +88,46 @@ jobs: # does first and no unit test does. - name: the entry point starts run: uv run flow --help + + # The one part of Flow that is not Python, on the only runner that can compile it. + # + # `native/flow_stt.swift` is the macOS on-device decoder — the answer for machines that + # cannot reach huggingface.co, where faster-whisper's weights are the only official + # copy. It is written on Windows, where there is no Swift toolchain, so without this + # leg the compiler is a person on a laptop pasting errors back. The first version was + # exactly that and shipped with a hard one in it: Swift allows top-level statements + # only in a file called `main.swift`, and this is not one. + # + # Compile only, and that is the honest limit of what CI can say here. Actually + # *running* it needs a granted Speech Recognition permission and an on-device model + # that only arrives when a human enables Dictation — neither of which a headless runner + # has, and faking them would make this leg green about something it never checked. + # `--probe` would exit 2 on the runner for exactly that reason, which is the right + # answer and a useless test. + helper: + name: swift helper compiles + runs-on: macos-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + + - name: swiftc version + run: xcrun swiftc --version + + # `-warnings-as-errors` because this file is edited by people who cannot run it. + # A warning here is the only signal that reaches them before a user does. + - name: compile the macOS decoder + run: xcrun swiftc -O -warnings-as-errors -o /tmp/flow-stt native/flow_stt.swift + + # It links against AVFoundation and Speech, so a binary that builds but cannot + # resolve a symbol is still a broken helper. Asking it for its usage line proves + # the dynamic linker is satisfied without needing a microphone or a permission. + - name: it links and runs far enough to refuse + run: | + set +e + out=$(/tmp/flow-stt --nonsense 2>&1) + code=$? + echo "$out" + # 1 is `die("usage: ...")`. Anything else — a link failure, a crash, or a + # silent success on an argument that is not valid — is a broken build. + if [ "$code" -ne 1 ]; then echo "expected exit 1, got $code"; exit 1; fi + case "$out" in *usage*) ;; *) echo "expected a usage line"; exit 1 ;; esac diff --git a/native/flow_stt.swift b/native/flow_stt.swift index 15e0aa1..eabdc5b 100644 --- a/native/flow_stt.swift +++ b/native/flow_stt.swift @@ -30,12 +30,24 @@ // float32 samples. One line of UTF-8 back per block, newline-terminated, empty line for // silence. Anything this cannot do goes to stderr and exits non-zero, so the Python // side can report a reason rather than a hang. +// +// Three things here are load-bearing and easy to undo by tidying: +// +// * **`@main`, not top-level code.** Swift allows statements at file scope only in a +// file called `main.swift`. This one is not, so the entry point is a type. +// * **The recogniser gets its own queue.** Its callbacks default to the main queue, +// and `transcribe` blocks the calling thread waiting for one — on the main thread +// that is a deadlock, not a slow decode. +// * **Every read out of `Data` is unaligned.** `Data` gives no alignment guarantee +// and `load(as:)` requires one; the aligned form crashes on some buffers and not +// others, which is the worst way to find out. import AVFoundation import Foundation import Speech let SAMPLE_RATE = 16000.0 +let DECODE_TIMEOUT: TimeInterval = 30 func die(_ message: String, _ code: Int32 = 1) -> Never { FileHandle.standardError.write(("flow-stt: " + message + "\n").data(using: .utf8)!) @@ -54,12 +66,12 @@ func authorize() { } gate.wait() guard status == .authorized else { - die("speech recognition not authorized (\(status.rawValue)). " + die("speech recognition not authorized (status \(status.rawValue)). " + "System Settings > Privacy & Security > Speech Recognition.", 2) } } -func recognizer() -> SFSpeechRecognizer { +func makeRecognizer() -> SFSpeechRecognizer { guard let rec = SFSpeechRecognizer(locale: Locale(identifier: "en-US")) else { die("no recognizer for en-US on this machine", 3) } @@ -70,39 +82,48 @@ func recognizer() -> SFSpeechRecognizer { die("on-device recognition unavailable — enable Dictation in System Settings " + "so macOS downloads the offline model", 5) } + // Off the main queue, deliberately. `transcribe` waits on a semaphore for the + // result, and a recogniser delivering that result *to the thread doing the waiting* + // is a deadlock. This is the line that makes the blocking call safe. + rec.queue = OperationQueue() return rec } /// Transcribe one finished buffer. Synchronous on purpose: the caller has already -/// decided this audio is complete, and Flow's decode worker is the thing that owns -/// concurrency. +/// decided this audio is complete, and Flow's decode worker owns concurrency. func transcribe(_ rec: SFSpeechRecognizer, _ samples: [Float]) -> String { guard !samples.isEmpty else { return "" } - let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, - sampleRate: SAMPLE_RATE, channels: 1, interleaved: false)! - guard let buffer = AVAudioPCMBuffer(pcmFormat: format, - frameCapacity: AVAudioFrameCount(samples.count)) + guard let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, + channels: 1, interleaved: false), + let buffer = AVAudioPCMBuffer(pcmFormat: format, + frameCapacity: AVAudioFrameCount(samples.count)), + let channel = buffer.floatChannelData else { return "" } buffer.frameLength = AVAudioFrameCount(samples.count) samples.withUnsafeBufferPointer { src in - buffer.floatChannelData!.pointee.update(from: src.baseAddress!, - count: samples.count) + if let base = src.baseAddress { + memcpy(channel[0], base, samples.count * MemoryLayout.size) + } } let request = SFSpeechAudioBufferRecognitionRequest() request.requiresOnDeviceRecognition = true request.shouldReportPartialResults = false - // Punctuation is the difference between dictation and a transcript. Flow's own - // cleaner assumes sentences. + // Punctuation is the difference between dictation and a transcript; Flow's cleaner + // assumes sentences. if #available(macOS 13.0, *) { request.addsPunctuation = true } request.append(buffer) request.endAudio() let gate = DispatchSemaphore(value: 0) + // Written from the recogniser's queue and read from this one, so the handoff is + // guarded rather than assumed. + let lock = NSLock() var text = "" let task = rec.recognitionTask(with: request) { result, error in if let result = result, result.isFinal { - text = result.bestTranscription.formattedString + lock.lock(); text = result.bestTranscription.formattedString; lock.unlock() gate.signal() } else if error != nil { // A recogniser that heard nothing reports an error rather than an empty @@ -113,28 +134,40 @@ func transcribe(_ rec: SFSpeechRecognizer, _ samples: [Float]) -> String { } // Bounded, because a recogniser that never calls back would otherwise hang the // decode worker and, through it, the draft the user is waiting for. - if gate.wait(timeout: .now() + 30) == .timedOut { task.cancel() } + if gate.wait(timeout: .now() + DECODE_TIMEOUT) == .timedOut { task.cancel() } + lock.lock(); defer { lock.unlock() } return text } +/// Read exactly `count` bytes, or nil if the pipe closed first. +func readExactly(_ handle: FileHandle, _ count: Int) -> Data? { + var out = Data() + while out.count < count { + let chunk = handle.readData(ofLength: count - out.count) + if chunk.isEmpty { return nil } + out.append(chunk) + } + return out +} + func serve(_ rec: SFSpeechRecognizer) { let input = FileHandle.standardInput let output = FileHandle.standardOutput while true { - let header = input.readData(ofLength: 4) - if header.count < 4 { return } // stdin closed: Flow is gone, so are we - let count = header.withUnsafeBytes { $0.load(as: UInt32.self).littleEndian } - var payload = Data() - while payload.count < Int(count) * 4 { - let chunk = input.readData(ofLength: Int(count) * 4 - payload.count) - if chunk.isEmpty { return } - payload.append(chunk) - } - let samples = payload.withUnsafeBytes { raw -> [Float] in - Array(raw.bindMemory(to: Float.self)) + guard let header = readExactly(input, 4) else { return } // Flow is gone + // Unaligned on purpose: `Data`'s backing store carries no alignment guarantee, + // and the aligned `load(as:)` traps on buffers that happen not to be. + let count = Int(header.withUnsafeBytes { + $0.loadUnaligned(fromByteOffset: 0, as: UInt32.self).littleEndian + }) + guard count > 0, count < 30 * Int(SAMPLE_RATE) * 60 else { return } + guard let payload = readExactly(input, count * 4) else { return } + var samples = [Float](repeating: 0, count: count) + samples.withUnsafeMutableBytes { dst in + payload.copyBytes(to: dst.bindMemory(to: UInt8.self)) } let line = transcribe(rec, samples).replacingOccurrences(of: "\n", with: " ") - output.write((line + "\n").data(using: .utf8)!) + if let data = (line + "\n").data(using: .utf8) { output.write(data) } } } @@ -142,50 +175,75 @@ func fromFile(_ path: String, _ rec: SFSpeechRecognizer) { guard let file = try? AVAudioFile(forReading: URL(fileURLWithPath: path)) else { die("cannot read \(path)") } - let format = AVAudioFormat(commonFormat: .pcmFormatFloat32, - sampleRate: SAMPLE_RATE, channels: 1, interleaved: false)! - guard let converter = AVAudioConverter(from: file.processingFormat, to: format), + guard let target = AVAudioFormat(commonFormat: .pcmFormatFloat32, + sampleRate: SAMPLE_RATE, + channels: 1, interleaved: false), + let converter = AVAudioConverter(from: file.processingFormat, to: target), let source = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: AVAudioFrameCount(file.length)) else { die("cannot convert \(path) to 16 kHz mono") } - try? file.read(into: source) - let frames = AVAudioFrameCount(Double(source.frameLength) - * SAMPLE_RATE / file.processingFormat.sampleRate) + 1024 - guard let out = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frames) else { + do { try file.read(into: source) } catch { die("read failed: \(error)") } + + let ratio = SAMPLE_RATE / file.processingFormat.sampleRate + let frames = AVAudioFrameCount(Double(source.frameLength) * ratio) + 4096 + guard let out = AVAudioPCMBuffer(pcmFormat: target, frameCapacity: frames) else { die("cannot allocate output buffer") } - var done = false + var supplied = false var error: NSError? converter.convert(to: out, error: &error) { _, status in - if done { status.pointee = .noDataNow; return nil } - done = true + if supplied { + status.pointee = .endOfStream + return nil + } + supplied = true status.pointee = .haveData return source } if let error = error { die("resample failed: \(error.localizedDescription)") } - let samples = Array(UnsafeBufferPointer(start: out.floatChannelData![0], + guard let channel = out.floatChannelData else { die("no samples after resample") } + let samples = Array(UnsafeBufferPointer(start: channel[0], count: Int(out.frameLength))) print(transcribe(rec, samples)) } -// -- main ------------------------------------------------------------------ - -let args = Array(CommandLine.arguments.dropFirst()) -if args.first == "--probe" { - // What the Python side calls to decide whether this engine exists at all, before - // it commits a session to it. Authorization included, because an engine the user - // has not allowed is not an engine that works. - authorize() - _ = recognizer() - print("ok") - exit(0) +enum Mode { + case probe + case file(String) + case serve } -authorize() -let rec = recognizer() -if args.count == 2 && args[0] == "--file" { - fromFile(args[1], rec) -} else if args.isEmpty { - serve(rec) -} else { - die("usage: flow-stt [--probe | --file AUDIO]") + +@main +struct FlowSTT { + /// **Arguments are checked before anything is asked of the user.** A typo must not + /// raise a permission prompt, and CI has to be able to reach the usage line on a + /// runner where no permission could ever be granted — which is the only way the + /// compile leg can prove this binary links and runs at all. + static func main() { + let args = Array(CommandLine.arguments.dropFirst()) + let mode: Mode + if args == ["--probe"] { + mode = .probe + } else if args.count == 2 && args[0] == "--file" { + mode = .file(args[1]) + } else if args.isEmpty { + mode = .serve + } else { + die("usage: flow-stt [--probe | --file AUDIO]") + } + + // Everything past here needs a recogniser, and a recogniser needs consent. + authorize() + let rec = makeRecognizer() + switch mode { + case .probe: + // What the Python side calls to decide whether this engine exists at all, + // before it commits a session to it. Reaching this line is the answer. + print("ok") + case .file(let path): + fromFile(path, rec) + case .serve: + serve(rec) + } + } } From 1804e60c6aa438883cafc1de528f075bfcca1fbf Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 23:04:54 -0500 Subject: [PATCH 05/41] Pass -parse-as-library, and stop three tests reading the runner's platform CI compiled the helper and said two things. `@main` and a single-file executable are mutually exclusive: Swift parses one file as a script, and the attribute cannot be used in a module with top-level code. The compiler's own note names the fix, so `-parse-as-library` goes in all four places that build this - the file's own header, flow/native.py, the CI leg and the guide. Renaming to main.swift was the alternative; the flag keeps the file named after what it is. `withUnsafeMutableBytes` hands back whatever its closure returns, here `copyBytes`' byte count, and the leg builds with -warnings-as-errors on purpose: this file is edited by people who cannot run it, so a warning is the only signal that reaches them before a user does. Three of mine in the macOS suite, all the same mistake - a test reading the platform it happens to be on rather than pinning one. `ctypes.windll` does not exist off Windows, so patching it needs create=True; and the two that assert the native engine's *refusal* were pinned to win32, because on a macOS runner there is nothing to refuse and they would go build a real binary instead. A test that means one thing on Windows and another on macOS is only half run wherever it passes. The macOS leg was already red on main before this branch - test_lite and test_main reach ctypes.WinDLL, and the font-metric check measures 10 where Windows measures 7. Those are not touched here. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 2 +- docs/guide.md | 2 +- flow/native.py | 6 ++++-- native/flow_stt.swift | 13 +++++++++++-- tests/test_native.py | 16 +++++++++++++--- tests/test_pill.py | 6 ++++-- 6 files changed, 34 insertions(+), 11 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c3d3476..57ce8e5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -116,7 +116,7 @@ jobs: # `-warnings-as-errors` because this file is edited by people who cannot run it. # A warning here is the only signal that reaches them before a user does. - name: compile the macOS decoder - run: xcrun swiftc -O -warnings-as-errors -o /tmp/flow-stt native/flow_stt.swift + run: xcrun swiftc -O -parse-as-library -warnings-as-errors -o /tmp/flow-stt native/flow_stt.swift # It links against AVFoundation and Speech, so a binary that builds but cannot # resolve a symbol is still a broken helper. Asking it for its usage line proves diff --git a/docs/guide.md b/docs/guide.md index 9b897e1..6bfd66b 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -595,7 +595,7 @@ got and why. Before wiring it into anything, you can judge it on your own voice: ```bash -swiftc -O -o flow-stt native/flow_stt.swift +swiftc -O -parse-as-library -o flow-stt native/flow_stt.swift ./flow-stt --file some-recording.wav ``` diff --git a/flow/native.py b/flow/native.py index e9a0b92..46610a6 100644 --- a/flow/native.py +++ b/flow/native.py @@ -82,8 +82,10 @@ def build(force: bool = False) -> Path: raise NotAvailable("no Swift toolchain — run: xcode-select --install") BUILD_DIR.mkdir(parents=True, exist_ok=True) try: - built = _run(["xcrun", "swiftc", "-O", "-o", str(BINARY), str(SOURCE)], - BUILD_TIMEOUT_SEC) + # `-parse-as-library` is required, not tuning: a single-file executable is + # parsed as a script, and `@main` cannot coexist with script mode. + built = _run(["xcrun", "swiftc", "-O", "-parse-as-library", + "-o", str(BINARY), str(SOURCE)], BUILD_TIMEOUT_SEC) except (OSError, subprocess.SubprocessError) as exc: raise NotAvailable(f"build failed: {exc}") from exc if built.returncode != 0 or not BINARY.exists(): diff --git a/native/flow_stt.swift b/native/flow_stt.swift index eabdc5b..63ea0b2 100644 --- a/native/flow_stt.swift +++ b/native/flow_stt.swift @@ -12,7 +12,13 @@ // download at all: the models belong to the OS. // // Build: -// swiftc -O -o flow-stt native/flow_stt.swift +// swiftc -O -parse-as-library -o flow-stt native/flow_stt.swift +// +// `-parse-as-library` is not optional and not decoration. A single-file executable is +// treated as a script, and `@main` and script mode are mutually exclusive — the +// compiler says so in as many words. The alternative was renaming this to `main.swift` +// and going back to top-level statements; the flag keeps the file named after what it +// is, and every caller that builds it passes the flag (`flow/native.py`, CI, the guide). // // Two ways to run it, and the first exists so the second is worth doing: // @@ -163,7 +169,10 @@ func serve(_ rec: SFSpeechRecognizer) { guard count > 0, count < 30 * Int(SAMPLE_RATE) * 60 else { return } guard let payload = readExactly(input, count * 4) else { return } var samples = [Float](repeating: 0, count: count) - samples.withUnsafeMutableBytes { dst in + // `_ =` because `withUnsafeMutableBytes` hands back whatever the closure + // returns, here `copyBytes`' byte count, and this leg builds with + // `-warnings-as-errors` so an ignored result is a failure rather than a note. + _ = samples.withUnsafeMutableBytes { dst in payload.copyBytes(to: dst.bindMemory(to: UInt8.self)) } let line = transcribe(rec, samples).replacingOccurrences(of: "\n", with: " ") diff --git a/tests/test_native.py b/tests/test_native.py index dd3e6df..e35e202 100644 --- a/tests/test_native.py +++ b/tests/test_native.py @@ -167,15 +167,25 @@ def test_the_environment_is_left_as_it_was_found(self): class TestItRefusesOffAMacWithAReason(unittest.TestCase): + """Pinned to a non-Mac platform rather than reading the runner's. + + These assert the *refusal*, and on a macOS runner there is nothing to refuse — it + would go and build a real binary instead, which is a different test and a slow one. + The suite runs on both platforms, so a test that means one thing on Windows and + another on macOS is a test that is only half run wherever it passes. + """ + def test_build_names_the_platform(self): - with self.assertRaises(native.NotAvailable) as caught: - native.build() + with mock.patch.object(sys, "platform", "win32"): + with self.assertRaises(native.NotAvailable) as caught: + native.build() self.assertIn("macOS", str(caught.exception)) def test_available_answers_false_and_why_rather_than_raising(self): # `available()` is called during startup. A raise there is a launch that dies # over a feature the machine was never going to have. - ok, why = native.available() + with mock.patch.object(sys, "platform", "win32"): + ok, why = native.available() self.assertFalse(ok) self.assertTrue(why) diff --git a/tests/test_pill.py b/tests/test_pill.py index efa5538..ce03761 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -551,7 +551,9 @@ def test_it_degrades_to_the_primary_work_area_rather_than_raising(self): # A repaint is not a place to handle a Win32 failure. `_NoHands` is this # module's own idea of "nothing happened", and the fallback has to land # somewhere drawable rather than at (0, 0) with no size. - with mock.patch.object(ui.ctypes, "windll", ui._NoHands()): + # `create=True` because `ctypes.windll` does not exist off Windows at all, and + # this test asserts the degradation that matters most *on* those platforms. + with mock.patch.object(ui.ctypes, "windll", ui._NoHands(), create=True): full, work = ui._pointer_monitor(1280, 720) self.assertEqual(full, work) self.assertGreater(full[2], full[0]) @@ -711,7 +713,7 @@ def test_the_desktop_is_every_monitor_and_not_the_primary_one(self): self.assertGreater(bottom, top) def test_it_degrades_to_the_screen_rather_than_raising(self): - with mock.patch.object(ui.ctypes, "windll", ui._NoHands()): + with mock.patch.object(ui.ctypes, "windll", ui._NoHands(), create=True): self.assertEqual(ui._virtual_desktop(1280, 720), (0, 0, 1280, 720)) def test_parking_moves_the_window_and_never_unmaps_it(self): From c8bd61aa274ba2422ab003e0c56de5e8333be114 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 23:22:27 -0500 Subject: [PATCH 06/41] Let auto use what is ready, and make naming the engine the thing that builds it CI found a real one, and it was not about CI: the macOS leg went from 35 seconds on main to 643 on this branch. Every launch on a Mac without Whisper models was compiling Swift and then sitting on `--probe` until it timed out, because the probe waits for an authorization dialog that a headless machine never answers. A user's first launch would have done exactly the same thing. A full minute of nothing before a pill appears, on a machine that asked for none of it, to decide whether to offer an engine it might not use. So `--engine auto` passes compile_if_missing=False and a ten second probe: it uses the helper if it is already built and says how to build it if not. `--engine native` still compiles, because somebody who typed it has asked for the wait. Once it has been run once, auto finds the binary from then on. A timeout is now named as the dialog it is rather than as "probe failed: TimeoutExpired", which is true and useless when the fix is a click. And every one of those reasons reaches `say()`, which writes to a cp437 console - so the em dashes in them were a launch that died on its own explanation. That is how this one was found; there is a test over the module's string literals now so the next one is caught here rather than by a user on Windows. Co-Authored-By: Claude Opus 5 --- flow/__main__.py | 5 ++- flow/native.py | 26 ++++++++++-- native/flow_stt.swift | 2 +- tests/test_native.py | 92 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 120 insertions(+), 5 deletions(-) diff --git a/flow/__main__.py b/flow/__main__.py index eee7d69..0524f04 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -182,7 +182,10 @@ def _engine(args, partial_name: str, final_name: str) -> tuple[str, str]: # auto. Ask the cheap question first: are the models here? if _models_present(partial_name, final_name): return "whisper", "" - ok, why = native_available() + # `compile_if_missing=False`, and a short probe. A launch is not the place to + # discover how slow `swiftc` is, and the probe blocks on a permission dialog that + # nobody has been shown yet — measured at a full minute per launch before this. + ok, why = native_available(compile_if_missing=False, timeout=10.0) if ok: return "native", " (whisper models not found on this machine)" return "whisper", f" (not found locally, and no native engine: {why})" diff --git a/flow/native.py b/flow/native.py index 46610a6..8402014 100644 --- a/flow/native.py +++ b/flow/native.py @@ -79,7 +79,7 @@ def build(force: bool = False) -> Path: except (OSError, subprocess.SubprocessError) as exc: raise NotAvailable(f"no Swift toolchain: {exc}") from exc if which.returncode != 0: - raise NotAvailable("no Swift toolchain — run: xcode-select --install") + raise NotAvailable("no Swift toolchain - run: xcode-select --install") BUILD_DIR.mkdir(parents=True, exist_ok=True) try: # `-parse-as-library` is required, not tuning: a single-file executable is @@ -96,7 +96,8 @@ def build(force: bool = False) -> Path: return BINARY -def available() -> tuple[bool, str]: +def available(compile_if_missing: bool = True, + timeout: float = 60.0) -> tuple[bool, str]: """`(usable, why not)` for this machine, without starting a session on it. Runs the helper's own `--probe`, which is the only honest check: the engine exists @@ -104,13 +105,32 @@ def available() -> tuple[bool, str]: which needs Dictation enabled so macOS has downloaded the offline model — and the user has granted the permission. Anything less is a guess that fails later, in the middle of somebody's first sentence. + + **`compile_if_missing=False` is what `--engine auto` uses, and it is not a + micro-optimisation.** Measured in CI: with this asked unconditionally at startup, the + macOS suite went from 35 seconds to 643. Every launch on a Mac without Whisper models + was compiling Swift and then sitting on `--probe` until the timeout, because the + probe waits for an authorization dialog that a headless machine never answers — and a + user's first launch would have done exactly the same thing, for a full minute, before + showing a pill. + + So the rule is: **`auto` uses what is ready, and asking for the engine by name is + what builds it.** Once `--engine native` has been run once the binary is there, and + `auto` finds it from then on. """ + if not compile_if_missing and sys.platform == "darwin" and not BINARY.exists(): + return False, ("not built yet - run once with --engine native to compile it") try: binary = build() except NotAvailable as exc: return False, str(exc) try: - probe = _run([str(binary), "--probe"], 60.0) + probe = _run([str(binary), "--probe"], timeout) + except subprocess.TimeoutExpired: + # Almost always the permission dialog, unanswered. Named as itself rather than + # as a generic failure, because the fix is a click and the user should hear so. + return False, ("probe timed out - grant Speech Recognition under " + "System Settings > Privacy & Security") except (OSError, subprocess.SubprocessError) as exc: return False, f"probe failed: {exc}" if probe.returncode != 0: diff --git a/native/flow_stt.swift b/native/flow_stt.swift index 63ea0b2..a8c5257 100644 --- a/native/flow_stt.swift +++ b/native/flow_stt.swift @@ -85,7 +85,7 @@ func makeRecognizer() -> SFSpeechRecognizer { // The whole point. Without this the audio goes to Apple's servers, which is a // different product from the one Flow is: local by construction. guard rec.supportsOnDeviceRecognition else { - die("on-device recognition unavailable — enable Dictation in System Settings " + die("on-device recognition unavailable - enable Dictation in System Settings " + "so macOS downloads the offline model", 5) } // Off the main queue, deliberately. `transcribe` waits on a semaphore for the diff --git a/tests/test_native.py b/tests/test_native.py index e35e202..01a6e3c 100644 --- a/tests/test_native.py +++ b/tests/test_native.py @@ -119,6 +119,98 @@ def _engine_result(): return _engine(args(), "base.en", "small.en") +class TestAutoNeverPaysForAnEngineItMayNotUse(unittest.TestCase): + """The regression CI found, pinned so it cannot come back. + + Asking `available()` unconditionally at startup took the macOS CI leg from **35 + seconds to 643**. Every launch on a Mac without Whisper models was compiling Swift + and then sitting on `--probe` until it timed out, because the probe waits for an + authorization dialog a headless machine never answers. + + A user's first launch would have done the same thing: a full minute of nothing + before a pill appeared, on a machine that had asked for none of it. So `auto` uses + what is *ready*, and naming the engine is what builds it. + """ + + def test_auto_will_not_compile_anything(self): + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["compile"] = compile_if_missing + seen["timeout"] = timeout + return False, "not built yet" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", fake): + _engine(args(), "base.en", "small.en") + self.assertFalse(seen["compile"]) + + def test_and_will_not_wait_a_minute_on_a_permission_dialog(self): + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["timeout"] = timeout + return False, "not built yet" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch("flow.__main__._models_present", return_value=False), \ + mock.patch.object(native, "available", fake): + _engine(args(), "base.en", "small.en") + self.assertLessEqual(seen["timeout"], 15.0) + + def test_naming_the_engine_is_what_builds_it(self): + # The other half of the rule. Somebody who typed `--engine native` has asked for + # the compile and is willing to wait for it. + seen = {} + + def fake(compile_if_missing=True, timeout=60.0): + seen["compile"] = compile_if_missing + return True, "" + + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(native, "available", fake): + self.assertEqual(_engine(args("native"), "base.en", "small.en")[0], + "native") + self.assertTrue(seen["compile"]) + + def test_an_unbuilt_helper_says_how_to_build_it(self): + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", return_value=False): + ok, why = native.available(compile_if_missing=False) + self.assertFalse(ok) + self.assertIn("--engine native", why) + + def test_a_probe_that_hangs_is_named_as_the_dialog_it_is(self): + # "probe failed: TimeoutExpired" is true and useless. The fix is a click, and + # the sentence should say so. + with mock.patch.object(sys, "platform", "darwin"), \ + mock.patch.object(Path, "exists", return_value=True), \ + mock.patch.object(native, "_run", + side_effect=subprocess.TimeoutExpired("probe", 10)): + ok, why = native.available(timeout=10.0) + self.assertFalse(ok) + self.assertIn("Speech Recognition", why) + + def test_every_reason_survives_the_console_the_startup_line_prints_to(self): + # `say()` writes to a cp437 console on Windows, and these strings reach it + # through `_engine`. An em dash here is a launch that dies on its own + # explanation — which is exactly how this was found. + for reason in ("not built yet", "no Swift toolchain", "probe timed out"): + with self.subTest(reason=reason): + pass + import inspect + + source = inspect.getsource(native) + for line in source.splitlines(): + stripped = line.strip() + if stripped.startswith("#") or '"' not in line: + continue + for chunk in line.split('"')[1::2]: + with self.subTest(chunk=chunk[:40]): + chunk.encode("cp437") + + class TestTheModelPresenceCheckNeverDownloads(unittest.TestCase): """It is asked *because* the network may be unusable; it must not use it.""" From 9f48a9d5f4675947bcd7336610a2f978f6e658fe Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 23:42:10 -0500 Subject: [PATCH 07/41] Ask a Mac which way of building a frameless window actually works Two guesses at Aqua have been spent already. The pill came back with a title bar and three traffic lights while the panels above it were correctly bare, and the difference between them is that `Pill` *is* the root window - `class Pill(tk.Tk)` - where Bubble and ConversationCard are Toplevels. Aqua builds the root's NSWindow before Tk can restyle it, so what works for a Toplevel need not work for `.`. Five windows, five techniques, labelled on screen. The answer is which ones have no title bar, which is a thing a person can see and this machine cannot. Co-Authored-By: Claude Opus 5 --- scripts/mac_frame_probe.py | 108 +++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 scripts/mac_frame_probe.py diff --git a/scripts/mac_frame_probe.py b/scripts/mac_frame_probe.py new file mode 100644 index 0000000..58afe43 --- /dev/null +++ b/scripts/mac_frame_probe.py @@ -0,0 +1,108 @@ +"""Which of five ways of asking Aqua for a frameless window actually works. + +Reported from a Mac: the pill sat there with a title bar and three traffic lights on it, +while the panels above it were correctly frameless. The difference between them is that +`Pill` **is** the root window — `class Pill(tk.Tk)` — and `Bubble`/`ConversationCard` are +`Toplevel`s. Aqua creates the root's `NSWindow` before Tk can restyle it, so what works +for a Toplevel does not necessarily work for `.`. + +Two wrong guesses have already been spent on this, so this asks the machine instead. +It opens five small windows in a row down the left of the screen, each labelled with the +technique that made it, and each *saying* whether it is a root window or a Toplevel. + + uv run python scripts/mac_frame_probe.py + +**Look at them and tell me which ones have no title bar.** That is the whole output; the +console text is only there to say what you are looking at. It closes itself after 20 +seconds. +""" + +import sys +import tkinter as tk +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +HOLD_SEC = 20 +W, H = 420, 64 + + +def style(win) -> str: + """`::tk::unsupported::MacWindowStyle`, reporting rather than assuming.""" + try: + win.tk.call("::tk::unsupported::MacWindowStyle", "style", + win._w, "plain", "noActivates") + return "ok" + except tk.TclError as exc: + return f"FAILED ({exc})" + + +def label(win, text: str, y: int) -> None: + win.geometry(f"{W}x{H}+40+{y}") + tk.Label(win, text=text, bg="#12141a", fg="#e6e8ee", + font=("Helvetica", 13)).pack(fill="both", expand=True) + win.configure(bg="#12141a") + + +def main() -> None: + notes = [] + + # 1. The root, styled the way Flow does it today. This is the one that came back + # with traffic lights on it. + root = tk.Tk() + root.overrideredirect(True) + root.attributes("-topmost", True) + notes.append(("1 root: overrideredirect then MacWindowStyle", style(root))) + label(root, "1 ROOT overrideredirect + MacWindowStyle", 60) + + # 2. The root again, but restyled while unmapped. Aqua builds the NSWindow when the + # window is first mapped, so a style asked for afterwards may never be applied to + # anything — withdrawing and remapping is the documented way to force a rebuild. + two = tk.Toplevel(root) # stands in for a root; see 3 for the real second root + two.withdraw() + two.overrideredirect(True) + notes.append(("2 toplevel: withdraw, overrideredirect, deiconify", style(two))) + label(two, "2 TOPLEVEL withdraw -> style -> deiconify", 150) + two.deiconify() + two.attributes("-topmost", True) + + # 3. A Toplevel with nothing but overrideredirect — what the bubble and card do, and + # what already works in the app. The control. + three = tk.Toplevel(root) + three.overrideredirect(True) + three.attributes("-topmost", True) + label(three, "3 TOPLEVEL overrideredirect only (the control)", 240) + + # 4. A Toplevel with the style and no overrideredirect, to find out which of the two + # is actually doing the work. + four = tk.Toplevel(root) + notes.append(("4 toplevel: MacWindowStyle only", style(four))) + label(four, "4 TOPLEVEL MacWindowStyle only", 330) + four.attributes("-topmost", True) + + # 5. The root, withdrawn and remapped. If this one is bare, the fix is three lines + # and `Pill` can stay a `tk.Tk`. If it is not, the pill has to become a Toplevel + # under a hidden root, which is a real change to how the app starts. + root.withdraw() + root.overrideredirect(True) + notes.append(("5 root: withdraw -> overrideredirect -> deiconify", style(root))) + root.deiconify() + root.attributes("-topmost", True) + root.geometry(f"{W}x{H}+40+60") + + print(f"platform {sys.platform}, tk {tk.TkVersion}\n") + for name, got in notes: + print(f" {name:<52} {got}") + print(f""" +Five windows are on screen now, down the left. Window 1 is the root as Flow builds +it today and window 5 is the same root after a withdraw/deiconify - if 5 is bare and +1 is not, the fix is three lines. If neither root is bare, `Pill` has to stop being +`tk.Tk`, which is a larger change. + +Say which numbers have no title bar. Closing in {HOLD_SEC}s.""") + root.after(HOLD_SEC * 1000, root.destroy) + root.mainloop() + + +if __name__ == "__main__": + main() From 4c62509cec6de88de242f4f757581528a08e0597 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Sun, 30 Aug 2026 23:51:36 -0500 Subject: [PATCH 08/41] Stop asking Aqua for a window class that has a title bar on it `MacWindowStyle plain` is not frameless, and believing it was is what put a title bar and three traffic lights on the pill. `plain` is a window *class*. A Mac running scripts/mac_frame_probe.py settled it: a Toplevel given only the style comes up decorated, one given only overrideredirect(True) comes up bare, and the call reports `ok` either way - it succeeds at doing the wrong thing. So this was a regression I introduced. Before it, `_no_activate` returned False off Windows and did nothing, and every window was bare because overrideredirect had already done the work. Applying the style afterwards put the frame back on. overrideredirect strips the frame here as it does everywhere else, and the style call stays for `noActivates` alone - the half that keeps a click on the pill from pulling focus off whatever is being dictated into. It is asked for first and the redirect re-asserted after, so the class can never win. The root needs it knocked off twice. Aqua builds the NSWindow when the window is first mapped and a redirect asked for afterwards does not restyle what already exists, which is why Bubble and ConversationCard were bare while the pill - which *is* the root, `class Pill(tk.Tk)` - was not. Withdrawing and remapping forces the rebuild, and only for a window already on screen: the panels are withdrawn when `_no_activate` runs over all three, and deiconifying them would put two empty surfaces in front of the user. Also from that probe: the Mac is on Tk 9.0, not 8.6. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 54 ++++++++++++++++++++++++++++++---------------- tests/test_pill.py | 39 +++++++++++++++++++++++++++------ 2 files changed, 67 insertions(+), 26 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 57b5edd..3722ca3 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -155,32 +155,48 @@ def toplevel_hwnd(win) -> int: def _mac_window_style(win) -> bool: - """Aqua's answer to `overrideredirect` + `WS_EX_NOACTIVATE`, in one call. - - `overrideredirect(True)` is what strips the title bar on Windows and X11. On Aqua it - is not enough on its own — reported from a real Mac as a pill sitting there with its - close and maximize buttons showing — because Tk maps a Toplevel to a real - `NSWindow` whose style mask is set from the *window class*, not from the redirect - flag. `MacWindowStyle` is the supported way to choose that class, and `plain` is the - one with no frame at all. - - `noActivates` is the second half and the more important one: it is Aqua's - `WS_EX_NOACTIVATE`. Without it, clicking the pill pulls focus away from whatever the - user is dictating into — which on Windows is the defect that made every paste land - in the wrong window, and there is no reason to ship it on a Mac having fixed it once - already. - - Named `::tk::unsupported::` by Tk itself, which is why this is wrapped and reported - rather than trusted: it is the interface Aqua actually offers and it may not be - there. A False here costs a title bar, not a session. + """Aqua's `WS_EX_NOACTIVATE`, without letting it put a title bar back on. + + **`MacWindowStyle plain` is not frameless, and believing it was is what put a title + bar and three traffic lights on the pill.** `plain` is a window *class*, and + `scripts/mac_frame_probe.py` settled what it looks like on a real Mac: a Toplevel + given the style and nothing else comes up decorated, while one given only + `overrideredirect(True)` comes up bare. The call reports `ok` either way — it + succeeds at doing the wrong thing. + + So `overrideredirect` is what strips the frame here as it does everywhere else, and + this exists for `noActivates` alone — the half that keeps a click on the pill from + pulling focus off whatever is being dictated into. It is asked for *first* and the + redirect is re-asserted after, so the class can never win. + + **The root window needs the frame knocked off twice.** Aqua builds its `NSWindow` + when it is first mapped, and a redirect asked for afterwards does not restyle what + already exists — which is why `Bubble` and `ConversationCard` were bare while the + pill, which *is* the root (`class Pill(tk.Tk)`), was not. Withdrawing and remapping + forces the rebuild. Only for a window that is already on screen: the panels are + withdrawn at this point in startup and deiconifying them would put two empty + surfaces in front of the user. + + Named `::tk::unsupported::` by Tk itself, which is why this is wrapped rather than + trusted. A False costs the focus behaviour, not a session. """ + styled = True try: win.update_idletasks() win.tk.call("::tk::unsupported::MacWindowStyle", "style", win._w, "plain", "noActivates") - return True + except tk.TclError: + styled = False + try: + if win.winfo_ismapped(): + win.withdraw() + win.overrideredirect(True) + win.deiconify() + else: + win.overrideredirect(True) except tk.TclError: return False + return styled def _no_activate(win) -> bool: diff --git a/tests/test_pill.py b/tests/test_pill.py index ce03761..b471600 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -832,27 +832,52 @@ class TestTheMacWindowStyle(unittest.TestCase): to an `NSWindow` whose style comes from the window *class*. """ - def test_it_asks_for_a_plain_window_that_does_not_activate(self): - # Both halves matter. `plain` is the frameless class; `noActivates` is what - # stops a click on the pill pulling focus off whatever is being dictated into — - # the defect that made every paste land in the wrong window on Windows. + def test_it_asks_for_no_activates_and_then_takes_the_frame_off_again(self): + # The order is the fix. `plain` is a window *class* and it is **not** frameless — + # a Mac showed a Toplevel given only the style coming up decorated. So the + # redirect is re-asserted after the style, and the class can never win. win = mock.Mock() + win.winfo_ismapped.return_value = False self.assertTrue(ui._mac_window_style(win)) args = win.tk.call.call_args[0] self.assertEqual(args[0], "::tk::unsupported::MacWindowStyle") - self.assertIn("plain", args) self.assertIn("noActivates", args) + win.overrideredirect.assert_called_once_with(True) - def test_an_unsupported_build_costs_a_title_bar_and_not_a_session(self): - # Tk calls the interface `::tk::unsupported::` itself, so it may not be there. + def test_a_window_already_on_screen_is_remapped_to_force_the_rebuild(self): + # The root window's NSWindow is built when it is first mapped, and a redirect + # asked for afterwards does not restyle what already exists. That is why the + # panels were bare and the pill — which *is* the root — was not. win = mock.Mock() + win.winfo_ismapped.return_value = True + ui._mac_window_style(win) + win.withdraw.assert_called_once() + win.deiconify.assert_called_once() + + def test_a_withdrawn_window_is_not_put_on_screen_by_the_fix(self): + # `_no_activate` runs over all three windows at startup, and the panels are + # withdrawn at that point. Deiconifying them would put two empty surfaces in + # front of the user. + win = mock.Mock() + win.winfo_ismapped.return_value = False + ui._mac_window_style(win) + win.withdraw.assert_not_called() + win.deiconify.assert_not_called() + + def test_the_frame_still_comes_off_when_the_style_call_is_unavailable(self): + # Tk calls it `::tk::unsupported::` itself. Losing it should cost the focus + # behaviour, not put a title bar back on the pill. + win = mock.Mock() + win.winfo_ismapped.return_value = False win.tk.call.side_effect = ui.tk.TclError("no such command") self.assertFalse(ui._mac_window_style(win)) + win.overrideredirect.assert_called_once_with(True) def test_no_activate_routes_to_it_on_a_mac_and_refuses_elsewhere(self): # One name for "take this window out of the activation chain", answered by # whichever platform API can actually do it. win = mock.Mock() + win.winfo_ismapped.return_value = False with mock.patch.object(ui.sys, "platform", "darwin"): self.assertTrue(ui._no_activate(win)) win.tk.call.assert_called_once() From 544e4d0e0bbbb93108a4e488339bb8afb4efb28e Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 00:04:51 -0500 Subject: [PATCH 09/41] Check that the maximised probe was actually maximised A Mac put the pill in the top-left corner, and the reason was a sanity check that only asked "is this a rectangle". `state("zoomed")` on Aqua does not raise and does not maximise either - it is accepted and ignored. So the probe stayed the 200x120 it was asked for, at +80+80, and (80, 80, 280, 200) passed a test that wanted a positive rectangle no bigger than the screen. The stack was then placed against a work area 200 px wide, which on a 1512-wide display is the top-left corner. Exactly where it was found. The test is not "is this a rectangle" but "did the window manager do the thing", and it is asked two ways because either alone has a hole. A window that never grew past what it asked for is the direct evidence. A rectangle far smaller than the display catches a window manager that grew it a little and stopped - a real work area is the screen minus a Dock or a taskbar, nowhere near half of it. Still exact against SystemParametersInfoW on Windows, where zoomed does work. Also: the remap that takes the frame off the root now puts the geometry back. A window that comes back from a withdraw has been placed by whoever remapped it, not by whoever positioned it. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 36 +++++++++++++++++++++++++++++++++--- tests/test_pill.py | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 3 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 3722ca3..702e531 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -189,9 +189,14 @@ def _mac_window_style(win) -> bool: styled = False try: if win.winfo_ismapped(): + # Remembered across the remap. A window that comes back has been placed by + # whoever remapped it, not by whoever positioned it, and losing that is a + # pill somewhere the user did not put it. + where = win.geometry() win.withdraw() win.overrideredirect(True) win.deiconify() + win.geometry(where) else: win.overrideredirect(True) except tk.TclError: @@ -301,11 +306,12 @@ def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: if _TK_WORK is not None: return _TK_WORK fallback = (0, 0, sw, sh) + asked_w, asked_h = 200, 120 probe = None try: probe = tk.Toplevel(win) probe.attributes("-alpha", 0.0) - probe.geometry("200x120+80+80") + probe.geometry(f"{asked_w}x{asked_h}+80+80") probe.state("zoomed") probe.update_idletasks() x, y = probe.winfo_rootx(), probe.winfo_rooty() @@ -322,13 +328,37 @@ def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: probe.destroy() except tk.TclError: pass - if not (found[2] > found[0] and found[3] > found[1] - and found[2] - found[0] <= sw and found[3] - found[1] <= sh): + if not _plausible_work_area(found, sw, sh, asked_w, asked_h): found = fallback _TK_WORK = found return found +def _plausible_work_area(rect, sw: int, sh: int, asked_w: int, asked_h: int) -> bool: + """Whether a maximised probe actually got maximised. + + **The check that was missing, and the bug it let through.** On Aqua, + `state("zoomed")` does not raise and does not maximise either — it is accepted and + ignored. The probe stayed the 200x120 it was asked for, that rectangle passed a + check which only asked "positive, and no bigger than the screen", and the pill was + placed against a work area 200 px wide. It landed in the top-left corner of a + 1512-wide display, which is exactly where a Mac reported finding it. + + So the test is not "is this a rectangle" but "did the window manager do the thing". + Two ways of asking, because either alone has a hole: a window that never grew is the + direct evidence, and a rectangle far smaller than the display is what catches a + window manager that grew it a little and stopped. A real work area is the screen + minus a Dock or a taskbar — nowhere near half of it. + """ + left, top, right, bottom = rect + w, h = right - left, bottom - top + if not (w > 0 and h > 0 and w <= sw and h <= sh): + return False + if w <= asked_w or h <= asked_h: + return False # it never grew: `zoomed` was accepted and ignored + return w >= sw * 0.6 and h >= sh * 0.6 + + #: `MonitorFromPoint`'s "nearest monitor" flag, for a cursor that is briefly nowhere — #: between two displays, or on a monitor that has just been unplugged. _MONITOR_DEFAULTTONEAREST = 2 diff --git a/tests/test_pill.py b/tests/test_pill.py index b471600..909859f 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -814,6 +814,38 @@ def test_a_nonsense_measurement_is_refused(self): self.assertEqual(ui._tk_work_area(mock.Mock(), 1280, 720), (0, 0, 1280, 720)) + def test_a_probe_that_was_never_maximised_is_refused(self): + # The one that got through, and put the pill in the top-left corner of a Mac. + # `state("zoomed")` on Aqua does not raise and does not maximise either: it is + # accepted and ignored, so the probe stayed the 200x120 it was asked for and + # that rectangle was believed. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((80, 80, 280, 200))): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1512, 982), + (0, 0, 1512, 982)) + + def test_a_window_grown_only_a_little_is_refused_too(self): + # The other hole: a window manager that honoured `zoomed` partially. A real work + # area is the screen minus a Dock or a taskbar, nowhere near half of it. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((0, 0, 700, 400))): + self.assertEqual(ui._tk_work_area(mock.Mock(), 1512, 982), + (0, 0, 1512, 982)) + + def test_the_fallback_still_puts_the_stack_on_screen(self): + # Refusing the measurement must not mean refusing to place anything. The whole + # screen is wrong by a Dock; the top-left corner is wrong by a screen. + ui._TK_WORK = None + with mock.patch.object(ui.tk, "Toplevel", + return_value=self.fake_win((80, 80, 280, 200))): + work = ui._tk_work_area(mock.Mock(), 1512, 982) + x, y = ui.bottom_centre(ui.PILL_W, ui.PILL_H, (0, 0, 1512, 982), work, + ui.PANEL_BOTTOM_OFFSET) + self.assertGreater(y, 982 * 0.8, "the stack is nowhere near the bottom") + self.assertGreater(x, 1512 * 0.3, "the stack is nowhere near the centre") + def test_the_bottom_edge_is_the_one_that_has_to_be_right(self): # It is what bottom-centre placement stands on, and getting it wrong is the # whole bug: 672 here against a screen of 720 is a 48 px Dock found. From b7ab288d06d42d1c53f5fc28d88d63920147e12a Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 00:11:46 -0500 Subject: [PATCH 10/41] Stop taking the windows out of the activation chain, which is where Send went Clicking Send stopped working on a Mac, and it was the second half of the same mistake as the title bar. `MacWindowStyle plain noActivates` asks Aqua for a window that never activates - and a window that never activates does not take clicks either. The app already said so. `_menu` borrows the foreground on Windows precisely because a WS_EX_NOACTIVATE window would get no input for its popup, and the comment there states that Lite needs none of that: "`_no_activate` cannot take, so the window is in the activation chain like any other and the popup gets its input the ordinary way." That was an invariant, and asking Aqua for the equivalent broke it. So `_no_activate` refuses off Windows again, as it did before any of this, and the frame is handled where it always belonged - beside the `overrideredirect` call in `_shell_window`. `_mac_reframe` withdraws and remaps a window that is already on screen, which is what makes the redirect reach an NSWindow Aqua had already built. That is the whole of the root-window fix; nothing asks for a window class any more. The test for it asserts over the source rather than a mock, because the mistake was *making the call*, and a mock cannot notice a call that is no longer there. The name still appears in prose explaining why it is not used - those comments are the point, so what is checked is `::tk::unsupported::`, the only form that can do damage. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 101 ++++++++++++++++------------------ tests/test_pill.py | 131 ++++++++++++++++++++++++++------------------- 2 files changed, 123 insertions(+), 109 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 702e531..9633bbf 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -154,56 +154,6 @@ def toplevel_hwnd(win) -> int: return _user32.GetParent(win.winfo_id()) or 0 -def _mac_window_style(win) -> bool: - """Aqua's `WS_EX_NOACTIVATE`, without letting it put a title bar back on. - - **`MacWindowStyle plain` is not frameless, and believing it was is what put a title - bar and three traffic lights on the pill.** `plain` is a window *class*, and - `scripts/mac_frame_probe.py` settled what it looks like on a real Mac: a Toplevel - given the style and nothing else comes up decorated, while one given only - `overrideredirect(True)` comes up bare. The call reports `ok` either way — it - succeeds at doing the wrong thing. - - So `overrideredirect` is what strips the frame here as it does everywhere else, and - this exists for `noActivates` alone — the half that keeps a click on the pill from - pulling focus off whatever is being dictated into. It is asked for *first* and the - redirect is re-asserted after, so the class can never win. - - **The root window needs the frame knocked off twice.** Aqua builds its `NSWindow` - when it is first mapped, and a redirect asked for afterwards does not restyle what - already exists — which is why `Bubble` and `ConversationCard` were bare while the - pill, which *is* the root (`class Pill(tk.Tk)`), was not. Withdrawing and remapping - forces the rebuild. Only for a window that is already on screen: the panels are - withdrawn at this point in startup and deiconifying them would put two empty - surfaces in front of the user. - - Named `::tk::unsupported::` by Tk itself, which is why this is wrapped rather than - trusted. A False costs the focus behaviour, not a session. - """ - styled = True - try: - win.update_idletasks() - win.tk.call("::tk::unsupported::MacWindowStyle", "style", - win._w, "plain", "noActivates") - except tk.TclError: - styled = False - try: - if win.winfo_ismapped(): - # Remembered across the remap. A window that comes back has been placed by - # whoever remapped it, not by whoever positioned it, and losing that is a - # pill somewhere the user did not put it. - where = win.geometry() - win.withdraw() - win.overrideredirect(True) - win.deiconify() - win.geometry(where) - else: - win.overrideredirect(True) - except tk.TclError: - return False - return styled - - def _no_activate(win) -> bool: """Take `win` out of the activation chain, and report whether it took. @@ -212,11 +162,18 @@ def _no_activate(win) -> bool: number, and there is no other way to tell them apart. The one thing this window style has to be is true. - Off Windows this is `_mac_window_style`'s job, which does both halves at once - because Aqua sets the frame and the activation policy from the same window class. + **Off Windows this cannot take, and the app is built on that.** `_menu` borrows the + foreground on Windows precisely because a `WS_EX_NOACTIVATE` window would otherwise + get no input for its popup, and says in as many words that Lite needs none of it — + the window is in the activation chain like any other and the popup gets its input the + ordinary way. + + Aqua does offer an equivalent, `MacWindowStyle ... noActivates`, and asking for it + was a mistake: it took the windows out of that chain, and a window that never + activates does not take clicks either. Send stopped working on a Mac. The frame is + handled in `_shell_window` now, where `overrideredirect` already lives and where it + always belonged. """ - if sys.platform == "darwin": - return _mac_window_style(win) if sys.platform != "win32": return False try: @@ -243,6 +200,7 @@ def _shell_window(win, lite: bool, alpha: float) -> str: one that contradicts what was applied to it. """ win.overrideredirect(True) + _mac_reframe(win) win.attributes("-topmost", True) win.attributes("-alpha", alpha) if lite: @@ -252,6 +210,41 @@ def _shell_window(win, lite: bool, alpha: float) -> str: return TRANSPARENT +def _mac_reframe(win) -> None: + """Make Aqua build the window again, so `overrideredirect` reaches its NSWindow. + + Reported from a Mac: the pill sat there with a title bar and three traffic lights + while the panels above it were correctly bare. The difference is that `Pill` **is** + the root window — `class Pill(tk.Tk)` — and the panels are Toplevels. Aqua builds the + root's NSWindow when it is first mapped, which has already happened by the time + anything asks for a redirect, and a redirect does not restyle a window that exists. + Withdrawing and remapping forces the rebuild. + + **Not `MacWindowStyle`.** `plain` sounds frameless and is not — a Toplevel given only + that style comes up decorated, which `scripts/mac_frame_probe.py` settled on a real + machine. Its `noActivates` half was worse: it took the windows out of the activation + chain, and a window that never activates does not take clicks, so Send stopped + working. `_no_activate` says why the whole app depends on that not happening here. + + Only for a window already on screen. The panels are withdrawn when this runs and + deiconifying them would put two empty surfaces in front of the user. Geometry is put + back because a window returning from a withdraw has been placed by whoever remapped + it, not by whoever positioned it. + """ + if sys.platform != "darwin": + return + try: + if not win.winfo_ismapped(): + return + where = win.geometry() + win.withdraw() + win.overrideredirect(True) + win.deiconify() + win.geometry(where) + except tk.TclError: + pass + + def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: """The desktop minus the taskbar, in the same pixels `geometry` uses. diff --git a/tests/test_pill.py b/tests/test_pill.py index 909859f..7304ab7 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -856,65 +856,86 @@ def test_the_bottom_edge_is_the_one_that_has_to_be_right(self): self.assertLessEqual(y + ui.PILL_H, 672) -class TestTheMacWindowStyle(unittest.TestCase): - """Aqua's `overrideredirect` + `WS_EX_NOACTIVATE`, which is one call and unsupported. +class TestTheMacFrame(unittest.TestCase): + """Stripping the root window's frame on Aqua, and what must not be asked for to do it. - Reported from a real Mac: the pill sat there with its close and maximize buttons - showing. `overrideredirect(True)` is not enough on Aqua, because Tk maps a Toplevel - to an `NSWindow` whose style comes from the window *class*. + Reported from a Mac: the pill had a title bar and three traffic lights while the + panels above it were bare. `Pill` **is** the root (`class Pill(tk.Tk)`), the panels + are Toplevels, and Aqua builds the root's NSWindow before anything asks for a + redirect. + + Two wrong answers were tried before this one, and both are asserted against here + because both looked right: `MacWindowStyle plain` is not frameless, and its + `noActivates` half broke clicking Send. """ - def test_it_asks_for_no_activates_and_then_takes_the_frame_off_again(self): - # The order is the fix. `plain` is a window *class* and it is **not** frameless — - # a Mac showed a Toplevel given only the style coming up decorated. So the - # redirect is re-asserted after the style, and the class can never win. - win = mock.Mock() - win.winfo_ismapped.return_value = False - self.assertTrue(ui._mac_window_style(win)) - args = win.tk.call.call_args[0] - self.assertEqual(args[0], "::tk::unsupported::MacWindowStyle") - self.assertIn("noActivates", args) - win.overrideredirect.assert_called_once_with(True) - - def test_a_window_already_on_screen_is_remapped_to_force_the_rebuild(self): - # The root window's NSWindow is built when it is first mapped, and a redirect - # asked for afterwards does not restyle what already exists. That is why the - # panels were bare and the pill — which *is* the root — was not. - win = mock.Mock() - win.winfo_ismapped.return_value = True - ui._mac_window_style(win) - win.withdraw.assert_called_once() - win.deiconify.assert_called_once() - - def test_a_withdrawn_window_is_not_put_on_screen_by_the_fix(self): - # `_no_activate` runs over all three windows at startup, and the panels are - # withdrawn at that point. Deiconifying them would put two empty surfaces in - # front of the user. - win = mock.Mock() - win.winfo_ismapped.return_value = False - ui._mac_window_style(win) - win.withdraw.assert_not_called() - win.deiconify.assert_not_called() - - def test_the_frame_still_comes_off_when_the_style_call_is_unavailable(self): - # Tk calls it `::tk::unsupported::` itself. Losing it should cost the focus - # behaviour, not put a title bar back on the pill. - win = mock.Mock() - win.winfo_ismapped.return_value = False - win.tk.call.side_effect = ui.tk.TclError("no such command") - self.assertFalse(ui._mac_window_style(win)) - win.overrideredirect.assert_called_once_with(True) - - def test_no_activate_routes_to_it_on_a_mac_and_refuses_elsewhere(self): - # One name for "take this window out of the activation chain", answered by - # whichever platform API can actually do it. - win = mock.Mock() - win.winfo_ismapped.return_value = False + def win(self, mapped=True): + w = mock.Mock() + w.winfo_ismapped.return_value = mapped + w.geometry.return_value = "205x40+538+608" + return w + + def test_a_mapped_window_is_remapped_to_force_the_rebuild(self): + w = self.win() + with mock.patch.object(ui.sys, "platform", "darwin"): + ui._mac_reframe(w) + w.withdraw.assert_called_once() + w.overrideredirect.assert_called_once_with(True) + w.deiconify.assert_called_once() + + def test_the_position_survives_the_remap(self): + # A window that comes back has been placed by whoever remapped it, not by + # whoever positioned it. + w = self.win() + with mock.patch.object(ui.sys, "platform", "darwin"): + ui._mac_reframe(w) + w.geometry.assert_called_with("205x40+538+608") + + def test_a_withdrawn_window_is_not_put_on_screen_by_it(self): + # The panels are withdrawn when this runs at startup. Deiconifying them would + # put two empty surfaces in front of the user. + w = self.win(mapped=False) with mock.patch.object(ui.sys, "platform", "darwin"): - self.assertTrue(ui._no_activate(win)) - win.tk.call.assert_called_once() - with mock.patch.object(ui.sys, "platform", "linux"): - self.assertFalse(ui._no_activate(mock.Mock())) + ui._mac_reframe(w) + w.withdraw.assert_not_called() + w.deiconify.assert_not_called() + + def test_it_does_nothing_at_all_off_a_mac(self): + w = self.win() + for platform in ("win32", "linux"): + with self.subTest(platform=platform): + w.reset_mock() + with mock.patch.object(ui.sys, "platform", platform): + ui._mac_reframe(w) + w.withdraw.assert_not_called() + + def test_nothing_asks_aqua_for_a_window_class(self): + # `MacWindowStyle plain` sounds frameless and is not - a Toplevel given only + # that style comes up decorated, which a real Mac settled. Asserted over the + # source because the mistake is *making the call at all*, and a mock cannot + # notice a call that is no longer there. + # + # The name still appears twice in prose, explaining why it is not used. Those + # comments are the point of this test, not a violation of it - so what is + # checked is the invocation, which is the only form that can do damage. + source = (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") + for line in source.splitlines(): + stripped = line.strip() + if stripped.startswith("#") or stripped.startswith("*"): + continue + with self.subTest(line=stripped[:60]): + self.assertNotIn("::tk::unsupported::", line) + + def test_no_activate_refuses_off_windows_because_the_app_depends_on_it(self): + # `_menu` borrows the foreground on Windows precisely because a non-activating + # window gets no input for its popup, and says Lite needs none of that. Asking + # Aqua for `noActivates` took the windows out of the activation chain and Send + # stopped taking clicks. + for platform in ("darwin", "linux"): + with self.subTest(platform=platform): + with mock.patch.object(ui.sys, "platform", platform): + self.assertFalse(ui._no_activate(mock.Mock())) class TestMix(unittest.TestCase): From 6a54c2dc701308c696238e04ff67c0efb7054f83 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 00:24:01 -0500 Subject: [PATCH 11/41] Take the remap back out; it was fixing a problem the style call had created A Mac showed the window and then hid it. That was the withdraw-and-remap cycle added to "force Aqua to rebuild the NSWindow" - and it existed only because MacWindowStyle had put a frame back on, which is a thing that call does and overrideredirect had never failed to prevent. Re-reading the probe with that in mind says so plainly. Window 4, the style asked for on a *mapped* window, came up decorated. Window 2, the same style asked for while the window was withdrawn, came up bare. The app was case 4: overrideredirect early, style later, on a window long since mapped. The root was never the problem and `class Pill(tk.Tk)` was a red herring. So both attempts are gone and what is left is the line that had been doing the work all along. Three commits to arrive back at one call, with two ways of making it worse now written down. The tests assert the absence rather than a mechanism, which is the honest shape for this: there is nothing new to exercise, only two things that must not come back. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 45 +++++----------------- tests/test_pill.py | 95 +++++++++++++++++----------------------------- 2 files changed, 43 insertions(+), 97 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 9633bbf..12f6735 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -196,11 +196,19 @@ def _shell_window(win, lite: bool, alpha: float) -> str: rectangle where the app should be — and `-toolwindow` does not exist off Windows at all. Asking for either is a `TclError` before anything is drawn. + **`overrideredirect` is the whole of it on Aqua too, and two attempts to help it were + both harm.** A Mac reported the pill wearing a title bar, and the cause was not this + line failing — it was `MacWindowStyle` being asked for *afterwards*, which put a + frame back on and took the window out of the activation chain, so Send stopped taking + clicks. Removing that call was the fix. A withdraw-and-remap cycle added alongside it, + to "force Aqua to rebuild the NSWindow", was solving a problem the style call had + created — and on a real machine it left the window hidden after the remap. Both are + gone. What is left is the line that was always doing the work. + The background is returned rather than left to the caller so a window cannot be given one that contradicts what was applied to it. """ win.overrideredirect(True) - _mac_reframe(win) win.attributes("-topmost", True) win.attributes("-alpha", alpha) if lite: @@ -210,41 +218,6 @@ def _shell_window(win, lite: bool, alpha: float) -> str: return TRANSPARENT -def _mac_reframe(win) -> None: - """Make Aqua build the window again, so `overrideredirect` reaches its NSWindow. - - Reported from a Mac: the pill sat there with a title bar and three traffic lights - while the panels above it were correctly bare. The difference is that `Pill` **is** - the root window — `class Pill(tk.Tk)` — and the panels are Toplevels. Aqua builds the - root's NSWindow when it is first mapped, which has already happened by the time - anything asks for a redirect, and a redirect does not restyle a window that exists. - Withdrawing and remapping forces the rebuild. - - **Not `MacWindowStyle`.** `plain` sounds frameless and is not — a Toplevel given only - that style comes up decorated, which `scripts/mac_frame_probe.py` settled on a real - machine. Its `noActivates` half was worse: it took the windows out of the activation - chain, and a window that never activates does not take clicks, so Send stopped - working. `_no_activate` says why the whole app depends on that not happening here. - - Only for a window already on screen. The panels are withdrawn when this runs and - deiconifying them would put two empty surfaces in front of the user. Geometry is put - back because a window returning from a withdraw has been placed by whoever remapped - it, not by whoever positioned it. - """ - if sys.platform != "darwin": - return - try: - if not win.winfo_ismapped(): - return - where = win.geometry() - win.withdraw() - win.overrideredirect(True) - win.deiconify() - win.geometry(where) - except tk.TclError: - pass - - def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: """The desktop minus the taskbar, in the same pixels `geometry` uses. diff --git a/tests/test_pill.py b/tests/test_pill.py index 7304ab7..adba3f4 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -857,86 +857,59 @@ def test_the_bottom_edge_is_the_one_that_has_to_be_right(self): class TestTheMacFrame(unittest.TestCase): - """Stripping the root window's frame on Aqua, and what must not be asked for to do it. + """What strips the frame on Aqua, asserted as the absence of two wrong answers. - Reported from a Mac: the pill had a title bar and three traffic lights while the - panels above it were bare. `Pill` **is** the root (`class Pill(tk.Tk)`), the panels - are Toplevels, and Aqua builds the root's NSWindow before anything asks for a - redirect. + A Mac reported the pill wearing a title bar and three traffic lights while the + panels above it were bare, and two fixes were tried before the right one. Both are + pinned here because both looked correct and each broke something else: - Two wrong answers were tried before this one, and both are asserted against here - because both looked right: `MacWindowStyle plain` is not frameless, and its - `noActivates` half broke clicking Send. + **`MacWindowStyle plain` is not frameless.** `plain` is a window *class*, and a + Toplevel given only that style comes up decorated - settled on a real machine by + `scripts/mac_frame_probe.py`. Asked for on a mapped window it put the frame back. + + **`noActivates` is worse.** It takes the window out of the activation chain, and a + window that never activates does not take clicks: Send stopped working. `_menu` + already depended on the opposite, and says so. + + What was left after removing both is the line that had been doing the work all + along, which is why this class asserts an absence rather than a mechanism. """ - def win(self, mapped=True): - w = mock.Mock() - w.winfo_ismapped.return_value = mapped - w.geometry.return_value = "205x40+538+608" - return w - - def test_a_mapped_window_is_remapped_to_force_the_rebuild(self): - w = self.win() - with mock.patch.object(ui.sys, "platform", "darwin"): - ui._mac_reframe(w) - w.withdraw.assert_called_once() - w.overrideredirect.assert_called_once_with(True) - w.deiconify.assert_called_once() - - def test_the_position_survives_the_remap(self): - # A window that comes back has been placed by whoever remapped it, not by - # whoever positioned it. - w = self.win() - with mock.patch.object(ui.sys, "platform", "darwin"): - ui._mac_reframe(w) - w.geometry.assert_called_with("205x40+538+608") - - def test_a_withdrawn_window_is_not_put_on_screen_by_it(self): - # The panels are withdrawn when this runs at startup. Deiconifying them would - # put two empty surfaces in front of the user. - w = self.win(mapped=False) - with mock.patch.object(ui.sys, "platform", "darwin"): - ui._mac_reframe(w) - w.withdraw.assert_not_called() - w.deiconify.assert_not_called() - - def test_it_does_nothing_at_all_off_a_mac(self): - w = self.win() - for platform in ("win32", "linux"): - with self.subTest(platform=platform): - w.reset_mock() - with mock.patch.object(ui.sys, "platform", platform): - ui._mac_reframe(w) - w.withdraw.assert_not_called() + def source(self) -> str: + return (Path(__file__).resolve().parent.parent + / "flow" / "ui.py").read_text(encoding="utf-8") def test_nothing_asks_aqua_for_a_window_class(self): - # `MacWindowStyle plain` sounds frameless and is not - a Toplevel given only - # that style comes up decorated, which a real Mac settled. Asserted over the - # source because the mistake is *making the call at all*, and a mock cannot - # notice a call that is no longer there. - # - # The name still appears twice in prose, explaining why it is not used. Those - # comments are the point of this test, not a violation of it - so what is - # checked is the invocation, which is the only form that can do damage. - source = (Path(__file__).resolve().parent.parent - / "flow" / "ui.py").read_text(encoding="utf-8") - for line in source.splitlines(): + # Asserted over the source because the mistake is *making the call at all*, and + # a mock cannot notice a call that is no longer there. The name still appears in + # prose explaining why it is not used - those comments are the point of this + # test, so what is checked is the invocation. + for line in self.source().splitlines(): stripped = line.strip() if stripped.startswith("#") or stripped.startswith("*"): continue with self.subTest(line=stripped[:60]): self.assertNotIn("::tk::unsupported::", line) + def test_no_window_is_withdrawn_and_remapped_to_restyle_it(self): + # The second wrong answer. It was solving a problem the first one created, and + # on a real Mac it left the window hidden after the remap - shown, then gone. + self.assertNotIn("_mac_reframe", self.source()) + def test_no_activate_refuses_off_windows_because_the_app_depends_on_it(self): # `_menu` borrows the foreground on Windows precisely because a non-activating - # window gets no input for its popup, and says Lite needs none of that. Asking - # Aqua for `noActivates` took the windows out of the activation chain and Send - # stopped taking clicks. + # window gets no input for its popup, and states that Lite needs none of that. + # Asking Aqua for the equivalent broke Send. for platform in ("darwin", "linux"): with self.subTest(platform=platform): with mock.patch.object(ui.sys, "platform", platform): self.assertFalse(ui._no_activate(mock.Mock())) + def test_the_shell_still_asks_for_the_one_thing_that_works(self): + win = mock.Mock() + ui._shell_window(win, lite=True, alpha=0.94) + win.overrideredirect.assert_called_once_with(True) + class TestMix(unittest.TestCase): def test_the_ends_are_exact(self): From 8a758df3565ec70877ddef6be0da413bdcdd36f2 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 00:31:32 -0500 Subject: [PATCH 12/41] Put the picture and the numbers in one file, so a Mac question costs one round trip Diagnosing this platform from a Windows machine has been costing three messages a question - send a photo, now the log, now run the other probe - and a phone photo of a screen cannot answer "is it 24 px above the Dock or 240". `scripts/mac_report.py` drives the real windows, screenshots them, renders the numbers beside them and writes one PNG carrying both. Platform and Tk build, what each work-area method answers, where the stack is therefore placed, how far off centre it is, whether the native engine is ready. A neutral backdrop goes behind it first. `shots.py` does that to keep captures clean; here it is also a privacy line, because the whole point of this file is that the image gets sent to somebody and a full-screen grab of a working machine carries every window that happened to be open on it. The crop is read off the windows rather than reserved with a margin. The first version guessed 320 px above the pill and 60 below, which cropped in the taskbar and cut off the pill's own bottom edge - which is exactly where a title bar or a broken dock seam shows. A black capture is named as the missing Screen Recording permission it is, rather than looking like a bug in Flow. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 16 +++ scripts/mac_report.py | 238 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 254 insertions(+) create mode 100644 scripts/mac_report.py diff --git a/docs/guide.md b/docs/guide.md index 6bfd66b..4123dd1 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -592,6 +592,22 @@ comes back unbiased. Those are real differences, and changing what Flow hears wi being asked would be the wrong default. The startup line always says which engine you got and why. +**When something looks wrong on a Mac**, one command answers it: + +```bash +uv run --with pillow python scripts/mac_report.py +``` + +It drives the real windows, screenshots them against a neutral backdrop, renders the +numbers beside them, and writes a single `~/flow-mac-report.png` — platform and Tk build, +what each work-area method answers, where the stack is therefore placed and by how much +it misses centre, and whether the native engine is ready. One file, and the geometry in +it is exact rather than a photo of a screen. + +macOS asks for **Screen Recording** the first time, because that is the permission a +screenshot of other windows needs. The report says so out loud if the capture came back +black, rather than leaving you with an empty picture. + Before wiring it into anything, you can judge it on your own voice: ```bash diff --git a/scripts/mac_report.py b/scripts/mac_report.py new file mode 100644 index 0000000..6efcd73 --- /dev/null +++ b/scripts/mac_report.py @@ -0,0 +1,238 @@ +"""Everything I need to see about Flow on a Mac, in one PNG. + +Diagnosing this platform from a Windows machine has cost several rounds of "send me a +photo, now send me the log, now run the other probe". This collapses that: it drives the +real windows, screenshots them, renders the numbers beside them, and writes **one image** +carrying both. One file to send back, and the geometry in it is pixel-exact rather than +a phone photo of a screen at an angle. + + uv run --with pillow python scripts/mac_report.py + +Pillow is a `--with`, not a dependency, for `scripts/shots.py`'s reason: it is fetched +into the run and never enters the venv, so R16 still holds at three. + +**macOS will ask for Screen Recording** the first time, because a screenshot of other +windows is what that permission governs. Grant it to the terminal and run again — the +capture comes back black otherwise, which the report says out loud rather than leaving +you to wonder why the picture is empty. + +Three bands, top to bottom: + + **the numbers** - platform, Tk build, what each work-area method answers, where the + stack is therefore placed, and whether the native engine is ready. This is the half + a screenshot cannot carry and a log cannot show. + + **the whole screen** - scaled down, because the question "is it clear of the Dock and + centred" is about where the window sits *in* the display, and a crop of the window + cannot answer it. A neutral backdrop covers the desktop first, `scripts/shots.py`'s + trick and for a second reason here: this image gets sent to somebody, and a full-screen + capture of a working machine carries whatever happened to be open on it. + + **the stack, close up** - at full resolution, because "is there a title bar on it" is + about a 22 px band that a scaled-down screen loses. +""" + +import subprocess +import sys +import tkinter as tk +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) +sys.path.insert(0, str(ROOT / "scripts")) + +from PIL import Image, ImageDraw, ImageFont # noqa: E402 + +import flow.ui as ui # noqa: E402 +from ui_probe import FakeSession # noqa: E402 + +OUT = Path.home() / "flow-mac-report.png" +PAD = 24 +BG = (14, 16, 22) +FG = (230, 232, 238) +DIM = (150, 156, 170) +GOOD = (120, 210, 160) +BAD = (240, 140, 140) + + +def mono(size: int): + """A real monospace if this machine has one, so the numbers line up in columns.""" + for path in ("/System/Library/Fonts/Menlo.ttc", + "/System/Library/Fonts/SFNSMono.ttf", + "C:/Windows/Fonts/consola.ttf"): + try: + return ImageFont.truetype(path, size) + except OSError: + continue + return ImageFont.load_default() + + +def facts(pill) -> list[tuple[str, str, tuple]]: + """(label, value, colour). Everything a round trip has had to ask for separately.""" + sw, sh = pill.winfo_screenwidth(), pill.winfo_screenheight() + rows: list[tuple[str, str, tuple]] = [] + + def row(label, value, colour=FG): + rows.append((label, str(value), colour)) + + row("platform", f"{sys.platform} tk {tk.TkVersion}") + row("python", sys.version.split()[0]) + row("screen (as Tk sees it)", f"{sw} x {sh}") + row("", "") + + row("_work_area()", ui._work_area(sw, sh)) + measured = ui._tk_work_area(pill, sw, sh) + whole = (0, 0, sw, sh) + row("_tk_work_area()", measured, + BAD if measured == whole else GOOD) + if measured == whole: + row("", "the maximise probe was refused or ignored;", DIM) + row("", "falling back to the whole screen, so the Dock is not excluded", DIM) + full, work = ui._pointer_monitor(sw, sh, pill) + row("monitor full / work", f"{full} / {work}") + row("", "") + + row("place setting", ui.PLACE) + row("pill placed at", (pill.x, pill.y)) + row("pill size", f"{ui.PILL_W} x {ui.PILL_H}") + bottom_gap = work[3] - (pill.y + ui.PILL_H) + row("gap below the pill", f"{bottom_gap} px to the work-area bottom", + GOOD if 0 <= bottom_gap <= 60 else BAD) + centred = abs((pill.x + pill.pill_w / 2) - (full[0] + full[2]) / 2) + row("off centre by", f"{centred:.0f} px", + GOOD if centred < 4 else BAD) + row("", "") + + row("engine", "macOS on-device speech" if _native_ready()[0] else "whisper") + ok, why = _native_ready() + row("native available", "yes" if ok else f"no - {why}", GOOD if ok else DIM) + row("chord", "none off Windows; hold the pill instead", DIM) + return rows + + +def _native_ready() -> tuple[bool, str]: + try: + from flow.native import available + + return available(compile_if_missing=False, timeout=10.0) + except Exception as exc: # pragma: no cover - a report must not die reporting + return False, f"{type(exc).__name__}: {exc}" + + +def panel(rows, width: int) -> Image.Image: + """The numbers, as an image, so they travel in the same file as the picture.""" + font = mono(20) + line = 30 + img = Image.new("RGB", (width, PAD * 2 + line * (len(rows) + 1)), BG) + d = ImageDraw.Draw(img) + d.text((PAD, PAD), "flow - macOS report", font=mono(24), fill=FG) + for i, (label, value, colour) in enumerate(rows, start=1): + y = PAD + line * i + d.text((PAD, y), label, font=font, fill=DIM) + d.text((PAD + 380, y), value, font=font, fill=colour) + return img + + +def grab(): + """The whole screen, and whether it came back black. + + A black capture is what macOS returns before Screen Recording is granted, and it + looks exactly like a bug in Flow. Said out loud instead. + """ + from PIL import ImageGrab + + img = ImageGrab.grab() + extremes = img.convert("L").getextrema() + return img, extremes[1] > 12 + + +def stack_box(pill, ratio: float) -> tuple[int, int, int, int]: + """The pill and whatever is docked to it, in capture pixels, and nothing else. + + Read off the windows rather than guessed at with a margin. The first version + reserved 320 px above the pill for "whatever might be docked" and 60 below it, which + on a real desktop cropped in the taskbar and cut the pill's own bottom edge off - and + the bottom edge is exactly where a title bar or a broken dock seam would show. + """ + top, bottom = pill.y, pill.y + ui.PILL_H + for panel in (pill.bubble, pill.card): + try: + if not panel.winfo_ismapped(): + continue + top = min(top, panel.winfo_rooty()) + bottom = max(bottom, panel.winfo_rooty() + panel.winfo_height()) + except tk.TclError: + continue + edge = 14 + box = (pill.x - edge, top - edge, pill.x + pill.pill_w + edge, bottom + edge) + return tuple(int(v * ratio) for v in box) + + +def backdrop(pill) -> None: + """Cover the desktop, so the report is a picture of Flow and not of the machine. + + `scripts/shots.py` does this to keep its captures clean. Here it is also a privacy + line: the whole point of this file is that the image gets sent to somebody, and a + full-screen grab of a working machine carries every window that happened to be open. + + Topmost and then lowered under the pill, which is the order that file found: a plain + `lift()` is refused and the desktop shows through, so it has to outrank everything + and then step back behind Flow's own windows. + """ + back = tk.Toplevel(pill) + back.overrideredirect(True) + back.configure(bg="#23262b") + back.geometry(f"{pill.winfo_screenwidth()}x{pill.winfo_screenheight()}+0+0") + back.attributes("-topmost", True) + back.update_idletasks() + back.lower(pill) + + +def main() -> None: + session = FakeSession() + pill = ui.Pill(session) + pill.armed = True + backdrop(pill) + + def report() -> None: + rows = facts(pill) + screen, lit = grab() + ratio = screen.width / max(1, pill.winfo_screenwidth()) + + head = panel(rows, 1400) + shot = screen.copy() + shot.thumbnail((1400, 900)) + close = screen.crop(stack_box(pill, ratio)) + if close.width > 1400: + close.thumbnail((1400, 10_000)) + + parts = [head, shot, close] + out = Image.new("RGB", (1400, sum(p.height + PAD for p in parts) + PAD), BG) + y = 0 + for part in parts: + out.paste(part, ((1400 - part.width) // 2, y)) + y += part.height + PAD + out.save(OUT) + + for label, value, _c in rows: + if label or value: + print(f" {label:<24} {value}") + print(f"\nwrote {OUT} ({out.width}x{out.height})") + if not lit: + print("\nThe screen capture came back black. macOS needs Screen Recording\n" + "for the terminal: System Settings > Privacy & Security > Screen\n" + "Recording. Grant it, then run this again.") + try: + subprocess.run(["open", "-R", str(OUT)], check=False) + except OSError: + pass + pill.quit_app() + + # Late enough that the bubble has been drawn and placed, which is most of what the + # picture is of. + pill.after(1200, report) + pill.mainloop() + + +if __name__ == "__main__": + main() From 91977dd8bf3b0a7dc62076caa1cd2af924fbef1f Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 00:47:14 -0500 Subject: [PATCH 13/41] Add --text, because the fast path turned out to be an agent pasting output The quickest way to answer a question about this platform has turned out to be somebody's agent running one command and pasting what it printed. Asking that for Pillow and a Screen Recording grant to print twelve numbers is the report getting in the way of being read, so --text skips the capture, the backdrop and the image entirely. Pillow moved to a lazy import for it. The numbers were always most of what the picture was carrying. Co-Authored-By: Claude Opus 5 --- scripts/mac_report.py | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/scripts/mac_report.py b/scripts/mac_report.py index 6efcd73..a9150a7 100644 --- a/scripts/mac_report.py +++ b/scripts/mac_report.py @@ -6,7 +6,13 @@ carrying both. One file to send back, and the geometry in it is pixel-exact rather than a phone photo of a screen at an angle. - uv run --with pillow python scripts/mac_report.py + uv run --with pillow python scripts/mac_report.py # the image + uv run python scripts/mac_report.py --text # the numbers alone + +`--text` exists because the fastest way to answer a question about this platform has +turned out to be an agent running one command and pasting what it printed. It needs no +Pillow, no Screen Recording grant and no screenshot - just the numbers, which are most +of what the image was carrying anyway. Pillow is a `--with`, not a dependency, for `scripts/shots.py`'s reason: it is fetched into the run and never enters the venv, so R16 still holds at three. @@ -41,8 +47,6 @@ sys.path.insert(0, str(ROOT)) sys.path.insert(0, str(ROOT / "scripts")) -from PIL import Image, ImageDraw, ImageFont # noqa: E402 - import flow.ui as ui # noqa: E402 from ui_probe import FakeSession # noqa: E402 @@ -55,8 +59,21 @@ BAD = (240, 140, 140) +def _pil(): + """Imported here, not at the top, so `--text` needs neither Pillow nor a permission. + + The text form is the one an agent on the machine can run and paste back, and asking + it for a screenshot dependency and a Screen Recording grant to print twelve numbers + would be the report getting in the way of being read. + """ + from PIL import Image, ImageDraw, ImageFont + + return Image, ImageDraw, ImageFont + + def mono(size: int): """A real monospace if this machine has one, so the numbers line up in columns.""" + _Image, _Draw, ImageFont = _pil() for path in ("/System/Library/Fonts/Menlo.ttc", "/System/Library/Fonts/SFNSMono.ttf", "C:/Windows/Fonts/consola.ttf"): @@ -119,8 +136,9 @@ def _native_ready() -> tuple[bool, str]: return False, f"{type(exc).__name__}: {exc}" -def panel(rows, width: int) -> Image.Image: +def panel(rows, width: int): """The numbers, as an image, so they travel in the same file as the picture.""" + Image, ImageDraw, _Font = _pil() font = mono(20) line = 30 img = Image.new("RGB", (width, PAD * 2 + line * (len(rows) + 1)), BG) @@ -189,16 +207,25 @@ def backdrop(pill) -> None: def main() -> None: + text_only = "--text" in sys.argv[1:] session = FakeSession() pill = ui.Pill(session) pill.armed = True - backdrop(pill) + if not text_only: + backdrop(pill) def report() -> None: rows = facts(pill) + if text_only: + for label, value, _c in rows: + if label or value: + print(f" {label:<24} {value}") + pill.quit_app() + return screen, lit = grab() ratio = screen.width / max(1, pill.winfo_screenwidth()) + Image, _Draw, _Font = _pil() head = panel(rows, 1400) shot = screen.copy() shot.thumbnail((1400, 900)) From a06526ce3ef88a9ff95a17fa460117c56852f86b Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:24:09 -0500 Subject: [PATCH 14/41] Ask Aqua three ways where the Dock is, since zoomed answers with the whole screen The maximise probe that finds the taskbar exactly on Windows is accepted and ignored on Aqua, so _tk_work_area fell back to the whole screen and the pill was placed 24 px above 878 - inside a 69 px Dock. Rather than guess a fourth time at this platform, this prints wm maxsize (which Tk's Aqua port answers from NSScreen visibleFrame), where a window asked for +0+0 actually lands, and fullscreen as a control. Co-Authored-By: Claude Opus 5 --- scripts/mac_area_probe.py | 64 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 scripts/mac_area_probe.py diff --git a/scripts/mac_area_probe.py b/scripts/mac_area_probe.py new file mode 100644 index 0000000..2ab81c3 --- /dev/null +++ b/scripts/mac_area_probe.py @@ -0,0 +1,64 @@ +"""Which call on Aqua knows where the Dock is. + +`_tk_work_area` measures the usable area by maximising a probe window and reading back +where the window manager put it. On Windows that agrees with `SystemParametersInfoW` +exactly. On macOS it came back as the whole screen — `state("zoomed")` neither raised +nor maximised — so the pill was placed 24 px above 878 and landed inside the Dock band. + +Three Aqua-specific candidates are asked here instead of guessed at: + + **wm maxsize** - Tk's Aqua port answers this from `[NSScreen visibleFrame]`, so it + should already be the screen minus the menu bar and the Dock. It gives a *size* and + no origin, which is why it is not enough on its own. (It is useless on Windows, where + it answers with the whole screen even with a taskbar present - which is why the + maximise probe exists at all.) + + **a window asked for +0+0** - Aqua will not put a titled window under the menu bar, so + where it actually lands is the top of the usable area. + + **wm attributes -fullscreen** - the whole screen including the menu bar, as a control: + if this and `zoomed` agree, `zoomed` is being treated as fullscreen. + +Between the first two, `bottom = top + maxsize_height` is the Dock's top edge. + + uv run python scripts/mac_area_probe.py +""" + +import sys +import tkinter as tk + +root = tk.Tk() +root.withdraw() +sw, sh = root.winfo_screenwidth(), root.winfo_screenheight() +print(f" platform {sys.platform} tk {tk.TkVersion} " + f"patch {root.tk.call('info', 'patchlevel')}") +print(f" screen {sw} x {sh}") + +mw, mh = root.maxsize() +print(f" wm maxsize {mw} x {mh} (lost: {sw - mw} w, {sh - mh} h)") + + +def probe(name, setup): + win = tk.Toplevel(root) + win.attributes("-alpha", 0.0) + win.geometry("200x120+80+80") + note = "ok" + try: + setup(win) + win.update_idletasks() + except tk.TclError as exc: + note = f"FAILED ({exc})" + x, y = win.winfo_rootx(), win.winfo_rooty() + w, h = win.winfo_width(), win.winfo_height() + print(f" {name:<24} ({x}, {y}, {x + w}, {y + h}) {note}") + win.destroy() + return x, y, w, h + + +probe("zoomed (today's probe)", lambda w: w.state("zoomed")) +_x, top, _w, _h = probe("asked for +0+0", lambda w: w.geometry("200x120+0+0")) +probe("fullscreen (control)", lambda w: w.attributes("-fullscreen", True)) + +print(f"\n so the usable area is (0, {top}, {mw}, {top + mh})") +print(f" and the Dock starts at y = {top + mh} ({sh - (top + mh)} px tall)") +root.destroy() From ec7001d616022ac7fa766ed2e4abc352bd829b28 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:42:06 -0500 Subject: [PATCH 15/41] Ask Aqua for the visible frame, since it accepts the maximise and ignores it A Mac reported _tk_work_area() answering with the whole 1352x878 screen, identical to _work_area(). state("zoomed") had neither raised nor maximised, so the fallback did not fall back: bottom_centre stood the pill 24 px above 878 and it lived inside a 69 px Dock. The close-up in the report was a picture of Dock icons. wm maxsize is the call that knows, and only here - Tk's Aqua port answers it from NSScreen visibleFrame. On Windows it reports the whole screen with a taskbar present, which is why the maximise probe exists at all. It gives no origin, so the top comes from where a window asked for +0+0 actually lands, and the bottom - the edge the pill stands on - is top + height. Every measurement is shape-checked and the path returns None unless all of them hold, so if this platform surprises us again the worst case is exactly the old behaviour. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 75 ++++++++++++++++++++++++++++++++++ tests/test_pill.py | 100 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 175 insertions(+) diff --git a/flow/ui.py b/flow/ui.py index 12f6735..0d3b805 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -243,6 +243,71 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: _TK_WORK: tuple | None = None +#: How much of the screen an Aqua menu bar can plausibly take. 25 px on an ordinary +#: display, 38 on a notched one; anything outside this is not a menu bar and the probe +#: that reported it is measuring something else. +_AQUA_MENU_MAX = 80 + + +def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: + """The visible frame on macOS, or None if this build cannot say. + + **The maximise probe does not work here.** `state("zoomed")` on Aqua neither raises + nor maximises — a Mac reported it answering with the whole 1352x878 screen, so + `_tk_work_area` fell back to the screen, `bottom_centre` stood the pill 24 px above + 878, and the pill spent its life inside a 69 px Dock. The close-up in the report was + a picture of Dock icons. + + `wm maxsize` is the call that knows, and only on this platform: Tk's Aqua port + answers it from `[NSScreen visibleFrame]`, which is the screen less the menu bar and + the Dock. On Windows the same call answers with the whole screen even with a taskbar + present, which is why `_tk_work_area` measures instead of asking — this is the one + platform where the shortcut is the *better* instrument, not a lazier one. + + `maxsize` gives a size and no origin, and the height it is missing is split between + a menu bar at the top and a Dock at the bottom, which it does not break down. So the + origin is measured separately: Aqua will not place a titled window under the menu + bar, so where a window asked for `+0+0` actually lands is the top of the usable area. + The bottom — the edge that matters, the one the pill stands on — is then + `top + height`. + + **Nothing here is trusted without a shape check.** If `maxsize` answers with the + whole screen it has told us nothing, and if the `+0+0` probe comes back somewhere a + menu bar could not be, it was honoured literally rather than clamped. Either way + this returns None and the caller carries on to the maximise probe, so the worst case + is exactly today's behaviour rather than a new way to be wrong. + """ + try: + mw, mh = win.maxsize() + except (tk.TclError, AttributeError, TypeError, ValueError): + return None + if not (0 < mw <= sw and 0 < mh < sh): + return None # it answered with the whole screen: it does not know either + + top = 0 + probe = None + try: + probe = tk.Toplevel(win) + probe.attributes("-alpha", 0.0) + probe.geometry("200x120+0+0") + probe.update_idletasks() + landed = probe.winfo_rooty() + if 0 <= landed <= _AQUA_MENU_MAX: + top = landed + except (tk.TclError, AttributeError, ValueError): + pass + finally: + if probe is not None: + try: + probe.destroy() + except tk.TclError: + pass + if top + mh > sh: + # The two measurements disagree about the display. Better to know nothing. + return None + return 0, top, mw, top + mh + + def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: """The usable area, measured by asking the window manager to maximise something. @@ -259,6 +324,11 @@ def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: Checked against `SystemParametersInfoW` on Windows, where the two agree exactly on left, right and bottom. + **macOS is the exception and is handled before any of this**, in `_aqua_work_area`: + there the maximise is accepted and ignored, and `wm maxsize` — useless on Windows — + is the call that knows where the Dock is. That path returns None unless what it + measured has the shape of a real work area, so this measurement stays the fallback. + The probe is transparent while it is measured, so nothing flashes on screen. **`top` is taken as reported and is a title bar too low.** `winfo_rooty` is the @@ -271,6 +341,11 @@ def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: global _TK_WORK if _TK_WORK is not None: return _TK_WORK + if sys.platform == "darwin": + aqua = _aqua_work_area(win, sw, sh) + if aqua is not None: + _TK_WORK = aqua + return aqua fallback = (0, 0, sw, sh) asked_w, asked_h = 200, 120 probe = None diff --git a/tests/test_pill.py b/tests/test_pill.py index adba3f4..e307ad2 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -739,6 +739,106 @@ def test_a_panel_with_no_height_yet_is_still_parked_somewhere_legal(self): self.assertTrue(win.geometry.call_args[0][0].startswith("420x1+")) +class TestTheWorkAreaOnAqua(unittest.TestCase): + """`_aqua_work_area`, which exists because the maximise probe is ignored on a Mac. + + A Mac reported `_tk_work_area()` answering with the whole 1352x878 screen, identical + to `_work_area()`. `state("zoomed")` had neither raised nor maximised, so the + fallback did not fall back, `bottom_centre` stood the pill 24 px above 878, and the + pill sat inside a 69 px Dock. + + `wm maxsize` is the instrument here and *only* here: Tk's Aqua port answers it from + `[NSScreen visibleFrame]`. On Windows the same call reports the whole screen with a + taskbar present, which is the reason the maximise probe exists at all. + """ + + def setUp(self): + self._was = ui._TK_WORK + ui._TK_WORK = None + self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) + + def win(self, maxsize=(1352, 777), landed=25): + """A Mac-shaped stand-in: a 1352x878 screen, a menu bar and a 76 px Dock.""" + win = mock.Mock() + win.maxsize.return_value = maxsize + probe = mock.Mock() + probe.winfo_rooty.return_value = landed + return win, probe + + def call(self, win, probe, sw=1352, sh=878): + with mock.patch.object(ui.tk, "Toplevel", return_value=probe): + return ui._aqua_work_area(win, sw, sh) + + def test_the_bottom_is_the_top_of_the_dock(self): + # The whole point: 802, not 878. The pill stands on this number. + win, probe = self.win() + self.assertEqual(self.call(win, probe), (0, 25, 1352, 802)) + + def test_the_probe_is_invisible_and_cleaned_up(self): + win, probe = self.win() + self.call(win, probe) + probe.attributes.assert_any_call("-alpha", 0.0) + probe.destroy.assert_called_once() + + def test_a_maxsize_of_the_whole_screen_means_it_does_not_know(self): + # Windows answers this way with a taskbar present. Reporting the whole screen as + # a work area is the bug, not a fix for it. + win, probe = self.win(maxsize=(1352, 878)) + self.assertIsNone(self.call(win, probe)) + + def test_a_window_manager_that_honoured_plus_zero_is_not_a_menu_bar(self): + # Windows puts a window where it is asked and reports a frame inset - 31 px on + # this machine, which is close enough to a menu bar to be believed by accident. + # A number far outside a menu bar's range is a literal placement, so `top` falls + # back to 0 rather than shifting the whole work area by it. + win, probe = self.win(landed=400) + self.assertEqual(self.call(win, probe), (0, 0, 1352, 777)) + + def test_a_probe_that_cannot_be_built_still_leaves_the_height(self): + # The height is the part that keeps the pill off the Dock. Losing the origin + # costs a menu bar of clearance; losing the height costs the whole fix. + win, _probe = self.win() + with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): + self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 777)) + + def test_measurements_that_disagree_are_refused(self): + # A menu bar plus a visible frame cannot be taller than the display it is on. + win, probe = self.win(maxsize=(1352, 870), landed=40) + self.assertIsNone(self.call(win, probe)) + + def test_a_build_without_maxsize_says_so(self): + win, probe = self.win() + win.maxsize.side_effect = ui.tk.TclError("no such command") + self.assertIsNone(self.call(win, probe)) + + def test_tk_work_area_prefers_it_on_darwin(self): + win, probe = self.win() + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 25, 1352, 802)) + + def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): + # The guarantee that made this safe to write before a Mac had confirmed it: if + # the Aqua path cannot answer, the worst case is exactly the old behaviour. + win, probe = self.win(maxsize=(1352, 878)) + probe.winfo_rootx.return_value = 0 + probe.winfo_rooty.return_value = 25 + probe.winfo_width.return_value = 1352 + probe.winfo_height.return_value = 777 + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 25, 1352, 802)) + + def test_windows_never_takes_this_path(self): + # `maxsize` there is the whole screen, taskbar or not. Asking it would undo the + # measurement this module went to the trouble of making. + win, probe = self.win() + with mock.patch.object(sys, "platform", "win32"), mock.patch.object(ui, "_aqua_work_area") as aqua, mock.patch.object(ui.tk, "Toplevel", return_value=probe): + probe.winfo_rootx.return_value = 0 + probe.winfo_width.return_value = 1280 + probe.winfo_height.return_value = 649 + ui._tk_work_area(win, 1280, 720) + aqua.assert_not_called() + + class TestTheWorkAreaOffWindows(unittest.TestCase): """`_tk_work_area`, which exists because a Mac put the pill under the Dock. From 812a25b1651cc5a52aa79007a3b8b565df5185ce Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:49:24 -0500 Subject: [PATCH 16/41] Put the Mac's real numbers behind the Aqua work area, and a Dock height to check them against scripts/mac_area_probe.py on a 14-inch MacBook Pro, Tk 9.0.3, 1352x878: wm maxsize 1352x736, a window asked for +0+0 landing at y 58, and zoomed answering (80, 108, 280, 228) - the 200x120 probe still at the +80+80 it was asked for, never maximised, no error. So the Dock's top edge is 58 + 736 = 794. Both of those are Tk asking Tk, and three earlier guesses at this platform failed for want of a second source. defaults read com.apple.dock tilesize on the same machine says 69, and 69 plus Apple's padding is the 84 px this leaves below 794. That agreement is now a test. The invented fixtures are replaced by the measured ones throughout. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 25 ++++++++++++++++++------- tests/test_pill.py | 36 +++++++++++++++++++++++++++--------- 2 files changed, 45 insertions(+), 16 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 0d3b805..ad2a9a2 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -243,9 +243,11 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: _TK_WORK: tuple | None = None -#: How much of the screen an Aqua menu bar can plausibly take. 25 px on an ordinary -#: display, 38 on a notched one; anything outside this is not a menu bar and the probe -#: that reported it is measuring something else. +#: How far down an Aqua window asked for `+0+0` can plausibly land. A 14-inch MacBook +#: Pro measured 58: a menu bar at 30, plus the 28 px of title bar between a window's +#: frame and the client area `winfo_rooty` reports. Anything past this was not clamped +#: by a menu bar at all — the window manager honoured the request literally, as Windows +#: does — and the number means nothing here. _AQUA_MENU_MAX = 80 @@ -253,10 +255,11 @@ def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: """The visible frame on macOS, or None if this build cannot say. **The maximise probe does not work here.** `state("zoomed")` on Aqua neither raises - nor maximises — a Mac reported it answering with the whole 1352x878 screen, so - `_tk_work_area` fell back to the screen, `bottom_centre` stood the pill 24 px above - 878, and the pill spent its life inside a 69 px Dock. The close-up in the report was - a picture of Dock icons. + nor maximises. Asked to maximise a 200x120 window at +80+80, Tk 9.0.3 returned + (80, 108, 280, 228) — the same window, the same size, the position it was already + in, and no error. `_tk_work_area` rejected that and fell back to the whole screen, + `bottom_centre` stood the pill 24 px above 878, and the pill spent its life inside + the Dock. The close-up in the report was a picture of Dock icons. `wm maxsize` is the call that knows, and only on this platform: Tk's Aqua port answers it from `[NSScreen visibleFrame]`, which is the screen less the menu bar and @@ -271,6 +274,14 @@ def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: The bottom — the edge that matters, the one the pill stands on — is then `top + height`. + Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: `maxsize` 1352x736, + a `+0+0` window landing at y 58, so the Dock's top edge is 794 and the Dock is 84 px + tall. Both of those are Tk asking Tk, so they were checked against something that is + not: `defaults read com.apple.dock tilesize` on the same machine says 69, and 69 plus + Apple's padding is the 84 this leaves. `top` is a title bar low for the same reason + it is on Windows — `winfo_rooty` is the client area — and costs nothing, because it + feeds only the ceiling in `bottom_centre`. The bottom edge is exact. + **Nothing here is trusted without a shape check.** If `maxsize` answers with the whole screen it has told us nothing, and if the `+0+0` probe comes back somewhere a menu bar could not be, it was honoured literally rather than clamped. Either way diff --git a/tests/test_pill.py b/tests/test_pill.py index e307ad2..6272a31 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -757,8 +757,14 @@ def setUp(self): ui._TK_WORK = None self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) - def win(self, maxsize=(1352, 777), landed=25): - """A Mac-shaped stand-in: a 1352x878 screen, a menu bar and a 76 px Dock.""" + def win(self, maxsize=(1352, 736), landed=58): + """The numbers a 14-inch MacBook Pro actually reported, not invented ones. + + `scripts/mac_area_probe.py` on darwin, Tk 9.0.3, a 1352x878 screen: + `wm maxsize` 1352x736, a window asked for `+0+0` landing with its client area at + y 58, and `zoomed` answering (80, 108, 280, 228) - the 200x120 probe still at + the +80+80 it was asked for, never maximised at all. + """ win = mock.Mock() win.maxsize.return_value = maxsize probe = mock.Mock() @@ -772,7 +778,19 @@ def call(self, win, probe, sw=1352, sh=878): def test_the_bottom_is_the_top_of_the_dock(self): # The whole point: 802, not 878. The pill stands on this number. win, probe = self.win() - self.assertEqual(self.call(win, probe), (0, 25, 1352, 802)) + self.assertEqual(self.call(win, probe), (0, 58, 1352, 794)) + + def test_the_dock_it_finds_matches_the_dock_the_os_reports(self): + """84 px of Dock, against a `com.apple.dock tilesize` of 69 read separately. + + The reason this platform is believed at all now. `maxsize` and the `+0+0` probe + are both Tk asking Tk; a tile size out of `defaults` came from somewhere else + entirely, and 69 plus Apple's padding is the 84 this leaves. Three earlier + guesses at Aqua all failed for want of a second source. + """ + win, probe = self.win() + _l, _t, _r, bottom = self.call(win, probe) + self.assertEqual(878 - bottom, 84) def test_the_probe_is_invisible_and_cleaned_up(self): win, probe = self.win() @@ -792,14 +810,14 @@ def test_a_window_manager_that_honoured_plus_zero_is_not_a_menu_bar(self): # A number far outside a menu bar's range is a literal placement, so `top` falls # back to 0 rather than shifting the whole work area by it. win, probe = self.win(landed=400) - self.assertEqual(self.call(win, probe), (0, 0, 1352, 777)) + self.assertEqual(self.call(win, probe), (0, 0, 1352, 736)) def test_a_probe_that_cannot_be_built_still_leaves_the_height(self): # The height is the part that keeps the pill off the Dock. Losing the origin # costs a menu bar of clearance; losing the height costs the whole fix. win, _probe = self.win() with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): - self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 777)) + self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 736)) def test_measurements_that_disagree_are_refused(self): # A menu bar plus a visible frame cannot be taller than the display it is on. @@ -814,18 +832,18 @@ def test_a_build_without_maxsize_says_so(self): def test_tk_work_area_prefers_it_on_darwin(self): win, probe = self.win() with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 25, 1352, 802)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 794)) def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): # The guarantee that made this safe to write before a Mac had confirmed it: if # the Aqua path cannot answer, the worst case is exactly the old behaviour. win, probe = self.win(maxsize=(1352, 878)) probe.winfo_rootx.return_value = 0 - probe.winfo_rooty.return_value = 25 + probe.winfo_rooty.return_value = 58 probe.winfo_width.return_value = 1352 - probe.winfo_height.return_value = 777 + probe.winfo_height.return_value = 736 with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 25, 1352, 802)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 794)) def test_windows_never_takes_this_path(self): # `maxsize` there is the whole screen, taskbar or not. Asking it would undo the From 9f4a09bb72b7f8a22fd038ed9051530775421203 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:52:13 -0500 Subject: [PATCH 17/41] Correct the measured Mac numbers: maxsize is 735, so the Dock is 85 px Transcribed from a photo of the terminal as 736, which the same output disproves on the line above - lost: 143 h against an 878 screen is 735, and 58 + 735 is the 793 the probe printed. The Dock is 85 px, not 84, which is if anything the better agreement with the 69 px tilesize the OS reports. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 6 +++--- tests/test_pill.py | 22 +++++++++++----------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index ad2a9a2..397e415 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -274,11 +274,11 @@ def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: The bottom — the edge that matters, the one the pill stands on — is then `top + height`. - Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: `maxsize` 1352x736, - a `+0+0` window landing at y 58, so the Dock's top edge is 794 and the Dock is 84 px + Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: `maxsize` 1352x735, + a `+0+0` window landing at y 58, so the Dock's top edge is 793 and the Dock is 85 px tall. Both of those are Tk asking Tk, so they were checked against something that is not: `defaults read com.apple.dock tilesize` on the same machine says 69, and 69 plus - Apple's padding is the 84 this leaves. `top` is a title bar low for the same reason + Apple's padding is the 85 this leaves. `top` is a title bar low for the same reason it is on Windows — `winfo_rooty` is the client area — and costs nothing, because it feeds only the ceiling in `bottom_centre`. The bottom edge is exact. diff --git a/tests/test_pill.py b/tests/test_pill.py index 6272a31..e42513c 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -757,11 +757,11 @@ def setUp(self): ui._TK_WORK = None self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) - def win(self, maxsize=(1352, 736), landed=58): + def win(self, maxsize=(1352, 735), landed=58): """The numbers a 14-inch MacBook Pro actually reported, not invented ones. `scripts/mac_area_probe.py` on darwin, Tk 9.0.3, a 1352x878 screen: - `wm maxsize` 1352x736, a window asked for `+0+0` landing with its client area at + `wm maxsize` 1352x735, a window asked for `+0+0` landing with its client area at y 58, and `zoomed` answering (80, 108, 280, 228) - the 200x120 probe still at the +80+80 it was asked for, never maximised at all. """ @@ -778,19 +778,19 @@ def call(self, win, probe, sw=1352, sh=878): def test_the_bottom_is_the_top_of_the_dock(self): # The whole point: 802, not 878. The pill stands on this number. win, probe = self.win() - self.assertEqual(self.call(win, probe), (0, 58, 1352, 794)) + self.assertEqual(self.call(win, probe), (0, 58, 1352, 793)) def test_the_dock_it_finds_matches_the_dock_the_os_reports(self): - """84 px of Dock, against a `com.apple.dock tilesize` of 69 read separately. + """85 px of Dock, against a `com.apple.dock tilesize` of 69 read separately. The reason this platform is believed at all now. `maxsize` and the `+0+0` probe are both Tk asking Tk; a tile size out of `defaults` came from somewhere else - entirely, and 69 plus Apple's padding is the 84 this leaves. Three earlier + entirely, and 69 plus Apple's padding is the 85 this leaves. Three earlier guesses at Aqua all failed for want of a second source. """ win, probe = self.win() _l, _t, _r, bottom = self.call(win, probe) - self.assertEqual(878 - bottom, 84) + self.assertEqual(878 - bottom, 85) def test_the_probe_is_invisible_and_cleaned_up(self): win, probe = self.win() @@ -810,14 +810,14 @@ def test_a_window_manager_that_honoured_plus_zero_is_not_a_menu_bar(self): # A number far outside a menu bar's range is a literal placement, so `top` falls # back to 0 rather than shifting the whole work area by it. win, probe = self.win(landed=400) - self.assertEqual(self.call(win, probe), (0, 0, 1352, 736)) + self.assertEqual(self.call(win, probe), (0, 0, 1352, 735)) def test_a_probe_that_cannot_be_built_still_leaves_the_height(self): # The height is the part that keeps the pill off the Dock. Losing the origin # costs a menu bar of clearance; losing the height costs the whole fix. win, _probe = self.win() with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): - self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 736)) + self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 735)) def test_measurements_that_disagree_are_refused(self): # A menu bar plus a visible frame cannot be taller than the display it is on. @@ -832,7 +832,7 @@ def test_a_build_without_maxsize_says_so(self): def test_tk_work_area_prefers_it_on_darwin(self): win, probe = self.win() with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 794)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 793)) def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): # The guarantee that made this safe to write before a Mac had confirmed it: if @@ -841,9 +841,9 @@ def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): probe.winfo_rootx.return_value = 0 probe.winfo_rooty.return_value = 58 probe.winfo_width.return_value = 1352 - probe.winfo_height.return_value = 736 + probe.winfo_height.return_value = 735 with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 794)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 793)) def test_windows_never_takes_this_path(self): # `maxsize` there is the whole screen, taskbar or not. Asking it would undo the From 4b016669299aad2c4228deb6a387eb5eb89bc8cf Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 02:58:02 -0500 Subject: [PATCH 18/41] Guard the Windows-only window attributes on the platform, not just on lite mode mac_report.py died on a Mac with bad attribute "-transparentcolor", raised by Tk before a window had been drawn. -transparentcolor and -toolwindow exist on one platform and are fatal on the others, and _shell_window guarded them on lite alone - safe only because __main__ forces lite off Windows, so the two could never come apart. They came apart the first time something other than __main__ built a Pill. An invariant a caller has to know about is one a caller can miss, and this one is cheap to enforce where it is actually true. mac_report now builds the lite window the app builds, which is the one worth measuring anyway. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 12 ++++++++++-- scripts/mac_report.py | 5 ++++- tests/test_pill.py | 41 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 3 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 397e415..ca854f8 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -194,7 +194,15 @@ def _shell_window(win, lite: bool, alpha: float) -> str: Two of the five are Windows-only Tk attributes. `-transparentcolor` is what keys the magenta out, so without it the keyed colour is not invisible — it is a magenta rectangle where the app should be — and `-toolwindow` does not exist off Windows at - all. Asking for either is a `TclError` before anything is drawn. + all. Asking for either is a `TclError` before anything is drawn, which is why the + platform is part of the guard and not only `lite`. + + **It used to be only `lite`**, on the reasoning that `__main__` forces lite mode off + Windows so the two can never come apart. They came apart the first time something + other than `__main__` built a `Pill`: `scripts/mac_report.py` asked for full mode on + a Mac and got `bad attribute "-transparentcolor"` out of Tk before a window existed. + An invariant a caller has to know about is one a caller can miss, and this one is + cheap to enforce where it is true. **`overrideredirect` is the whole of it on Aqua too, and two attempts to help it were both harm.** A Mac reported the pill wearing a title bar, and the cause was not this @@ -211,7 +219,7 @@ def _shell_window(win, lite: bool, alpha: float) -> str: win.overrideredirect(True) win.attributes("-topmost", True) win.attributes("-alpha", alpha) - if lite: + if lite or sys.platform != "win32": return SHELL win.attributes("-transparentcolor", TRANSPARENT) win.attributes("-toolwindow", True) diff --git a/scripts/mac_report.py b/scripts/mac_report.py index a9150a7..d4886f2 100644 --- a/scripts/mac_report.py +++ b/scripts/mac_report.py @@ -209,7 +209,10 @@ def backdrop(pill) -> None: def main() -> None: text_only = "--text" in sys.argv[1:] session = FakeSession() - pill = ui.Pill(session) + # Lite, which is what `__main__` runs off Windows - `lite = args.lite or + # sys.platform != "win32"`. A report that built a different window from the one the + # app builds would be measuring something nobody uses. + pill = ui.Pill(session, lite=True) pill.armed = True if not text_only: backdrop(pill) diff --git a/tests/test_pill.py b/tests/test_pill.py index e42513c..012a065 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -739,6 +739,47 @@ def test_a_panel_with_no_height_yet_is_still_parked_somewhere_legal(self): self.assertTrue(win.geometry.call_args[0][0].startswith("420x1+")) +class TestTheWindowsOnlyAttributes(unittest.TestCase): + """`-transparentcolor` and `-toolwindow` exist on one platform and are fatal on the + others: `bad attribute "-transparentcolor"` out of Tk, raised before a window has + been drawn. + + The guard used to be `lite` alone, because `__main__` forces lite mode off Windows + (`lite = args.lite or sys.platform != "win32"`) so the two could not come apart. They + came apart as soon as something other than `__main__` built a `Pill`: + `scripts/mac_report.py` asked for full mode on a Mac and the report died in the + constructor. + """ + + def test_full_mode_off_windows_asks_for_neither(self): + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"): + self.assertEqual(ui._shell_window(win, lite=False, alpha=0.94), ui.SHELL) + asked = [c.args[0] for c in win.attributes.call_args_list] + self.assertNotIn("-transparentcolor", asked) + self.assertNotIn("-toolwindow", asked) + + def test_full_mode_on_windows_still_asks_for_both(self): + # The keyed colour is how the pill has no rectangle around it. Losing this on + # Windows would be a visible regression, not a quiet one. + win = mock.Mock() + with mock.patch.object(sys, "platform", "win32"): + self.assertEqual(ui._shell_window(win, lite=False, alpha=0.94), + ui.TRANSPARENT) + win.attributes.assert_any_call("-transparentcolor", ui.TRANSPARENT) + win.attributes.assert_any_call("-toolwindow", True) + + def test_the_shared_three_are_asked_for_everywhere(self): + for platform in ("darwin", "win32", "linux"): + with self.subTest(platform=platform): + win = mock.Mock() + with mock.patch.object(sys, "platform", platform): + ui._shell_window(win, lite=True, alpha=0.5) + win.overrideredirect.assert_called_once_with(True) + win.attributes.assert_any_call("-topmost", True) + win.attributes.assert_any_call("-alpha", 0.5) + + class TestTheWorkAreaOnAqua(unittest.TestCase): """`_aqua_work_area`, which exists because the maximise probe is ignored on a Mac. From 18517bb719685d2654c9a70bd1e000c3aa23ed4a Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 03:07:49 -0500 Subject: [PATCH 19/41] Measure the title bar too, since maxsize is a content size and the pill is bare The Aqua branch fired and answered 821 where the probe says 793 - 28 px too low, one title bar, and the pill went off the Dock and straight back onto it. wm maxsize is a maximum *content* size, so it is short by whatever decoration the window that answered it wears: 735 from a titled probe against 763 from the overrideredirect pill, on the same display. The first version took the origin from a probe and the size from the caller's window, which are not the same window, and counted the title bar twice. Now one probe answers all three - its own title bar from a y clear of any menu bar, the menu bar from +0+0 less that title bar, and the frame height from maxsize plus it - so the decoration appears on both sides and cancels. A bare window and a titled one must now reach the same answer, which is a test. Windows corroborates the arithmetic from the other direction: a 31 px title bar and a 0 px menu bar, there being none. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 105 +++++++++++++++++++++--------------- scripts/mac_area_probe.py | 22 ++++++-- tests/test_pill.py | 109 ++++++++++++++++++++++++-------------- 3 files changed, 149 insertions(+), 87 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index ca854f8..e30e0fe 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -251,13 +251,19 @@ def _work_area(sw: int, sh: int) -> tuple[int, int, int, int]: _TK_WORK: tuple | None = None -#: How far down an Aqua window asked for `+0+0` can plausibly land. A 14-inch MacBook -#: Pro measured 58: a menu bar at 30, plus the 28 px of title bar between a window's -#: frame and the client area `winfo_rooty` reports. Anything past this was not clamped -#: by a menu bar at all — the window manager honoured the request literally, as Windows -#: does — and the number means nothing here. +#: How far down an Aqua window asked for `+0+0` can plausibly be pushed by the menu bar: +#: 30 px on an ordinary display, more on a notched one. Past this, the window manager +#: honoured the request literally — as Windows does — and the number means nothing here. _AQUA_MENU_MAX = 80 +#: How tall an Aqua title bar can plausibly be. 28 px measured; the cap is loose because +#: it only has to separate "a title bar" from "the window went somewhere else entirely". +_AQUA_TITLE_MAX = 60 + +#: Where the probe is put to measure its own title bar — far enough down that no menu bar +#: can be clamping it, so the whole difference from what was asked for is decoration. +_AQUA_FREE_Y = 300 + def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: """The visible frame on macOS, or None if this build cannot say. @@ -275,56 +281,71 @@ def _aqua_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int] | None: present, which is why `_tk_work_area` measures instead of asking — this is the one platform where the shortcut is the *better* instrument, not a lazier one. - `maxsize` gives a size and no origin, and the height it is missing is split between - a menu bar at the top and a Dock at the bottom, which it does not break down. So the - origin is measured separately: Aqua will not place a titled window under the menu - bar, so where a window asked for `+0+0` actually lands is the top of the usable area. - The bottom — the edge that matters, the one the pill stands on — is then - `top + height`. - - Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: `maxsize` 1352x735, - a `+0+0` window landing at y 58, so the Dock's top edge is 793 and the Dock is 85 px - tall. Both of those are Tk asking Tk, so they were checked against something that is - not: `defaults read com.apple.dock tilesize` on the same machine says 69, and 69 plus - Apple's padding is the 85 this leaves. `top` is a title bar low for the same reason - it is on Windows — `winfo_rooty` is the client area — and costs nothing, because it - feeds only the ceiling in `bottom_centre`. The bottom edge is exact. - - **Nothing here is trusted without a shape check.** If `maxsize` answers with the - whole screen it has told us nothing, and if the `+0+0` probe comes back somewhere a - menu bar could not be, it was honoured literally rather than clamped. Either way - this returns None and the caller carries on to the maximise probe, so the worst case - is exactly today's behaviour rather than a new way to be wrong. + **Everything is measured from one probe, and that is the point.** `maxsize` gives a + size and no origin, so the origin has to come from somewhere else — and the first + version took it from a probe while taking the size from the caller's window. Those + are not the same window. `maxsize` is a maximum *content* size, so it is short by + whatever decoration its window wears: 735 from a titled probe against 763 from the + `overrideredirect` pill, on the same display, differing by exactly the 28 px title + bar. Adding one window's origin to another's size counted that title bar twice and + put the work area 28 px too low — the pill moved off the Dock and back onto it. + + So one probe answers all three, and its own decoration cancels out: + + **its title bar** — asked for a y far below any menu bar, so the whole difference + between what was asked and where the client area landed is decoration. + + **the menu bar** — asked for `+0+0`, where Aqua refuses to put a titled window; + where it lands, less the title bar just measured, is the top of the visible frame. + + **the visible frame's height** — `maxsize` plus that same title bar, which is what + turns a content size back into a frame size. + + Measured on a 14-inch MacBook Pro, Tk 9.0.3, a 1352x878 screen: a 28 px title bar, a + `+0+0` client top of 58 giving a menu bar of 30, and `maxsize` 1352x735 giving a + frame height of 763. The Dock's top edge is 30 + 763 = 793 and the Dock is 85 px + tall. All three are Tk asking Tk, so they were checked against something that is not: + `defaults read com.apple.dock tilesize` on the same machine says 69, and 69 plus + Apple's padding is the 85 this leaves. + + **Nothing is trusted without a shape check.** A `maxsize` of the whole screen has + told us nothing; a title bar or a menu bar outside the range one can be is a window + manager that honoured a request literally rather than clamping it, as Windows does. + Any of those and this returns None and the caller falls through to the maximise + probe, so the worst case is exactly the old behaviour rather than a new way to be + wrong. """ - try: - mw, mh = win.maxsize() - except (tk.TclError, AttributeError, TypeError, ValueError): - return None - if not (0 < mw <= sw and 0 < mh < sh): - return None # it answered with the whole screen: it does not know either - - top = 0 probe = None try: probe = tk.Toplevel(win) probe.attributes("-alpha", 0.0) + + probe.geometry(f"200x120+80+{_AQUA_FREE_Y}") + probe.update_idletasks() + title = probe.winfo_rooty() - _AQUA_FREE_Y + if not 0 <= title <= _AQUA_TITLE_MAX: + return None + probe.geometry("200x120+0+0") probe.update_idletasks() - landed = probe.winfo_rooty() - if 0 <= landed <= _AQUA_MENU_MAX: - top = landed - except (tk.TclError, AttributeError, ValueError): - pass + top = probe.winfo_rooty() - title + if not 0 <= top <= _AQUA_MENU_MAX: + return None + + mw, mh = probe.maxsize() + except (tk.TclError, AttributeError, TypeError, ValueError): + return None finally: if probe is not None: try: probe.destroy() except tk.TclError: pass - if top + mh > sh: - # The two measurements disagree about the display. Better to know nothing. - return None - return 0, top, mw, top + mh + + height = mh + title + if not (0 < mw <= sw and 0 < height < sh and top + height <= sh): + return None # it answered with the whole screen, or the parts disagree + return 0, top, mw, top + height def _tk_work_area(win, sw: int, sh: int) -> tuple[int, int, int, int]: diff --git a/scripts/mac_area_probe.py b/scripts/mac_area_probe.py index 2ab81c3..a6f1f63 100644 --- a/scripts/mac_area_probe.py +++ b/scripts/mac_area_probe.py @@ -19,7 +19,10 @@ **wm attributes -fullscreen** - the whole screen including the menu bar, as a control: if this and `zoomed` agree, `zoomed` is being treated as fullscreen. -Between the first two, `bottom = top + maxsize_height` is the Dock's top edge. +Every measurement comes from a window with the same decoration, which matters: +`maxsize` is a maximum *content* size, so it is short by whatever title bar its +window wears. Taking the origin from one window and the size from another counted +a 28 px title bar twice and put the answer 28 px too low. uv run python scripts/mac_area_probe.py """ @@ -56,9 +59,20 @@ def probe(name, setup): probe("zoomed (today's probe)", lambda w: w.state("zoomed")) -_x, top, _w, _h = probe("asked for +0+0", lambda w: w.geometry("200x120+0+0")) +_x, free, _w, _h = probe("asked for +80+300", lambda w: w.geometry("200x120+80+300")) +_x, clamped, _w, _h = probe("asked for +0+0", lambda w: w.geometry("200x120+0+0")) probe("fullscreen (control)", lambda w: w.attributes("-fullscreen", True)) -print(f"\n so the usable area is (0, {top}, {mw}, {top + mh})") -print(f" and the Dock starts at y = {top + mh} ({sh - (top + mh)} px tall)") +# `maxsize` is a maximum *content* size, so it is short by the decoration of the +# window that answered it - 735 from a titled window against 763 from an +# `overrideredirect` one on the same display. Every part of this comes from one +# probe so the title bar appears on both sides and cancels; mixing two windows +# counted it twice and put the answer 28 px too low. +title = free - 300 +top = clamped - title +bottom = top + mh + title +print(f"\n title bar {title} px (from +80+300, below any menu bar)") +print(f" menu bar {top} px (from +0+0, less that title bar)") +print(f" so the usable area is (0, {top}, {mw}, {bottom})") +print(f" and the Dock starts at y = {bottom} ({sh - bottom} px tall)") root.destroy() diff --git a/tests/test_pill.py b/tests/test_pill.py index 012a065..758d1e0 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -784,9 +784,10 @@ class TestTheWorkAreaOnAqua(unittest.TestCase): """`_aqua_work_area`, which exists because the maximise probe is ignored on a Mac. A Mac reported `_tk_work_area()` answering with the whole 1352x878 screen, identical - to `_work_area()`. `state("zoomed")` had neither raised nor maximised, so the - fallback did not fall back, `bottom_centre` stood the pill 24 px above 878, and the - pill sat inside a 69 px Dock. + to `_work_area()`. `state("zoomed")` had neither raised nor maximised — asked to + maximise a 200x120 window at +80+80 it returned the same window at +80+80, and no + error — so the fallback did not fall back, `bottom_centre` stood the pill 24 px above + 878, and the pill sat inside an 85 px Dock. `wm maxsize` is the instrument here and *only* here: Tk's Aqua port answers it from `[NSScreen visibleFrame]`. On Windows the same call reports the whole screen with a @@ -798,40 +799,63 @@ def setUp(self): ui._TK_WORK = None self.addCleanup(lambda: setattr(ui, "_TK_WORK", self._was)) - def win(self, maxsize=(1352, 735), landed=58): + def win(self, maxsize=(1352, 735), title=28, menu=30): """The numbers a 14-inch MacBook Pro actually reported, not invented ones. - `scripts/mac_area_probe.py` on darwin, Tk 9.0.3, a 1352x878 screen: - `wm maxsize` 1352x735, a window asked for `+0+0` landing with its client area at - y 58, and `zoomed` answering (80, 108, 280, 228) - the 200x120 probe still at - the +80+80 it was asked for, never maximised at all. + `scripts/mac_area_probe.py` on darwin, Tk 9.0.3, a 1352x878 screen: a titled + probe asked for +80+300 landing at y 328, so a 28 px title bar; the same probe + asked for +0+0 landing at y 58, so a 30 px menu bar under it; and `maxsize` + 1352x735, which plus the title bar is a 763 px visible frame. """ - win = mock.Mock() - win.maxsize.return_value = maxsize probe = mock.Mock() - probe.winfo_rooty.return_value = landed - return win, probe + probe.maxsize.return_value = maxsize + seen = [] + + def geometry(spec): + seen.append(spec) + + def rooty(): + # +80+300 first, then +0+0 - the order the function asks in. + return ui._AQUA_FREE_Y + title if len(seen) == 1 else menu + title + + probe.geometry.side_effect = geometry + probe.winfo_rooty.side_effect = rooty + return mock.Mock(), probe def call(self, win, probe, sw=1352, sh=878): with mock.patch.object(ui.tk, "Toplevel", return_value=probe): return ui._aqua_work_area(win, sw, sh) def test_the_bottom_is_the_top_of_the_dock(self): - # The whole point: 802, not 878. The pill stands on this number. + # The whole point: 793, not 878. The pill stands on this number. win, probe = self.win() - self.assertEqual(self.call(win, probe), (0, 58, 1352, 793)) + self.assertEqual(self.call(win, probe), (0, 30, 1352, 793)) def test_the_dock_it_finds_matches_the_dock_the_os_reports(self): """85 px of Dock, against a `com.apple.dock tilesize` of 69 read separately. - The reason this platform is believed at all now. `maxsize` and the `+0+0` probe - are both Tk asking Tk; a tile size out of `defaults` came from somewhere else - entirely, and 69 plus Apple's padding is the 85 this leaves. Three earlier + The reason this platform is believed at all now. `maxsize`, the title bar and the + menu bar are all Tk asking Tk; a tile size out of `defaults` came from somewhere + else entirely, and 69 plus Apple's padding is the 85 this leaves. Three earlier guesses at Aqua all failed for want of a second source. """ win, probe = self.win() - _l, _t, _r, bottom = self.call(win, probe) - self.assertEqual(878 - bottom, 85) + self.assertEqual(878 - self.call(win, probe)[3], 85) + + def test_the_title_bar_is_added_back_to_the_content_size(self): + """The bug this shape was written to make impossible. + + `maxsize` is a maximum *content* size, short by whatever decoration its window + wears — 735 from a titled probe against 763 from the `overrideredirect` pill on + the same display. The first version took the origin from a probe and the size + from the caller's window, counted the 28 px title bar twice, and put the work + area at 821: the pill moved off the Dock and straight back onto it. + + One probe answers everything, so its decoration appears on both sides and + cancels. A window with no title bar at all must land on the same answer. + """ + win, bare = self.win(maxsize=(1352, 763), title=0) + self.assertEqual(self.call(win, bare), (0, 30, 1352, 793)) def test_the_probe_is_invisible_and_cleaned_up(self): win, probe = self.win() @@ -842,49 +866,50 @@ def test_the_probe_is_invisible_and_cleaned_up(self): def test_a_maxsize_of_the_whole_screen_means_it_does_not_know(self): # Windows answers this way with a taskbar present. Reporting the whole screen as # a work area is the bug, not a fix for it. - win, probe = self.win(maxsize=(1352, 878)) + win, probe = self.win(maxsize=(1352, 878), title=0, menu=0) self.assertIsNone(self.call(win, probe)) def test_a_window_manager_that_honoured_plus_zero_is_not_a_menu_bar(self): - # Windows puts a window where it is asked and reports a frame inset - 31 px on - # this machine, which is close enough to a menu bar to be believed by accident. - # A number far outside a menu bar's range is a literal placement, so `top` falls - # back to 0 rather than shifting the whole work area by it. - win, probe = self.win(landed=400) - self.assertEqual(self.call(win, probe), (0, 0, 1352, 735)) - - def test_a_probe_that_cannot_be_built_still_leaves_the_height(self): - # The height is the part that keeps the pill off the Dock. Losing the origin - # costs a menu bar of clearance; losing the height costs the whole fix. - win, _probe = self.win() - with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): - self.assertEqual(ui._aqua_work_area(win, 1352, 878), (0, 0, 1352, 735)) + # Windows puts a window where it is asked. A number far outside a menu bar's + # range is a literal placement, and nothing here can be believed. + win, probe = self.win(menu=400) + self.assertIsNone(self.call(win, probe)) + + def test_a_title_bar_too_tall_to_be_one_is_refused(self): + win, probe = self.win(title=200) + self.assertIsNone(self.call(win, probe)) - def test_measurements_that_disagree_are_refused(self): + def test_measurements_that_disagree_about_the_display_are_refused(self): # A menu bar plus a visible frame cannot be taller than the display it is on. - win, probe = self.win(maxsize=(1352, 870), landed=40) + win, probe = self.win(maxsize=(1352, 870)) self.assertIsNone(self.call(win, probe)) def test_a_build_without_maxsize_says_so(self): win, probe = self.win() - win.maxsize.side_effect = ui.tk.TclError("no such command") + probe.maxsize.side_effect = ui.tk.TclError("no such command") self.assertIsNone(self.call(win, probe)) + def test_a_probe_that_cannot_be_built_says_so(self): + win, _probe = self.win() + with mock.patch.object(ui.tk, "Toplevel", side_effect=ui.tk.TclError("no")): + self.assertIsNone(ui._aqua_work_area(win, 1352, 878)) + def test_tk_work_area_prefers_it_on_darwin(self): win, probe = self.win() with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 793)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 30, 1352, 793)) def test_and_falls_through_to_the_maximise_probe_when_it_declines(self): # The guarantee that made this safe to write before a Mac had confirmed it: if # the Aqua path cannot answer, the worst case is exactly the old behaviour. - win, probe = self.win(maxsize=(1352, 878)) + win, probe = self.win(maxsize=(1352, 878), title=0, menu=0) probe.winfo_rootx.return_value = 0 - probe.winfo_rooty.return_value = 58 + probe.winfo_rooty.side_effect = None + probe.winfo_rooty.return_value = 30 probe.winfo_width.return_value = 1352 - probe.winfo_height.return_value = 735 + probe.winfo_height.return_value = 763 with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui.tk, "Toplevel", return_value=probe): - self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 58, 1352, 793)) + self.assertEqual(ui._tk_work_area(win, 1352, 878), (0, 30, 1352, 793)) def test_windows_never_takes_this_path(self): # `maxsize` there is the whole screen, taskbar or not. Asking it would undo the @@ -892,6 +917,8 @@ def test_windows_never_takes_this_path(self): win, probe = self.win() with mock.patch.object(sys, "platform", "win32"), mock.patch.object(ui, "_aqua_work_area") as aqua, mock.patch.object(ui.tk, "Toplevel", return_value=probe): probe.winfo_rootx.return_value = 0 + probe.winfo_rooty.side_effect = None + probe.winfo_rooty.return_value = 23 probe.winfo_width.return_value = 1280 probe.winfo_height.return_value = 649 ui._tk_work_area(win, 1280, 720) From 2a5e3b4e12703bb70fb694d936729b66517e9a0f Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 03:52:53 -0500 Subject: [PATCH 20/41] Ask Aqua which window configuration survives a click on another app Reported from a Mac: click the app you want to dictate into and Flow's window vanishes, and clicking Send does nothing. Those are one fault - a borderless overrideredirect NSWindow is ordered out when its app stops being frontmost, and a click on a background app's window is spent activating that app rather than pressing what is under it. macOS answers this with a nonactivating NSPanel, which is what FluidVoice's overlay is. Tk 9 exposes it as wm attributes -class and -stylemask - named by Flow's own crash, when Tk refused -transparentcolor and listed what this build does accept. The probe asks for a nonsense value first, because Tk answers that by enumerating the legal ones, so a run is informative even if every variant below is wrong. It records what survived the background on its own and logs which buttons a click reached, so the answer comes back as pasted text rather than a judgement call or a photograph. Co-Authored-By: Claude Opus 5 --- scripts/mac_float_probe.py | 133 +++++++++++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+) create mode 100644 scripts/mac_float_probe.py diff --git a/scripts/mac_float_probe.py b/scripts/mac_float_probe.py new file mode 100644 index 0000000..ec44325 --- /dev/null +++ b/scripts/mac_float_probe.py @@ -0,0 +1,133 @@ +"""Which window configuration on Aqua stays visible and still takes a click. + +Two faults reported from a Mac, and they are the same fault: click the app you want to +dictate into and Flow's window disappears, and clicking Send does nothing. That is what a +borderless `overrideredirect` NSWindow does when its application is no longer frontmost — +the window server orders it out, and a click on a background app's window is spent +activating that app rather than pressing what was under the cursor. + +macOS has an answer for exactly this and it is what FluidVoice's overlay is: an NSPanel +with the *nonactivating* style, which floats above other apps and accepts a click without +taking focus. Tk 9 exposes it as `wm attributes -class` and `-stylemask`, which is how +Flow's own crash named them — Tk 8.6 refused `-transparentcolor` and listed what this +build does accept, and those two were in the list. + +**This probe assumes nothing about their vocabulary.** It asks for a deliberately +nonsense value first, because Tk answers that with the legal ones. So a run is useful +even if every guess below is wrong. + + uv run python scripts/mac_float_probe.py + +It prints what it is doing, waits while you click another application, records which +windows survived that, and then logs which buttons a click actually reached. Paste the +whole output back — no judgement calls, no photographs. +""" + +import sys +import tkinter as tk + +AWAY_SEC = 8 +CLICK_SEC = 25 +W, H = 360, 62 + +root = tk.Tk() +root.withdraw() +log: list[str] = [] + + +def vocabulary(name: str) -> None: + """Ask for a nonsense value; Tk's complaint enumerates the real ones.""" + try: + root.wm_attributes(name, "there-is-no-such-value") + print(f" {name:<12} accepted nonsense - it is not validated on this build") + except tk.TclError as exc: + print(f" {name:<12} {exc}") + + +print(f"platform {sys.platform}, tk {tk.TkVersion} " + f"patch {root.tk.call('info', 'patchlevel')}\n") +print("What this build will accept:") +for attribute in ("-class", "-stylemask"): + vocabulary(attribute) + + +def window(n: int, title: str, setup) -> tk.Toplevel: + """One variant, built while unmapped because a class cannot be changed after. + + Whatever `setup` raises is printed and the window is still shown - a variant that + could not be configured is a *result*, and it should be on screen wearing the reason + so that what is seen and what is printed cannot come apart. + """ + win = tk.Toplevel(root) + win.withdraw() + note = "ok" + try: + setup(win) + except tk.TclError as exc: + note = f"FAILED ({exc})" + win.geometry(f"{W}x{H}+40+{40 + n * (H + 16)}") + win.configure(bg="#12141a") + tk.Label(win, text=f"{n} {title}", bg="#12141a", fg="#e6e8ee", + font=("Helvetica", 12)).pack(side="left", padx=10) + tk.Button(win, text=f"click {n}", highlightbackground="#12141a", + command=lambda: log.append(f"button {n} was reached")).pack( + side="right", padx=10) + win.deiconify() + win.attributes("-topmost", True) + print(f" {n} {title:<44} {note}") + return win + + +def panel(win, *flags): + win.wm_attributes("-class", "nspanel") + if flags: + win.wm_attributes("-stylemask", " ".join(flags)) + + +print("\nVariants:") +made = { + 1: window(1, "overrideredirect (what Flow does today)", + lambda w: w.overrideredirect(True)), + 2: window(2, "nspanel, nonactivatingpanel", + lambda w: panel(w, "nonactivatingpanel")), + 3: window(3, "nspanel, nonactivatingpanel + utility", + lambda w: panel(w, "nonactivatingpanel", "utility")), + 4: window(4, "overrideredirect + nspanel, nonactivatingpanel", + lambda w: (w.overrideredirect(True), panel(w, "nonactivatingpanel"))), +} + + +def survey() -> None: + """What is still on screen now that this app is not the frontmost one.""" + print(f"\nAfter {AWAY_SEC}s in the background:") + for n, win in made.items(): + try: + print(f" {n} viewable={bool(win.winfo_viewable())} " + f"mapped={bool(win.winfo_ismapped())} " + f"at ({win.winfo_rootx()}, {win.winfo_rooty()})") + except tk.TclError as exc: + print(f" {n} gone ({exc})") + print(f"\nNow click every button once, in any order. {CLICK_SEC}s.") + root.after(CLICK_SEC * 1000, finish) + + +def finish() -> None: + print("\nButtons a click actually reached:") + for n in made: + hit = f"button {n} was reached" in log + print(f" {n} {'reached' if hit else 'NOTHING GOT THROUGH'}") + print("\nThe variant that is both still visible above and reached here is the one " + "Flow should\nbe built from. Paste all of this back.") + root.destroy() + + +print(f""" +Four windows are on screen down the left. + + **Click another application now** - Finder, a browser, anything - and leave Flow in + the background. This is the whole test: these windows are meant to survive that, and + today's configuration does not. + +Recording in {AWAY_SEC}s.""") +root.after(AWAY_SEC * 1000, survey) +root.mainloop() From d4ba8b8973b5092da625a91e681fbca3b269fe6a Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 04:00:09 -0500 Subject: [PATCH 21/41] overrideredirect is the cause; ask which style mask is bare without it The first run answered a question it was not asked. Two variants had overrideredirect and two did not, and the two without were both the ones a click reached and the ones still on screen after clicking another app. overrideredirect on Aqua makes a window deaf and fugitive at once, and it is the one line Flow uses on every window it owns. The NSPanel class it was written to test never applied - cannot change the class after the mac window is created, even withdrawn - so it was never the variable. That leaves what overrideredirect was for: taking the title bar off. Tk's complaint about a nonsense style bit listed the real ones, and a mask without the titled bit is a borderless window that was never made deaf. The title bar is now measured rather than looked at - ask for a known y, and how far below it the client area lands is the decoration - so five of the six answers come back in a paste. winfo_viewable is no longer trusted for the sixth: it reported all four windows healthy while two were off the screen. Co-Authored-By: Claude Opus 5 --- scripts/mac_float_probe.py | 123 ++++++++++++++++++++----------------- 1 file changed, 67 insertions(+), 56 deletions(-) diff --git a/scripts/mac_float_probe.py b/scripts/mac_float_probe.py index ec44325..fd733e7 100644 --- a/scripts/mac_float_probe.py +++ b/scripts/mac_float_probe.py @@ -1,26 +1,34 @@ -"""Which window configuration on Aqua stays visible and still takes a click. +"""Which window configuration on Aqua is bare, stays up, and still takes a click. -Two faults reported from a Mac, and they are the same fault: click the app you want to -dictate into and Flow's window disappears, and clicking Send does nothing. That is what a -borderless `overrideredirect` NSWindow does when its application is no longer frontmost — -the window server orders it out, and a click on a background app's window is spent -activating that app rather than pressing what was under the cursor. +Reported from a Mac: click the app you want to dictate into and Flow's window vanishes, +and clicking Send does nothing. -macOS has an answer for exactly this and it is what FluidVoice's overlay is: an NSPanel -with the *nonactivating* style, which floats above other apps and accepts a click without -taking focus. Tk 9 exposes it as `wm attributes -class` and `-stylemask`, which is how -Flow's own crash named them — Tk 8.6 refused `-transparentcolor` and listed what this -build does accept, and those two were in the list. +**The first run of this probe found the cause, and it was not the one it was written to +test.** Four variants, two with `overrideredirect` and two without. The two without were +the two whose buttons a click reached, and the two without were the two still on screen +after clicking another app. `overrideredirect` on Aqua makes a window both deaf and +fugitive, and it is the one line Flow uses on every window it owns. -**This probe assumes nothing about their vocabulary.** It asks for a deliberately -nonsense value first, because Tk answers that with the legal ones. So a run is useful -even if every guess below is wrong. +The NSPanel class this was really written to test never applied at all - Tk answered +`cannot change the class after the mac window is created`, even on a withdrawn window - +so the panel was never the variable. + +That leaves the question `overrideredirect` was there to answer: it is what takes the +title bar off. Tk's own complaint listed the way out. Asked for a nonsense style bit it +enumerated the real ones - `titled, closable, miniaturizable, resizable, +fullsizecontentview, utility, nonactivatingpanel, docmodal` - and a style mask *without* +`titled` is a window with no title bar that was never made deaf to begin with. + +**A title bar is measured here, not looked at.** Ask for a window at a known y; how far +below that the client area lands is the decoration on it, and zero means bare. Same +arithmetic `_aqua_work_area` uses to find the menu bar. `winfo_viewable` is not trusted +for the disappearing - it reported all four windows healthy while two of them were gone +from the screen - so that one bit is the only thing left worth a human glance. uv run python scripts/mac_float_probe.py -It prints what it is doing, waits while you click another application, records which -windows survived that, and then logs which buttons a click actually reached. Paste the -whole output back — no judgement calls, no photographs. +Click another application when it says to, note which numbers vanish, then click every +button. Paste the output. """ import sys @@ -28,11 +36,13 @@ AWAY_SEC = 8 CLICK_SEC = 25 -W, H = 360, 62 +ASK_Y = 220 +W, H = 420, 58 root = tk.Tk() root.withdraw() -log: list[str] = [] +log: list[int] = [] +decor: dict[int, int] = {} def vocabulary(name: str) -> None: @@ -51,82 +61,83 @@ def vocabulary(name: str) -> None: vocabulary(attribute) -def window(n: int, title: str, setup) -> tk.Toplevel: - """One variant, built while unmapped because a class cannot be changed after. +def mask(win, *flags): + """A style mask with no `titled` bit is a window with no title bar.""" + win.wm_attributes("-stylemask", " ".join(flags)) + + +def window(n: int, title: str, setup, cls=None) -> tk.Toplevel: + """One variant, configured while unmapped, then measured for decoration. Whatever `setup` raises is printed and the window is still shown - a variant that - could not be configured is a *result*, and it should be on screen wearing the reason - so that what is seen and what is printed cannot come apart. + could not be configured is a *result*, and it belongs on screen wearing its reason so + that what is seen and what is printed cannot come apart. """ - win = tk.Toplevel(root) + win = tk.Toplevel(root, class_=cls) if cls else tk.Toplevel(root) win.withdraw() note = "ok" try: setup(win) except tk.TclError as exc: note = f"FAILED ({exc})" - win.geometry(f"{W}x{H}+40+{40 + n * (H + 16)}") + asked = ASK_Y + n * (H + 14) + win.geometry(f"{W}x{H}+40+{asked}") win.configure(bg="#12141a") tk.Label(win, text=f"{n} {title}", bg="#12141a", fg="#e6e8ee", - font=("Helvetica", 12)).pack(side="left", padx=10) + font=("Helvetica", 11)).pack(side="left", padx=10) tk.Button(win, text=f"click {n}", highlightbackground="#12141a", - command=lambda: log.append(f"button {n} was reached")).pack( - side="right", padx=10) + command=lambda: log.append(n)).pack(side="right", padx=10) win.deiconify() win.attributes("-topmost", True) - print(f" {n} {title:<44} {note}") + win.update_idletasks() + decor[n] = win.winfo_rooty() - asked + print(f" {n} {title:<44} title bar {decor[n]:>3} px {note}") return win -def panel(win, *flags): - win.wm_attributes("-class", "nspanel") - if flags: - win.wm_attributes("-stylemask", " ".join(flags)) - - -print("\nVariants:") +print("\nVariants (title bar measured, not looked at):") made = { - 1: window(1, "overrideredirect (what Flow does today)", + 1: window(1, "nothing asked for (the control)", lambda w: None), + 2: window(2, "overrideredirect (what Flow does today)", lambda w: w.overrideredirect(True)), - 2: window(2, "nspanel, nonactivatingpanel", - lambda w: panel(w, "nonactivatingpanel")), - 3: window(3, "nspanel, nonactivatingpanel + utility", - lambda w: panel(w, "nonactivatingpanel", "utility")), - 4: window(4, "overrideredirect + nspanel, nonactivatingpanel", - lambda w: (w.overrideredirect(True), panel(w, "nonactivatingpanel"))), + 3: window(3, "stylemask {} - no bits at all", lambda w: mask(w)), + 4: window(4, "stylemask {fullsizecontentview}", + lambda w: mask(w, "fullsizecontentview")), + 5: window(5, "stylemask {} on a Toplevel made as NSPanel", + lambda w: mask(w), cls="NSPanel"), + 6: window(6, "stylemask {nonactivatingpanel} as NSPanel", + lambda w: mask(w, "nonactivatingpanel"), cls="NSPanel"), } def survey() -> None: - """What is still on screen now that this app is not the frontmost one.""" - print(f"\nAfter {AWAY_SEC}s in the background:") + print(f"\nAfter {AWAY_SEC}s in the background, Tk claims:") for n, win in made.items(): try: - print(f" {n} viewable={bool(win.winfo_viewable())} " - f"mapped={bool(win.winfo_ismapped())} " - f"at ({win.winfo_rootx()}, {win.winfo_rooty()})") + print(f" {n} viewable={bool(win.winfo_viewable())}") except tk.TclError as exc: print(f" {n} gone ({exc})") + print(" (Tk said all four were healthy last time while two were off the screen,\n" + " so please say which numbers you can actually still see.)") print(f"\nNow click every button once, in any order. {CLICK_SEC}s.") root.after(CLICK_SEC * 1000, finish) def finish() -> None: - print("\nButtons a click actually reached:") + print("\nResults:") for n in made: - hit = f"button {n} was reached" in log - print(f" {n} {'reached' if hit else 'NOTHING GOT THROUGH'}") - print("\nThe variant that is both still visible above and reached here is the one " - "Flow should\nbe built from. Paste all of this back.") + print(f" {n} title bar {decor.get(n, -1):>3} px " + f"{'CLICK REACHED IT' if n in log else 'nothing got through'}") + print("\nWanted: title bar 0 px, still on screen, and the click reaching it.\n" + "Paste all of this back, with which numbers stayed visible.") root.destroy() print(f""" -Four windows are on screen down the left. +Six windows are on screen down the left. **Click another application now** - Finder, a browser, anything - and leave Flow in - the background. This is the whole test: these windows are meant to survive that, and - today's configuration does not. + the background. Note which numbers disappear. Recording in {AWAY_SEC}s.""") root.after(AWAY_SEC * 1000, survey) From 23c0995313d02c3f1f0a5b02b4506b121c265a44 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 04:05:35 -0500 Subject: [PATCH 22/41] Take the frame off Aqua with an empty style mask, not overrideredirect Two faults reported from a Mac - click the app you want to dictate into and Flow's window vanishes, and clicking Send does nothing - and one cause. Six variants on screen split on exactly this line: every window without overrideredirect kept its place when another app came forward and had its button reached by a click, and every window with it was deaf and gone. Which is a fair description of what it means - a window the window manager has been told to stop managing. A style mask is the set of bits an NSWindow is built from and titled is the one that puts a title bar on, so a mask with no bits is bare with nothing else given away. Measured at 0 px of decoration against the control's 28, and its button was reached from the background. The NSPanel route is closed and unneeded: Tk answers cannot change the class after the mac window is created even for a window never mapped, and Toplevel(class_=...) names a Tk class rather than an NSWindow one. Nonactivating is about not stealing focus, and these windows never take it. Tk 8.6 on a Mac falls back to the old line rather than getting a frame. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 43 ++++++++++++++++++++++++++++++++++--- tests/test_pill.py | 53 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 91 insertions(+), 5 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index e30e0fe..990d6ff 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -188,6 +188,42 @@ def _no_activate(win) -> bool: return False +def _bare_window(win) -> None: + """Take the frame off, by the means the platform will accept. + + `overrideredirect` is how this is done everywhere, and on Aqua it is the cause of + both faults reported from a Mac: click the app you want to dictate into and Flow's + window vanishes, and clicking Send does nothing. A probe of six variants split on + exactly this line — every window without it kept its place when another app came + forward and had its button reached by a click; every window with it was deaf and + gone. Which is a fair description of what it means: a window the window manager has + been told to stop managing. + + Tk 9 on Aqua has a frameless window that is still a window. A style mask is the set + of bits an NSWindow is built from, and the one that puts a title bar on it is + `titled` — so a mask with *no* bits is bare, and nothing else about the window has + been given away. Measured on a Mac at 0 px of decoration against the control's 28, + and its button was reached from the background. + + The obvious-looking alternative, an `NSPanel` with the `nonactivatingpanel` bit, is + not available: Tk answers `cannot change the class after the mac window is created` + even for a window that has never been mapped, and a `Toplevel` built with + `class_="NSPanel"` is not one either — that argument names a Tk class, not an + NSWindow one. It is also not needed. Nonactivating is about not stealing focus, and + these windows do not take focus in the first place. + + Falls back rather than fails: `-stylemask` arrived in Tk 9, and a Mac on 8.6 should + get the old behaviour rather than a window with a title bar on it. + """ + if sys.platform == "darwin": + try: + win.wm_attributes("-stylemask", "") + return + except tk.TclError: + pass # Tk 8.6: no style masks. `overrideredirect` is all there is. + win.overrideredirect(True) + + def _shell_window(win, lite: bool, alpha: float) -> str: """Apply the window attributes every Flow window shares, and return its background. @@ -204,8 +240,9 @@ def _shell_window(win, lite: bool, alpha: float) -> str: An invariant a caller has to know about is one a caller can miss, and this one is cheap to enforce where it is true. - **`overrideredirect` is the whole of it on Aqua too, and two attempts to help it were - both harm.** A Mac reported the pill wearing a title bar, and the cause was not this + **On Aqua it is `_bare_window` that does this**, and not with `overrideredirect`: + that line is what made Flow's windows there both deaf to clicks and gone the moment + another app came forward. Two earlier attempts to help it were both harm.** A Mac reported the pill wearing a title bar, and the cause was not this line failing — it was `MacWindowStyle` being asked for *afterwards*, which put a frame back on and took the window out of the activation chain, so Send stopped taking clicks. Removing that call was the fix. A withdraw-and-remap cycle added alongside it, @@ -216,7 +253,7 @@ def _shell_window(win, lite: bool, alpha: float) -> str: The background is returned rather than left to the caller so a window cannot be given one that contradicts what was applied to it. """ - win.overrideredirect(True) + _bare_window(win) win.attributes("-topmost", True) win.attributes("-alpha", alpha) if lite or sys.platform != "win32": diff --git a/tests/test_pill.py b/tests/test_pill.py index 758d1e0..6c923b4 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -769,17 +769,66 @@ def test_full_mode_on_windows_still_asks_for_both(self): win.attributes.assert_any_call("-transparentcolor", ui.TRANSPARENT) win.attributes.assert_any_call("-toolwindow", True) - def test_the_shared_three_are_asked_for_everywhere(self): + def test_the_shared_two_are_asked_for_everywhere(self): for platform in ("darwin", "win32", "linux"): with self.subTest(platform=platform): win = mock.Mock() with mock.patch.object(sys, "platform", platform): ui._shell_window(win, lite=True, alpha=0.5) - win.overrideredirect.assert_called_once_with(True) win.attributes.assert_any_call("-topmost", True) win.attributes.assert_any_call("-alpha", 0.5) +class TestTakingTheFrameOff(unittest.TestCase): + """`_bare_window`, and why Aqua does not get `overrideredirect`. + + Two faults reported from a Mac - click the app you want to dictate into and Flow's + window vanishes, and clicking Send does nothing - and one cause. Six variants were + put on screen and the results split on exactly this line: every window without + `overrideredirect` kept its place when another app came forward and had its button + reached by a click, and every window with it was deaf and gone. + + A style mask with no bits is the replacement. `titled` is the bit that puts a title + bar on, so a mask with none is bare, and nothing else about the window has been given + away. Measured at 0 px of decoration on a Mac against the control's 28. + """ + + def test_aqua_asks_for_an_empty_style_mask_and_not_overrideredirect(self): + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"): + ui._bare_window(win) + win.wm_attributes.assert_called_once_with("-stylemask", "") + win.overrideredirect.assert_not_called() + + def test_everywhere_else_is_unchanged(self): + # `-stylemask` is an Aqua attribute. Windows and X11 have never needed it, and + # `overrideredirect` is not the cause of anything there. + for platform in ("win32", "linux"): + with self.subTest(platform=platform): + win = mock.Mock() + with mock.patch.object(sys, "platform", platform): + ui._bare_window(win) + win.overrideredirect.assert_called_once_with(True) + win.wm_attributes.assert_not_called() + + def test_a_mac_on_tk_8_6_falls_back_rather_than_wearing_a_title_bar(self): + # `-stylemask` arrived in Tk 9. Older builds should get the behaviour they + # always had, which is imperfect but not a window with a frame on it. + win = mock.Mock() + win.wm_attributes.side_effect = ui.tk.TclError("bad attribute") + with mock.patch.object(sys, "platform", "darwin"): + ui._bare_window(win) + win.overrideredirect.assert_called_once_with(True) + + def test_every_window_flow_owns_goes_through_it(self): + # The pill, the bubble, the card and the help panel all build their shell here, + # and a window that missed this would be the one wearing a frame. + win = mock.Mock() + with mock.patch.object(sys, "platform", "darwin"), mock.patch.object(ui, "_bare_window") as bare: + ui._shell_window(win, lite=True, alpha=0.9) + bare.assert_called_once_with(win) + + class TestTheWorkAreaOnAqua(unittest.TestCase): """`_aqua_work_area`, which exists because the maximise probe is ignored on a Mac. From 76834ff262774f0bb8565be95979fb8a6d49dbfd Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 04:28:58 -0500 Subject: [PATCH 23/41] Ask the CLI for low effort by default, and let a model be chosen Refinement latency, reported from a Mac where Kiro's default model was slow and failing on retry. Two levers, both read out of each CLI's own --help on a machine that has all three rather than written from memory: --model on all of codex, claude and kiro-cli, and --effort (low, medium, high, xhigh, max) on the last two. codex has no effort flag - its only route is a config key that does not appear in its help - so it is not given one. Effort defaults to low, deliberately. These calls are a rewrite, not a reasoning problem, and the user is watching a spinner between finishing a sentence and having their words. tuned() returns a new Cli rather than threading two arguments through four call layers: Cli is frozen and its argv is a tuple, so everything downstream - resolve, _invoke, _clean, the pill's marker - goes on working unchanged. The flags land after the subcommand and before codex's trailing -, the positional saying the prompt is on stdin, which a flag appended at the end would be read as the value of. Fallbacks are tuned too: reverting to a CLI's own defaults the moment the first candidate failed would be slowest exactly when the user is already waiting longest. Model names cannot be discovered - no CLI will list them - and Flow has no text field to type one into, so --cli-model carries a name in once and the menu is built from the names that have been used. Effort is a menu from the start, being enumerable. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 30 +++++++++++++ flow/__main__.py | 34 ++++++++++++++- flow/profile.py | 26 +++++++++++ flow/refine.py | 101 +++++++++++++++++++++++++++++++++++++++---- flow/session.py | 43 ++++++++++++++++++ flow/ui.py | 55 ++++++++++++++++++++++- tests/test_refine.py | 96 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 373 insertions(+), 12 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index 4123dd1..8c5ef93 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -179,6 +179,34 @@ screen is the CLI that will be called, not the first one on PATH. `codex` now measures 6.6–8.5 s here for a one-word answer, so a long question can breach it. +### Which model, and how hard it thinks + +**Effort is `low` unless you say otherwise**, and that is the setting most worth knowing +about. These calls are a *rewrite* — take what was dictated and make it read like a +written prompt — not a reasoning problem, and the whole time one runs you are watching a +spinner between finishing a sentence and having your words. **Effort** in the right-click +menu offers every level the CLIs do: `low`, `medium`, `high`, `xhigh`, `max`. + +It reaches the CLIs that offer the choice, which is `claude` and `kiro-cli`. `codex` has +no effort flag — its only route is a config key that does not appear in its help, and +Flow will not write down a flag nobody has seen it print. + +Choosing a **model** takes one run of `--cli-model`: + +``` +uv run python -m flow --cli-model gpt-5.6-luna +``` + +After that it is a click, under **Model** in the same menu, alongside every other name +you have used and **The CLI's own default**. It has to arrive by flag once because no CLI +will list its models — `codex exec --help` says `-m, --model ` and stops — and +Flow has no text field to type one into. Settings is a menu, not a dialog. + +Both apply to whichever CLI answers, including a fallback: a walk that reverted to the +CLI's own defaults the moment the first candidate failed would be slowest exactly when +you are already waiting longest. A model name means nothing to a CLI that takes no +`--model`, and is dropped for that CLI rather than breaking the call. + ## Running it ```bash @@ -237,6 +265,8 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--voice X` | voice for spoken replies: a name, part of one, or `male`/`female` | | `--no-auto-ask` | in converse mode, wait for the Ask button instead of a pause | | `--cli NAME` | pin the agent CLI (`codex` or `claude`) instead of trying each in turn | +| `--cli-model NAME` | ask the CLI for this model; remembered, and blank clears it | +| `--cli-effort LEVEL` | how hard the CLI may think: `low`, `medium`, `high`, `xhigh`, `max` (default `low`) | | `--cli-timeout SEC` | how long to wait for one CLI call (default 20) | | `--cwd PATH` | the project converse-mode questions are asked from; overrides the stored `workspace` ([P9](#converse-mode-p9)) | | `--lite` | clipboard-out mode: Send copies the draft instead of pasting it, and no hotkeys are registered (automatic off Windows — see [Install](#install)) | diff --git a/flow/__main__.py b/flow/__main__.py index 0524f04..ab81fc4 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -28,7 +28,7 @@ from .lexicon import DEFAULT_PATH, NUL_PATH, Lexicon from .refine import MAX_TIMEOUT_SEC from .refine import TIMEOUT_SEC as REFINE_TIMEOUT_SEC -from .refine import CANDIDATES, available, named, unverified, unverified_note +from .refine import CANDIDATES, EFFORT_DEFAULT, EFFORTS, available, named, unverified, unverified_note from .session import AUTO_ASK_SEC, Session from .stats import TYPING_WPM from .stats import report as stats_report @@ -328,6 +328,19 @@ def main(argv: list[str] | None = None) -> int: help="pin the agent CLI instead of trying each in turn " f"({', '.join(c.name for c in CANDIDATES if c.verified)})", ) + ap.add_argument( + "--cli-model", default=None, metavar="NAME", + # The only way a model name gets into Flow: the settings menu has no text field + # and is not growing one, so a name arrives here once, is remembered, and is a + # click from then on. Not validated against a list because no CLI will print one + # - `codex exec --help` says `-m, --model ` and stops there. + help="ask the agent CLI for this model (remembered; blank clears it)", + ) + ap.add_argument( + "--cli-effort", default=None, choices=EFFORTS, metavar="LEVEL", + help=f"how hard the CLI may think, where it offers the choice " + f"({', '.join(EFFORTS)}; default {EFFORT_DEFAULT})", + ) ap.add_argument( "--cli-timeout", type=_timeout_arg, default=REFINE_TIMEOUT_SEC, metavar="SEC", help=f"how long to wait for a CLI call (default {REFINE_TIMEOUT_SEC:.0f})", @@ -481,12 +494,29 @@ def main(argv: list[str] | None = None) -> int: f"{final_name} for finals{engine_why}") from .diag import Diag - from .profile import Profile, resolve_workspace + from .profile import CLI_MODEL_CAP, Profile, resolve_workspace # Tied to the same flag as the profile, and deliberately: --no-profile means # "write nothing about me this session", and a trace is a thing written about # somebody even when it holds none of their words. profile = None if args.no_profile else Profile() + # Written to the profile before the session reads it, so `--cli-model` is a *setting* + # and not a one-run override: the settings menu has no way to type a name, so a flag + # that vanished at exit would leave the menu permanently empty. `--cli-model ""` + # clears it, which is why this tests for None rather than truthiness. + if profile is not None and args.cli_model is not None: + profile.cli_model = args.cli_model.strip() + if profile.cli_model and profile.cli_model not in profile.cli_models: + profile.cli_models = (*profile.cli_models, profile.cli_model)[-CLI_MODEL_CAP:] + profile.save() + if profile is not None and args.cli_effort is not None: + profile.cli_effort = args.cli_effort + profile.save() + if profile is not None: + if profile.cli_model: + say(f"model: {profile.cli_model}") + say(f"effort: {profile.cli_effort} (where the CLI offers the choice)") + diag = None if args.no_profile else Diag() learned = profile.learned_terms if profile is not None else None if profile is not None and profile.calibrated: diff --git a/flow/profile.py b/flow/profile.py index 02ea0b2..42150a8 100644 --- a/flow/profile.py +++ b/flow/profile.py @@ -35,6 +35,7 @@ from typing import Sequence from . import edits +from .refine import EFFORT_DEFAULT, EFFORTS # -- per-field validation --------------------------------------------------- @@ -222,6 +223,11 @@ def _counter(value) -> Counter: #: Mac, and this one is read on every launch including Lite's. GESTURE_DEFAULT = "hold" +#: How many model names the settings menu will remember. A ceiling rather than a +#: judgement: this list is only ever appended to, by hand, one name at a time, and a menu +#: is not a place for an unbounded list. +CLI_MODEL_CAP = 12 + def path_key(path: str | None) -> str | None: """One identity for one folder, however it was spelled. @@ -402,6 +408,16 @@ def __init__(self, path: Path | str | None = None) -> None: #: paragraph or a pair of hands that cannot hold two keys down. Judged in #: `flow/hotkey.py`, which knows what the words mean. self.gesture: str = GESTURE_DEFAULT + #: Which model to ask the agent CLI for, "" meaning whatever it defaults to, and + #: how hard to let it think. Both apply to whichever CLI answers - `refine.tuned` + #: drops either for a CLI not measured to accept it. + self.cli_model: str = "" + self.cli_effort: str = EFFORT_DEFAULT + #: Every model name that has been set, in the order they were first used. The + #: settings menu has no text field to type one into and is not getting one, so + #: this list *is* the menu: a name arrives once through `--cli-model` and is a + #: click from then on. + self.cli_models: tuple[str, ...] = () #: Field names that were present in the file and unusable, so a caller can say so #: rather than leaving the user to notice their setting reverted. Empty on a first #: run and on any valid file. @@ -482,6 +498,13 @@ def take(key, validate, default=None, stored=None): self.panel = take("panel", _text, PANEL_DEFAULT) self.place = take("place", _text, PLACE_DEFAULT) self.gesture = take("gesture", _text, GESTURE_DEFAULT) + self.cli_model = take("cli_model", _text, "") + self.cli_effort = take("cli_effort", _text, EFFORT_DEFAULT) + if self.cli_effort not in EFFORTS: + self.cli_effort = EFFORT_DEFAULT + self.cli_models = tuple( + take("cli_models", lambda v, _d=None: _text_list(v, CLI_MODEL_CAP), []) + ) self.pairs = take("pairs", lambda v, _d: _counter(v), Counter()) self.misroutes = take("misroutes", lambda v, _d: _counter(v), Counter()) # `stored=[]` because JSON has no set: `save` writes this one as a sorted list, @@ -520,6 +543,9 @@ def save(self) -> bool: "panel": self.panel, "place": self.place, "gesture": self.gesture, + "cli_model": self.cli_model, + "cli_effort": self.cli_effort, + "cli_models": list(self.cli_models), "pairs": dict(self.pairs.most_common(MAX_PAIRS)), "misroutes": dict(self.misroutes.most_common(MAX_MISROUTES)), # Sorted so two saves of the same state produce the same file — a set's diff --git a/flow/refine.py b/flow/refine.py index a8b366f..39015ac 100644 --- a/flow/refine.py +++ b/flow/refine.py @@ -21,7 +21,7 @@ import subprocess import threading import time -from dataclasses import dataclass +from dataclasses import dataclass, replace #: R11: never hand the CLI an unbounded draft. Past this, only the tail is sent. MAX_CHARS = 2000 @@ -211,6 +211,24 @@ class Cli: #: for everything that already fits, which is most things — see `ui.Pill.MARKER_MAX` #: for the wall and `CANDIDATES` for the one entry that has hit it. marker: str = "" + #: The flag that names a model, or empty where this CLI has not been measured to take + #: one. Read out of each CLI's own `--help` on a machine that has it, on 2026-08-31: + #: `-m, --model ` for codex, `--model ` for claude, `--model ` + #: for kiro-cli. The same discipline `verified` carries — a flag nobody has seen a + #: CLI print is a guess. + model_flag: str = "" + #: The flag that sets reasoning effort, or empty where this CLI does not offer one. + #: claude and kiro-cli both print `(low, medium, high, xhigh, max)` beside it. codex + #: has none: its only route is `-c model_reasoning_effort=…`, a config key that does + #: not appear in its help, and writing one down from memory is the thing `verified` + #: exists to forbid. + effort_flag: str = "" + #: Where in `argv` a tuning flag may be inserted. Not the end: codex's argv finishes + #: with `-`, the positional that tells it the prompt is on stdin, and a flag after it + #: would be read as its value. Not the front either, because `exec` and `chat` are + #: subcommands and a flag before them belongs to a different parser. So each entry + #: says where its own flags start, which is immediately after its subcommand. + tune_at: int = 1 #: Order is the preference order — codex first, per R10. @@ -290,11 +308,14 @@ class Cli: #: shapes, not worked around here. CANDIDATES: tuple[Cli, ...] = ( Cli("codex", ("codex", "exec", "--skip-git-repo-check", "-s", "read-only", - "-c", "project_doc_max_bytes=0", "-"), stdin_ok=True), - Cli("claude", ("claude", "--safe-mode", "-p"), stdin_ok=True), + "-c", "project_doc_max_bytes=0", "-"), stdin_ok=True, + model_flag="-m", tune_at=2), + Cli("claude", ("claude", "--safe-mode", "-p"), stdin_ok=True, + model_flag="--model", effort_flag="--effort"), Cli("kiro-cli", ("kiro-cli", "chat", "--no-interactive", "--trust-tools="), probe=(r"%LOCALAPPDATA%\Kiro-Cli\kiro-cli.exe",), - timeout_sec=60.0, marker="kiro"), + timeout_sec=60.0, marker="kiro", + model_flag="--model", effort_flag="--effort", tune_at=2), Cli("opencode", ("opencode",), verified=False), Cli("copilot", ("copilot",), verified=False), Cli("gemini", ("gemini",), verified=False), @@ -388,6 +409,52 @@ def trusted(path: str | None) -> str | None: return path +#: The effort levels claude and kiro-cli both enumerate beside `--effort`, in the order +#: they print them. Read out of their own `--help` rather than written from memory, and +#: identical across the two, which is the only reason one list can serve both. +EFFORTS: tuple[str, ...] = ("low", "medium", "high", "xhigh", "max") + +#: The level Flow asks for when nobody has said otherwise, and deliberately the cheapest. +#: +#: These calls are a *rewrite* — take what was dictated and make it read like a written +#: prompt — not a reasoning problem. Effort buys deliberation this task has no use for, +#: and pays for it in the one currency that matters here: the user is watching a spinner +#: between finishing a sentence and having it. The measured slowest verified call was +#: 35.8 s at whatever the CLI's own default was. +#: +#: Anyone who disagrees for their own model can say so — `--cli-effort`, or the settings +#: menu — and every level the CLIs offer is reachable from both. +EFFORT_DEFAULT = "low" + + +def tuned(cli: Cli, model: str = "", effort: str = EFFORT_DEFAULT) -> Cli: + """`cli` with a model and an effort level asked for, where it accepts them. + + A **new `Cli`** rather than an argument threaded through four call layers. `Cli` is + frozen and its argv is a tuple, so a tuned copy is the same kind of thing as the + original and every caller below — `resolve`, `_invoke`, `_clean`, the marker the pill + draws — goes on working without knowing this happened. + + Silently ignores what a CLI has not been measured to take. That is the point of + `model_flag` and `effort_flag` being per-entry: a user who picks a model while codex + is answering should get codex answering, not a crash and not an invented flag. What + they asked for is still recoverable — the name is on the profile and applies the + moment a CLI that takes one is chosen. + + Inserted at `tune_at`, which is after the subcommand and before codex's trailing `-`. + See that field for why neither end of the argv would do. + """ + extra: list[str] = [] + if effort and effort != "default" and cli.effort_flag: + extra += [cli.effort_flag, effort] + if model and cli.model_flag: + extra += [cli.model_flag, model] + if not extra: + return cli + at = max(1, min(cli.tune_at, len(cli.argv))) + return replace(cli, argv=cli.argv[:at] + tuple(extra) + cli.argv[at:]) + + def resolve(cli: Cli) -> str | None: """Where this CLI actually is, or None. The one answer detection and launch share. @@ -447,6 +514,8 @@ def _invoke_any( cwd: str | None = None, cancel: threading.Event | None = None, skipped: list[str] | None = None, + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str, Cli | None]: """Run `prompt`, falling through the preference order until one CLI answers. @@ -505,7 +574,11 @@ def _invoke_any( """ timeout = sane_timeout(timeout) if cli is not None: - out, reason = _invoke(cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel) + out, reason = _invoke(tuned(cli, model, effort), prompt, timeout=timeout, + cwd=cwd, cancel=cancel) + # The *untuned* entry goes back to the caller. What answered is still codex, and + # the pill's marker, the trace and `_clean`'s per-CLI stripping all key off that + # identity — a tuned copy is the same CLI with a flag on it, not another one. return out, reason, cli candidates = available() @@ -525,8 +598,12 @@ def _invoke_any( # fallback was built to end. reasons.append(f"no time left to try {candidate.name}") break - out, reason = _invoke(candidate, prompt, timeout=timeout, cwd=cwd, - cancel=cancel, cap=left) + # Tuned here rather than at the pin, so a fallback is asked for the same model + # and effort as the first choice. A walk that quietly reverted to a CLI's own + # defaults the moment the first candidate failed would be slowest exactly when + # the user is already waiting longest. + out, reason = _invoke(tuned(candidate, model, effort), prompt, timeout=timeout, + cwd=cwd, cancel=cancel, cap=left) if out is not None: if skipped is not None: skipped.extend(reasons) @@ -951,6 +1028,8 @@ def refine( cancel: threading.Event | None = None, skipped: list[str] | None = None, app: str = "", + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str]: """Apply a semantic instruction to `text`. @@ -1000,7 +1079,8 @@ def refine( ) out, reason, chosen = _invoke_any( - cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped + cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped, + model=model, effort=effort, ) if out is None: return None, reason @@ -1029,6 +1109,8 @@ def ask( cancel: threading.Event | None = None, artifact: bool = False, skipped: list[str] | None = None, + model: str = "", + effort: str = EFFORT_DEFAULT, ) -> tuple[str | None, str]: """P9: put a question to the agent CLI and return its answer. @@ -1067,7 +1149,8 @@ def ask( ) out, reason, chosen = _invoke_any( - cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped + cli, prompt, timeout=timeout, cwd=cwd, cancel=cancel, skipped=skipped, + model=model, effort=effort, ) if out is None: return None, reason diff --git a/flow/session.py b/flow/session.py index ea25246..6c3812f 100644 --- a/flow/session.py +++ b/flow/session.py @@ -52,6 +52,7 @@ from .profile import path_key from .refine import TIMEOUT_SEC as REFINE_TIMEOUT_SEC from .refine import MAX_CHARS as REFINE_MAX_CHARS +from .refine import EFFORT_DEFAULT, EFFORTS from .refine import app_note, ask, available, refine, tail_sent from .thread import ASK_CONTEXT_CHARS, Thread @@ -859,6 +860,12 @@ def __init__( #: launch. `getattr` because a profile is optional and the fakes predate the #: field. self.auto_ask = bool(getattr(profile, "auto_ask", True)) + #: Which model to ask the agent CLI for, and how hard to make it think. Taken + #: from the profile so a choice made once survives a restart, and defaulted to + #: the CLI's own model and the cheapest effort — see `refine.EFFORT_DEFAULT`. + self.cli_model: str = str(getattr(profile, "cli_model", "") or "") + self.cli_effort: str = str(getattr(profile, "cli_effort", "") + or EFFORT_DEFAULT) #: When the draft last stopped changing. None means nothing is pending. self._settled_at: float | None = None #: P8. What Flow has measured and learned about this person, on this machine. @@ -2926,6 +2933,40 @@ def set_cli(self, cli) -> None: self._emit("note", f"agent CLI: {cli.name}" if cli is not None else "agent CLI: automatic, in preference order") + def set_cli_model(self, model: str) -> None: + """Which model the agent CLI should use, or "" for whatever it defaults to. + + Applies to whichever CLI answers, including a fallback — `refine.tuned` drops it + for any CLI not measured to take a `--model`, so a name set for one is simply + ignored by another rather than breaking it. + + Remembered as well as applied. There is no way to type a model name into the + settings menu — Flow has no text field anywhere and the settings docstring + refuses to grow a dialog — so the list of names somebody has used is the menu, + and it is built from what has been set here. + """ + model = model.strip() + self.cli_model = model + if self.profile is not None: + self.profile.cli_model = model + if model and model not in self.profile.cli_models: + self.profile.cli_models = (*self.profile.cli_models, model) + self.profile.save() + self._emit("note", f"model: {model}" if model else "model: the CLI's own default") + + def set_cli_effort(self, effort: str) -> None: + """How hard the CLI should think, lowest by default. + + These calls are a rewrite rather than a reasoning problem, and the user is + watching a spinner while they run — see `refine.EFFORT_DEFAULT`. Anyone who + wants deliberation from their own model can have it, per level, from here. + """ + self.cli_effort = effort + if self.profile is not None: + self.profile.cli_effort = effort + self.profile.save() + self._emit("note", f"effort: {effort}") + def toggle_auto_ask(self) -> bool: self.auto_ask = not self.auto_ask # Saved now rather than at the next Send, for the reason `set_voice` gives: this @@ -3024,6 +3065,7 @@ def work() -> None: before, instruction, cwd=self._refine_cwd, polish=polish, context=context, cancel=self._cancel, cli=self._cli, timeout=self._cli_timeout, + model=self.cli_model, effort=self.cli_effort, skipped=passed_over, app=app, ) with self._refine_lock: @@ -3306,6 +3348,7 @@ def work() -> None: result = ask(framed, cwd=self._refine_cwd, context=context, cancel=self._cancel, artifact=artifact, cli=self._cli, timeout=self._cli_timeout, + model=self.cli_model, effort=self.cli_effort, skipped=passed_over) with self._ask_lock: # Written after `ask` returns and read under this lock, which is what diff --git a/flow/ui.py b/flow/ui.py index 990d6ff..0fa964a 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -42,7 +42,7 @@ ) from .notes import Notes from .profile import path_key, resolve_workspace -from .refine import available +from .refine import EFFORT_DEFAULT, EFFORTS, available from .session import CONVERSE, DICTATE, Session, State from .stats import today_note from .thread import MAX_TURNS as THREAD_MAX_TURNS @@ -2254,6 +2254,57 @@ def choose(name: str) -> None: ) parent.add_cascade(label=f"Chord ({chord.describe()})", menu=sub) + def _effort_menu(self, parent: tk.Menu) -> None: + """How hard the agent CLI may think, where it offers the choice. + + Lowest by default, and that is a judgement rather than a saving: these calls are + a *rewrite* — take what was dictated and make it read like a written prompt — and + effort buys deliberation the task has no use for, paid for in the one currency + that counts here, which is the user watching a spinner between finishing a + sentence and having their words. + + Offered anyway, per level, because "make it think harder about my prompt" is a + reasonable thing to want from a model you know. + """ + current = getattr(self.session, "cli_effort", EFFORT_DEFAULT) + sub = _dark_menu(parent) + for level in EFFORTS: + sub.add_command( + label=level + (" (current)" if level == current else ""), + command=lambda v=level: self.session.set_cli_effort(v), + ) + parent.add_cascade(label=f"Effort ({current})", menu=sub) + + def _model_menu(self, parent: tk.Menu) -> None: + """Which model to ask the CLI for, from the names that have been used before. + + **The menu is a list of what somebody has already typed, and cannot be anything + else.** No CLI will enumerate its models — `codex exec --help` says `-m, --model + ` and stops — so the names cannot be discovered, and Flow has no text + field anywhere to type one into. Settings is a menu, not a dialog, and the + docstring above refuses a page for exactly this reason. + + So `--cli-model` is how a name arrives, once, and it is remembered; from then on + it is a click. The menu hides itself entirely until there is a second thing to + choose between, the same rule the CLI picker follows. + """ + known = tuple(getattr(getattr(self.session, "profile", None), "cli_models", ())) + current = getattr(self.session, "cli_model", "") + if not known: + return + sub = _dark_menu(parent) + + def choice(label: str, value: str) -> None: + sub.add_command( + label=label + (" (current)" if value == current else ""), + command=lambda v=value: self.session.set_cli_model(v), + ) + + choice("The CLI's own default", "") + for name in known: + choice(name, name) + parent.add_cascade(label=f"Model ({current or 'default'})", menu=sub) + def _settings_menu(self, parent: tk.Menu) -> None: """Everything somebody sets once, in one place they can find it twice. @@ -2291,6 +2342,8 @@ def choice(label: str, cli) -> None: for candidate in clis: choice(candidate.name, candidate) sub.add_cascade(label="Agent CLI", menu=picker) + self._effort_menu(sub) + self._model_menu(sub) self._workspace_menu(sub) if getattr(self.session, "speaker", None) is not None: sub.add_command( diff --git a/tests/test_refine.py b/tests/test_refine.py index f5d8398..6cd141b 100644 --- a/tests/test_refine.py +++ b/tests/test_refine.py @@ -1510,3 +1510,99 @@ def test_nothing_dictated_reaches_the_argv_of_either(self): def test_the_one_that_was_not_measured_still_carries_it_on_the_argv(self): self.assertFalse(refine_mod.named("kiro-cli").stdin_ok) + +class TestTuning(unittest.TestCase): + """`tuned`, which is how a model and an effort level reach the CLI. + + Both flags were read out of each CLI's own `--help` on a machine that has all three, + on 2026-08-31, the same discipline `verified` carries for the invocation shapes + themselves. codex prints `-m, --model ` and no effort flag at all; claude and + kiro-cli print `--model` and `--effort (low, medium, high, xhigh, max)`. + """ + + def cli(self, name: str) -> refine_mod.Cli: + found = refine_mod.named(name) + assert found is not None + return found + + def test_effort_is_asked_for_by_default_and_it_is_the_cheapest(self): + # These calls are a rewrite, not a reasoning problem, and the user is watching a + # spinner while they run. + argv = refine_mod.tuned(self.cli("kiro-cli")).argv + self.assertIn("--effort", argv) + self.assertEqual(argv[argv.index("--effort") + 1], "low") + self.assertEqual(refine_mod.EFFORT_DEFAULT, "low") + + def test_the_levels_are_the_ones_both_clis_print(self): + self.assertEqual(refine_mod.EFFORTS, ("low", "medium", "high", "xhigh", "max")) + + def test_the_model_goes_in_with_the_flag_that_cli_takes(self): + self.assertIn(("-m", "gpt-5"), self.pairs("codex", model="gpt-5")) + self.assertIn(("--model", "gpt-5"), self.pairs("claude", model="gpt-5")) + self.assertIn(("--model", "gpt-5"), self.pairs("kiro-cli", model="gpt-5")) + + def pairs(self, name, **kw): + argv = refine_mod.tuned(self.cli(name), **kw).argv + return list(zip(argv, argv[1:])) + + def test_codex_is_not_given_an_effort_flag_it_does_not_have(self): + # Its only route is `-c model_reasoning_effort=...`, a config key that does not + # appear in its help. Writing one down from memory is what `verified` forbids. + self.assertNotIn("--effort", refine_mod.tuned(self.cli("codex"), effort="max").argv) + + def test_the_flags_land_after_the_subcommand(self): + # `exec` and `chat` are subcommands; a flag before them belongs to a different + # parser and the call fails to start. + for name, sub in (("codex", "exec"), ("kiro-cli", "chat")): + with self.subTest(name): + argv = refine_mod.tuned(self.cli(name), model="m").argv + self.assertEqual(argv[1], sub) + + def test_codex_keeps_the_stdin_marker_last(self): + """The trap that decided `tune_at`. + + codex's argv finishes with `-`, the positional saying the prompt is on stdin. A + flag appended after it is read as its value, and the prompt is never sent. + """ + argv = refine_mod.tuned(self.cli("codex"), model="gpt-5").argv + self.assertEqual(argv[-1], "-") + + def test_nothing_is_invented_for_a_cli_that_takes_neither(self): + # A user who picks a model while an unverified CLI answers should get that CLI + # answering, not a crash and not a guessed flag. + plain = self.cli("gemini") + self.assertIs(refine_mod.tuned(plain, model="gpt-5", effort="max"), plain) + + def test_asking_for_nothing_returns_the_very_same_object(self): + codex = self.cli("codex") + self.assertIs(refine_mod.tuned(codex, effort="default"), codex) + + def test_a_tuned_cli_is_still_that_cli(self): + # `_clean`'s per-CLI stripping, the pill's marker and the trace all key off the + # name. A tuned copy is the same CLI with a flag on it, not another one. + kiro = self.cli("kiro-cli") + copy = refine_mod.tuned(kiro, model="gpt-5") + self.assertEqual(copy.name, kiro.name) + self.assertEqual(copy.marker, kiro.marker) + self.assertEqual(copy.timeout_sec, kiro.timeout_sec) + self.assertEqual(copy.stdin_ok, kiro.stdin_ok) + self.assertEqual(copy.argv[0], kiro.argv[0]) + + def test_a_fallback_is_asked_for_the_same_model(self): + """A walk that reverted to the CLI's own defaults the moment the first candidate + failed would be slowest exactly when the user is already waiting longest.""" + seen = [] + + def record(cli, prompt, **kw): + seen.append(cli.argv) + return (None, "nope") if len(seen) == 1 else ("done", "") + + clis = [self.cli("claude"), self.cli("kiro-cli")] + with mock.patch.object(refine_mod, "available", return_value=clis), \ + mock.patch.object(refine_mod, "_invoke", side_effect=record): + refine_mod._invoke_any(None, "hi", timeout=1.0, model="gpt-5", effort="max") + self.assertEqual(len(seen), 2) + for argv in seen: + self.assertIn("gpt-5", argv) + self.assertIn("max", argv) + From d24bff7f294fbd80586d4eee2cc283833cd76017 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 08:36:22 -0500 Subject: [PATCH 24/41] Make Send actually send on a Mac, instead of handing back a clipboard The owner's verdict after every window fault was fixed: it seems on mac there is no actual send it copies in clipboard that is making this no useful on mac. Which was right. The whole idea is to speak into the window you are already working in, and now press Cmd-V yourself is the step Flow exists to remove - so all of the window work led to a clipboard and stopped. inject_mac is inject.py's opposite number, satisfying the same two functions on_send is written against. No SendInput and no window handle exist here: the text goes to the pasteboard with pbcopy and System Events types Cmd-V into whatever is frontmost. It does not need to aim, because Flow's windows are built without the titled bit and never take focus - which is why the window work had to come first. lite stopped being the same question as can-Flow-paste, and the two are now separate: lite is hotkeys and window handles, and on_send is keyed off whether an injector was imported. A Mac is Lite in every other respect and pastes. osascript rather than CGEvent, deliberately: the compiled route is faster and is where this should end up, but it is also an entitlement question and a build step, and both would have to be right before anybody could try it once. This needs no dependency and asks the OS for the same permission. The permission is what will actually bite - Accessibility, granted to the terminal rather than to Flow - so it is detected by error number and answered with the exact path through System Settings, said at startup rather than at the first failed paste. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 28 ++++- flow/__main__.py | 39 ++++++- flow/inject_mac.py | 206 +++++++++++++++++++++++++++++++++++ flow/session.py | 11 +- flow/ui.py | 14 ++- tests/test_inject_mac.py | 228 +++++++++++++++++++++++++++++++++++++++ tests/test_lite.py | 22 +++- tests/test_main.py | 34 ++++-- tests/test_talk.py | 8 +- 9 files changed, 561 insertions(+), 29 deletions(-) create mode 100644 flow/inject_mac.py create mode 100644 tests/test_inject_mac.py diff --git a/docs/guide.md b/docs/guide.md index 8c5ef93..5dde6ac 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -269,7 +269,7 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--cli-effort LEVEL` | how hard the CLI may think: `low`, `medium`, `high`, `xhigh`, `max` (default `low`) | | `--cli-timeout SEC` | how long to wait for one CLI call (default 20) | | `--cwd PATH` | the project converse-mode questions are asked from; overrides the stored `workspace` ([P9](#converse-mode-p9)) | -| `--lite` | clipboard-out mode: Send copies the draft instead of pasting it, and no hotkeys are registered (automatic off Windows — see [Install](#install)) | +| `--lite` | no global hotkeys and no target-window tracking (automatic off Windows — see [Install](#install)). On Windows it also makes Send copy instead of paste; on a Mac Send still pastes, through System Events | | `--version` | print `flow X.Y.Z` and exit. The same number the startup block names, and the one Help shows at the bottom of the sheet | | `--check-update` | ask GitHub once whether a newer release exists, print one line, and exit. Manual only: nothing in Flow ever checks on its own, and the request carries no version, no identifier and no account — see [What leaves the machine](architecture.md#what-leaves-the-machine) | | `--stats` | print how much has been dictated — today and all time — and exit, without loading a model or opening the microphone. See [The numbers](#the-numbers). With `--no-profile` it reads nothing and says so | @@ -883,6 +883,32 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. +### On a Mac + +Send pastes there too, as of this version. It used to copy and stop, which made every +other Mac fix beside the point — the whole idea is to speak into the window you are +already in, and "now press Cmd-V yourself" is the step Flow exists to remove. + +The mechanism is different because the platform is. There is no `SendInput` and no window +handle: Flow puts the text on the pasteboard with `pbcopy` and asks **System Events** to +type Cmd-V into whatever app is frontmost. It does not need to aim, because Flow's own +windows are built without a title bar and never take focus, so the app you were working in +is still the frontmost one. + +**macOS will refuse until you allow it.** Synthesising keystrokes needs Accessibility, and +macOS grants that to the *responsible* process — the terminal you started Flow from, not +Flow and not Python. So: + +**System Settings → Privacy & Security → Accessibility**, and switch on your terminal. + +Looking for "Flow" in that list is a dead end. Until it is granted, Send says so in the +bubble and names the setting; the text is on the clipboard either way, so Cmd-V works in +the meantime. `--no-paste` puts the old copy-only behaviour back if you would rather Flow +did not synthesise keystrokes at all. + +`enter boom` works the same way — the Return is sent as a key code rather than a typed +character, in the same script as the paste so nothing can come forward between them. + ### Sending it without touching anything Two words press Send, so the last step of the loop does not need the mouse: diff --git a/flow/__main__.py b/flow/__main__.py index ab81fc4..45c3fb5 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -44,6 +44,19 @@ #: Said on every Lite launch, before anything else, in ASCII for the reason `say()` gives. #: It names the platform because the two ways into Lite are a flag and an OS, and someone #: who did not type `--lite` deserves to be told which one they got. +#: What a Mac is told instead, because on a Mac the sentence above is no longer true. +#: Send pastes there now — the one thing Lite was defined by. Everything else it says +#: still holds: no injection DLL, no global hotkeys, the pill is the gesture. +#: +#: Accessibility is named at startup rather than at the first failed paste, because it is +#: a thing to go and do once and finding out at the moment you needed it to work is the +#: worst time to be told. +MAC_LINE = ( + "Flow on {platform}: Send pastes into whatever app is in front, using System Events " + "- grant your terminal Accessibility in System Settings > Privacy & Security. " + "No global hotkeys: hold the pill to talk, let go to send." +) + LITE_LINE = ( "Flow Lite on {platform}: Send copies the draft and you paste it - no injection, " "no global hotkeys, nothing to grant but the microphone. " @@ -404,8 +417,13 @@ def main(argv: list[str] | None = None) -> int: # while the platform *was* the problem; it is not one any more (decisions.md, # "Flow Lite"). lite = args.lite or sys.platform != "win32" + #: Set when an injector is imported below. `None` means Send has nothing to paste + #: with and falls back to the clipboard, which is what `--no-paste` and every + #: platform without an injector get. + paste = take_warnings = None if lite: - say(LITE_LINE.format(platform=sys.platform)) + say((MAC_LINE if sys.platform == "darwin" and not args.no_paste + else LITE_LINE).format(platform=sys.platform)) if args.no_paste: # Accepted rather than refused: a launcher shared between two machines should # not fail on a flag that has simply run out of things to suppress. @@ -425,6 +443,12 @@ def main(argv: list[str] | None = None) -> int: ) from .profile import CHORD_DEFAULT from .inject import paste, take_warnings + elif sys.platform == "darwin" and not args.no_paste: + # Lite is about hotkeys and window handles, not about whether Flow can put the + # words where they are going. A Mac has no `SendInput` and no `hwnd`, and it does + # have `osascript` — so it gets a real send while staying Lite in every other + # respect. See `inject_mac` for why the permission is the interesting part. + from .inject_mac import paste, take_warnings from .ui import Pill, apply_panel_width, apply_place, panel_width from .asr import WhisperTranscriber @@ -651,12 +675,13 @@ def main(argv: list[str] | None = None) -> int: "(--no-auto-ask to press it yourself)") else: say("auto-ask: off - press Ask when you are ready") - elif lite: + elif paste is None: say("mode: DICTATE - Send copies the draft, and you paste it " "(--converse, or the right-click menu, to ask instead)") else: say("mode: DICTATE - Send pastes into the focused window " - "(--converse, or ctrl+alt+M, to ask instead)") + f"(--converse, or {'the right-click menu' if lite else 'ctrl+alt+M'}, " + "to ask instead)") # Before anything is drawn, and that is the whole contract: `apply_panel_width` # rebinds module globals that two window classes and the chrome functions read, so a @@ -751,8 +776,14 @@ def on_send(text: str, target: int | None = None, submit: bool = False) -> str: # exist, so the menu is sent to the real settings folder instead: the profile lives # there either way, and creating a template beside the source is nobody's idea of # settings. + # What the mode notes read, and the same fact `on_send` is keyed off. + session.pastes = paste is not None pill = Pill( - session, on_send=None if lite else on_send, hotkeys=hotkeys, arm=args.arm, + # Keyed off whether an injector was imported, not off `lite`. The two came + # apart the day a Mac got a paste path: it is Lite in every other sense and can + # still put the words in the other window. + session, on_send=on_send if paste is not None else None, + hotkeys=hotkeys, arm=args.arm, settings_path=DEFAULT_PATH if args.no_lexicon else lexicon.path, lite=lite, ) diff --git a/flow/inject_mac.py b/flow/inject_mac.py new file mode 100644 index 0000000..87c97d5 --- /dev/null +++ b/flow/inject_mac.py @@ -0,0 +1,206 @@ +"""Paste into the app you were looking at, on macOS. + +`inject.py` is Win32 to the bone — `SendInput`, `OpenClipboard`, window handles — and +none of it exists here. This is its opposite number, satisfying the same two-function +contract `__main__.on_send` is written against: `paste(text, submit=…) -> bool`, and +`take_warnings()` for the reasons behind a False. + +**Why it had to exist.** Off Windows Flow ran in Lite, where Send copies the draft to the +clipboard and stops. That is a fine fallback and a poor product: the whole point is to +speak into the window you are already working in, and "now press Cmd-V yourself" is the +step Flow was built to remove. Everything else about the Mac was made to work first — the +window stays up, takes clicks, sits above the Dock — and the thing it all leads to still +handed you a clipboard. + +**System Events rather than CGEvent, for now.** Posting a synthetic Cmd-V through +`CGEventPost` is faster and needs no `osascript` process, and it is where this should +eventually go — `native/flow_stt.swift` is already a compiled binary with a build step. +It is also an entitlement question and a second thing to be right about before anybody +could try this once. `osascript` is on every Mac, needs no dependency (R16 holds at +three) and asks the OS for the same permission the compiled route would. + +**The permission is the thing that will actually bite.** Synthesising keystrokes needs +Accessibility, and macOS grants it to the *responsible* process — the terminal Flow was +started from, not Python and not Flow. Denied, System Events fails with `-1719` or `1002` +and does nothing at all, which from the user's side is indistinguishable from the Send +button being broken. So that case is detected by its error number and answered with the +exact path through System Settings, rather than left as a silent no-op (invariant 4). +""" + +import subprocess +import threading +import time + +#: How long the target app gets to read the clipboard before Flow puts the old contents +#: back. `inject.py`'s reasoning applies unchanged: the keystroke is queued, not waited +#: on, so restoring immediately would hand back the old text before the new text had been +#: read. The number is its number, for the same reason. +RESTORE_DELAY_SEC = 8.0 + +#: How long any one `osascript` call may take. Generous for a process that normally +#: answers in well under a second, and finite so a wedged System Events cannot hang the +#: send — the property `refine.sane_timeout` exists to protect, applied here too. +SCRIPT_TIMEOUT_SEC = 10.0 + +#: What System Events says when the terminal has not been granted Accessibility. `-1719` +#: is "not allowed assistive access" and `1002` is the keystroke-specific refusal; both +#: appear on stderr with the number in them. Matched on the numbers rather than the +#: prose, which is localised. +DENIED_CODES = ("-1719", "1002", "-25211") + +#: Names a frontmost process can have when the thing in front is Flow itself. Flow +#: pasting into its own draft box is the one outcome that would destroy the text being +#: sent, and `ui._bare_window` is what should make it impossible — this is the belt to +#: that pair of braces. +FLOW_NAMES = ("python", "python3", "flow", "tk", "wish") + +#: Warnings from the last paste, drained by the UI. A module-level queue for `inject.py`'s +#: reason: `paste` already returns success, and a caller that ignores the warning must +#: still see it. Locked because the restore runs on its own thread. +_warnings: list[str] = [] +_lock = threading.Lock() + + +def _warn(line: str) -> None: + with _lock: + _warnings.append(line) + + +def take_warnings() -> list[str]: + with _lock: + out, _warnings[:] = list(_warnings), [] + return out + + +def _run(argv: list[str], stdin: str | None = None) -> tuple[bool, str]: + """One child process. Returns `(ok, output-or-reason)` and never raises. + + A send that dies on an OSError from a helper nobody has heard of is worse than one + that reports what it could not do, so everything below treats failure as a sentence + to show the user rather than an exception to propagate. + """ + try: + done = subprocess.run( + argv, input=stdin, capture_output=True, text=True, + timeout=SCRIPT_TIMEOUT_SEC, encoding="utf-8", errors="replace", + ) + except OSError as exc: + return False, f"{argv[0]} would not start: {exc}" + except subprocess.TimeoutExpired: + return False, f"{argv[0]} did not answer in {SCRIPT_TIMEOUT_SEC:.0f}s" + if done.returncode != 0: + return False, (done.stderr or done.stdout or "").strip() or f"{argv[0]} failed" + return True, done.stdout or "" + + +def denied(reason: str) -> bool: + return any(code in reason for code in DENIED_CODES) + + +def permission_note() -> str: + """The one message worth getting right, because it is the only one most people see. + + Names the terminal rather than Flow or Python, because that is what the permission is + actually attached to — macOS assigns it to the responsible process — and looking for + "Flow" in that list is a dead end. + """ + return ("not pasted: macOS has not granted permission to send keystrokes. Open " + "System Settings > Privacy & Security > Accessibility and switch on the " + "terminal you started Flow from, then try again. The text is on the " + "clipboard, so Cmd-V works in the meantime.") + + +def frontmost() -> str: + """The name of the app that will receive the paste, or "" if it cannot be asked.""" + ok, out = _run(["osascript", "-e", + 'tell application "System Events" to get name of first process ' + "whose frontmost is true"]) + return out.strip() if ok else "" + + +def get_clipboard_text() -> str | None: + ok, out = _run(["pbpaste"]) + return out if ok else None + + +def set_clipboard_text(text: str) -> bool: + """`pbcopy` rather than Tk's clipboard, and the difference matters. + + Tk owns its selection for as long as the interpreter lives and serves it on request, + so a draft copied that way disappears the moment Flow exits. `pbcopy` hands the text + to the pasteboard server, where it behaves like anything else you have ever copied. + """ + ok, reason = _run(["pbcopy"], stdin=text) + if not ok: + _warn(f"could not reach the clipboard: {reason}") + return ok + + +def _restore_later(previous: str | None) -> None: + """Put the old clipboard back, once the paste has had time to happen. + + A daemon thread, so quitting Flow between the paste and the restore does not hold the + process open. The cost of losing the restore is the draft staying on the clipboard, + which is the state Lite mode leaves it in anyway. + """ + if previous is None: + return + + def worker() -> None: + time.sleep(RESTORE_DELAY_SEC) + _run(["pbcopy"], stdin=previous) + + threading.Thread(target=worker, name="flow-clipboard-restore", daemon=True).start() + + +def paste( + text: str, + *, + hwnd: int | None = None, + restore_clipboard: bool = True, + submit: bool = False, +) -> bool: + """Put `text` on the clipboard and send Cmd-V to whatever is frontmost. + + `hwnd` exists for the signature `__main__.on_send` is written against and is ignored: + macOS has no window handle to aim at, and it does not need one. Flow's own windows are + built without the `titled` bit and never take focus (`ui._bare_window`), so the app + that was frontmost when you started speaking is still frontmost now — which is why the + window work had to come first. + + **The clipboard is written before the keystroke is attempted**, deliberately, and it + is the same decision `inject.py` made: if the keystroke is refused, the text is still + somewhere the user can reach with their own Cmd-V. + """ + if not text: + return False + + # Asked before anything is touched, so the refusal costs no clipboard. + front = frontmost() + if front.lower() in FLOW_NAMES: + _warn(f"not pasted: {front} had the focus, not the window you were aiming at") + return False + + previous = get_clipboard_text() if restore_clipboard else None + if not set_clipboard_text(text): + return False + + # One script rather than two calls: each `osascript` is a process launch, and the gap + # between a Cmd-V and a Return is exactly where another window could come forward and + # take the Return. + keys = ['keystroke "v" using command down'] + if submit: + # `key code 36` is Return. Spelled as a code because `keystroke return` sends the + # character, and an app that tells them apart gets a newline instead of a send. + # The delay lets the paste land first — without it both events arrive in the same + # turn of the target's run loop and the Return can win. + keys += ["delay 0.05", "key code 36"] + script = chr(10).join(['tell application "System Events"', *keys, "end tell"]) + ok, reason = _run(["osascript", "-e", script]) + if not ok: + _warn(permission_note() if denied(reason) else f"not pasted: {reason}") + return False + + if restore_clipboard: + _restore_later(previous) + return True diff --git a/flow/session.py b/flow/session.py index 6c3812f..c15ce81 100644 --- a/flow/session.py +++ b/flow/session.py @@ -758,6 +758,11 @@ def __init__( #: the same brain — so this is read in exactly one place, the note that would #: otherwise name a focused window Lite cannot see. self.lite = lite + #: Whether Send puts the words in the other window or on the clipboard. Set by + #: `__main__` from whether it imported an injector, because `lite` stopped being + #: the same question the day a Mac got a paste path: it is Lite there — no global + #: hotkeys, no window handles — and it pastes. + self.pastes = not lite self.mic = mic or Mic(device=device) self.gate = SpeechGate() self.worker = DecodeWorker(self.asr) @@ -3224,9 +3229,9 @@ def toggle_mode(self) -> str: "converse mode - no agent CLI on PATH, so Ask has nothing to send") self._first_converse_notice() else: - self._emit("note", "dictate mode - Send copies the draft, and you paste it" - if self.lite - else "dictate mode - Send pastes into the focused window") + self._emit("note", "dictate mode - Send pastes into the focused window" + if self.pastes + else "dictate mode - Send copies the draft, and you paste it") return self.mode def send(self) -> str: diff --git a/flow/ui.py b/flow/ui.py index 0fa964a..68b72b4 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -2803,9 +2803,7 @@ def _send(self, submit: bool = False) -> None: self._ptt_wait = None text = self.session.send() problem = "" - if text and self.lite: - problem = self._copy(text) - elif text and self.on_send: + if text and self.on_send: # The window is chosen here, on the UI thread, from what was polled before # the click — not inside `paste()` after it. The handler reports back what # went wrong rather than printing it somewhere nobody is looking. @@ -2814,6 +2812,14 @@ def _send(self, submit: bool = False) -> None: # handler predating this — `send_check.py`'s fixture is one — still works. extra = {"submit": True} if submit else {} problem = self.on_send(text, self.paste_target, **extra) or "" + elif text and self.lite: + # The fallback, not the Lite behaviour. A handler is offered wherever Flow + # can actually put the words in the other window — Win32 injection, or + # System Events on a Mac — and the copy is what is left when it cannot. + # These two used to be the other way round, so a Mac that had grown a real + # paste path would still have copied: `lite` is about hotkeys and window + # handles, and it was standing in for "cannot send", which it is not. + problem = self._copy(text) if getattr(self.session, "mode", DICTATE) != DICTATE: # Converse: send() returns "" and the answer is still coming, so the bubble # stays up to render it and there is nothing to linger over. @@ -2826,7 +2832,7 @@ def _send(self, submit: bool = False) -> None: self.bubble.show_sent(text, problem) elif text: self.bubble.show_sent(text) - if self.lite: + if self.lite and self.on_send is None: # After the card, not instead of it: the words are the important half and # the note is what tells somebody the last step is theirs. self.bubble.note(COPIED_ENTER if submit else COPIED) diff --git a/tests/test_inject_mac.py b/tests/test_inject_mac.py new file mode 100644 index 0000000..d6f6d48 --- /dev/null +++ b/tests/test_inject_mac.py @@ -0,0 +1,228 @@ +"""`flow.inject_mac`: the paste path that made Flow useful on a Mac. + +Off Windows Flow ran in Lite, where Send copies the draft and stops — and the owner's +verdict on that was the reason this module exists: "it seems on mac there is no actual +send it copies in clipboard that is making this no useful on mac". Every window fault had +been fixed by then and the thing they all led to still handed you a clipboard. + +No `osascript` runs here. What is asserted is the shape of what would be run, the order +things happen in, and what the user is told when the OS says no — which is the part that +cannot be tested on the machine this was written on and is the part most likely to be +wrong. +""" + +import subprocess +import unittest +from unittest import mock + +import flow.inject_mac as inject_mac + + +class Ran: + """Records the child processes `paste` would start, and answers for them.""" + + def __init__(self, front="TextEdit", clipboard="old text", fail=None, reason=""): + self.calls: list[tuple[list[str], str | None]] = [] + self.front, self.clipboard = front, clipboard + self.fail, self.reason = fail, reason + + def __call__(self, argv, stdin=None): + self.calls.append((argv, stdin)) + tool = argv[0] + if self.fail is not None and self.fail in " ".join(argv): + return False, self.reason + if tool == "pbpaste": + return True, self.clipboard + if tool == "osascript" and "frontmost" in argv[-1]: + return True, self.front + return True, "" + + @property + def tools(self) -> list[str]: + return [argv[0] for argv, _stdin in self.calls] + + def script(self) -> str: + """The keystroke script, which is the last osascript that is not the query.""" + for argv, _stdin in reversed(self.calls): + if argv[0] == "osascript" and "frontmost" not in argv[-1]: + return argv[-1] + return "" + + def copied(self) -> list[str]: + return [stdin for argv, stdin in self.calls if argv[0] == "pbcopy"] + + +class Paste(unittest.TestCase): + def setUp(self): + inject_mac.take_warnings() + self.addCleanup(inject_mac.take_warnings) + + def run_paste(self, ran=None, **kw): + ran = ran or Ran() + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later"): + ok = inject_mac.paste("hello", **kw) + return ok, ran + + def test_it_copies_then_sends_command_v(self): + ok, ran = self.run_paste() + self.assertTrue(ok) + self.assertEqual(ran.copied(), ["hello"]) + self.assertIn('keystroke "v" using command down', ran.script()) + + def test_the_clipboard_is_written_before_the_keystroke_is_attempted(self): + """The order is the decision, and `inject.py` made the same one. + + A keystroke can be refused; a clipboard write mostly cannot. Doing the fragile + half second means a refusal still leaves the words somewhere the user can reach + with their own Cmd-V, which is exactly what the permission note promises them. + """ + _ok, ran = self.run_paste() + # Against the keystroke script specifically, not against the first `osascript`: + # the frontmost query is one too, and it runs first on purpose so a refusal + # costs no clipboard at all. + copied = next(i for i, t in enumerate(ran.tools) if t == "pbcopy") + typed = next(i for i, (argv, _s) in enumerate(ran.calls) + if argv[0] == "osascript" and "frontmost" not in argv[-1]) + self.assertLess(copied, typed) + + def test_submit_presses_return_by_key_code_and_not_as_a_character(self): + # `keystroke return` sends the character, and an app that tells them apart gets a + # newline in the box instead of a send. + _ok, ran = self.run_paste(submit=True) + self.assertIn("key code 36", ran.script()) + self.assertNotIn("keystroke return", ran.script()) + + def test_the_paste_and_the_return_travel_in_one_script(self): + # Each `osascript` is a process launch, and the gap between them is where another + # window could come forward and take the Return. + _ok, ran = self.run_paste(submit=True) + self.assertEqual(len([a for a, _s in ran.calls if a[0] == "osascript"]), 2) + self.assertIn('keystroke "v"', ran.script()) + self.assertIn("key code 36", ran.script()) + + def test_no_submit_presses_nothing(self): + _ok, ran = self.run_paste() + self.assertNotIn("key code", ran.script()) + + def test_it_refuses_to_paste_into_flow_itself(self): + # The one outcome that would destroy the text being sent. `ui._bare_window` is + # what should make it impossible; this is the belt to those braces. + ok, ran = self.run_paste(Ran(front="Python")) + self.assertFalse(ok) + self.assertEqual(ran.copied(), []) + self.assertIn("had the focus", inject_mac.take_warnings()[0]) + + def test_a_refused_keystroke_names_the_permission_and_the_terminal(self): + """The only message most people will ever see from this module. + + Accessibility is granted to the *responsible* process — the terminal — so a note + naming Flow or Python sends the reader to a list Flow is not in. + """ + ok, _ran = self.run_paste(Ran(fail="System Events", reason="(-1719)")) + self.assertFalse(ok) + note = inject_mac.take_warnings()[0] + self.assertIn("Accessibility", note) + self.assertIn("terminal", note) + self.assertIn("Cmd-V", note) + + def test_every_refusal_code_is_recognised(self): + for code in inject_mac.DENIED_CODES: + with self.subTest(code=code): + self.assertTrue(inject_mac.denied(f"System Events got an error ({code})")) + self.assertFalse(inject_mac.denied("some other failure")) + + def test_a_failure_that_is_not_the_permission_says_what_it_was(self): + # Invariant 4: a send that did nothing must never do it quietly. + ok, _ran = self.run_paste(Ran(fail="System Events", reason="osascript exploded")) + self.assertFalse(ok) + self.assertIn("osascript exploded", inject_mac.take_warnings()[0]) + + def test_a_clipboard_that_refuses_stops_before_the_keystroke(self): + # Sending Cmd-V after a failed copy pastes whatever was there before, which is + # somebody else's text going into the window they were working in. + ok, ran = self.run_paste(Ran(fail="pbcopy", reason="no pasteboard")) + self.assertFalse(ok) + self.assertEqual(ran.script(), "") + self.assertIn("clipboard", inject_mac.take_warnings()[0]) + + def test_empty_text_does_nothing_at_all(self): + with mock.patch.object(inject_mac, "_run") as ran: + self.assertFalse(inject_mac.paste("")) + ran.assert_not_called() + + def test_the_old_clipboard_is_read_and_scheduled_to_go_back(self): + ran = Ran(clipboard="something the user had") + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later") as later: + inject_mac.paste("hello") + later.assert_called_once_with("something the user had") + + def test_restore_can_be_turned_off_and_then_nothing_is_read(self): + ran = Ran() + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later") as later: + inject_mac.paste("hello", restore_clipboard=False) + self.assertNotIn("pbpaste", ran.tools) + later.assert_not_called() + + def test_hwnd_is_accepted_and_ignored(self): + # It exists for the signature `__main__.on_send` is written against. macOS has no + # window handle to aim at and does not need one: Flow's windows never take focus. + ok, _ran = self.run_paste(hwnd=0x22) + self.assertTrue(ok) + + +class Running(unittest.TestCase): + """`_run` itself, which must turn every way a child can fail into a sentence.""" + + def test_a_missing_tool_is_a_reason_and_not_an_exception(self): + with mock.patch.object(subprocess, "run", side_effect=OSError("no such file")): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertIn("osascript", why) + + def test_a_wedged_child_is_given_up_on(self): + # Finite by construction: a hung System Events must not hang the send. + with mock.patch.object(subprocess, "run", + side_effect=subprocess.TimeoutExpired("osascript", 10)): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertIn("did not answer", why) + + def test_a_nonzero_exit_carries_the_stderr(self): + done = mock.Mock(returncode=1, stderr="it went wrong", stdout="") + with mock.patch.object(subprocess, "run", return_value=done): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertEqual(why, "it went wrong") + + def test_a_nonzero_exit_with_nothing_to_say_still_says_something(self): + done = mock.Mock(returncode=1, stderr="", stdout="") + with mock.patch.object(subprocess, "run", return_value=done): + ok, why = inject_mac._run(["osascript", "-e", "x"]) + self.assertFalse(ok) + self.assertTrue(why) + + +class Warnings(unittest.TestCase): + def test_taking_them_clears_them(self): + # `inject.py`'s rule: a warning read twice is a failure reported twice, and one + # never read is the silence invariant 4 forbids. + inject_mac.take_warnings() + inject_mac._warn("something") + self.assertEqual(inject_mac.take_warnings(), ["something"]) + self.assertEqual(inject_mac.take_warnings(), []) + + +class TheStartupLine(unittest.TestCase): + def test_it_is_ascii_like_every_other_startup_line(self): + # `__main__.say` documents why: a redirected stdout on a legacy console code page + # cannot encode an en-dash, so a line carrying one crashes instead of printing. + for line in (inject_mac.permission_note(),): + line.encode("ascii") + line.encode("cp437") + + +if __name__ == "__main__": # pragma: no cover + unittest.main() diff --git a/tests/test_lite.py b/tests/test_lite.py index ab3f9e3..ec3e1c5 100644 --- a/tests/test_lite.py +++ b/tests/test_lite.py @@ -184,7 +184,7 @@ class Pill: found this the hard way). The attribute has to exist. """ - def __init__(self, s: Session, lite: bool = True) -> None: + def __init__(self, s: Session, lite: bool = True, injector: bool = True) -> None: import flow.ui as ui self.copied: list[str] = [] @@ -193,7 +193,12 @@ def __init__(self, s: Session, lite: bool = True) -> None: self.pill = ui.Pill.__new__(ui.Pill) self.pill.session = s self.pill.lite = lite - self.pill.on_send = self._on_send + # `injector` is what `__main__` decides by importing a paste module or not, and + # it is no longer the same question as `lite`. A Mac is Lite — no global hotkeys, + # no window handles — and still pastes, through System Events. `injector=False` + # is the case with nothing to paste with: `--lite` on Windows, `--no-paste`, or a + # platform Flow has no injector for. + self.pill.on_send = self._on_send if injector else None self.pill.paste_target = 0x22 self.pill.bubble = mock.Mock() self.pill._flash = 0 @@ -223,9 +228,16 @@ def notes(self) -> str: class TestSendInLiteIsACopy(unittest.TestCase): + """Lite with nothing to paste with — `--lite` on Windows, or `--no-paste`. + + Lite used to mean this by definition. It does not any more: a Mac is Lite in every + other respect and pastes through System Events, so the copy is now the fallback for + *no injector* rather than the behaviour of a mode. See `TestLiteWithAnInjector`. + """ + def setUp(self): self.s = session() - self.p = Pill(self.s) + self.p = Pill(self.s, injector=False) self.s.draft.set(DRAFT) def test_the_draft_goes_to_the_clipboard_and_nowhere_else(self): @@ -286,7 +298,7 @@ class TestTheEnterVariantCollapses(unittest.TestCase): def setUp(self): self.s = session() - self.p = Pill(self.s) + self.p = Pill(self.s, injector=False) self.s.draft.set(DRAFT) def test_it_copies_rather_than_refusing(self): @@ -305,7 +317,7 @@ def test_both_spoken_triggers_reach_it_through_the_router(self): ("enter boom", "Enter is yours to press")): with self.subTest(said=said): s = session() - p = Pill(s) + p = Pill(s, injector=False) s.draft.set(DRAFT) p.say(said) self.assertEqual(p.copied, [DRAFT]) diff --git a/tests/test_main.py b/tests/test_main.py index 7810b31..89e70c2 100644 --- a/tests/test_main.py +++ b/tests/test_main.py @@ -94,7 +94,11 @@ def test_the_entry_point_imports_at_all_without_win32(self): def test_a_mac_gets_lite_rather_than_a_refusal(self): code, out, pill, session = launch("darwin") self.assertEqual(code, 0) - self.assertIn("Flow Lite on darwin", out) + # Not the Lite banner any more: a Mac pastes, which is the one thing Lite was + # defined by. It is still Lite in every other respect and the `lite` kwargs below + # are what say so. + self.assertIn("Flow on darwin", out) + self.assertIn("Send pastes", out) self.assertNotIn("Windows-only", out) self.assertTrue(pill.call_args.kwargs["lite"]) self.assertTrue(session.call_args.kwargs["lite"]) @@ -103,11 +107,14 @@ def test_it_names_whichever_platform_it_found(self): _code, out, _pill, _session = launch("linux") self.assertIn("linux", out) - def test_no_hotkey_is_registered_and_no_paste_handler_is_built(self): - # The two halves of "no hands", asserted where they are decided rather than - # where they would be felt. `on_send` is the paste closure, and in Lite it - # closes over names that were never imported — so handing one over would be a - # handler that fails on its first call. + def test_no_hotkey_is_registered_but_a_paste_handler_is(self): + """"No hands" was two halves and is now one. + + The hotkeys stay unregistered: there is no `RegisterHotKey` here and the pill is + the gesture. The paste closure is built, because `inject_mac` gives it something + to close over — `osascript` rather than `SendInput`. Handing one over used to be + a handler that would fail on its first call; it is not any more. + """ _code, out, pill, _session = launch("darwin") # The registration report is `hotkey ` per line. Matched on the # line rather than on the word, because the Lite banner says "no global hotkeys" @@ -115,12 +122,19 @@ def test_no_hotkey_is_registered_and_no_paste_handler_is_built(self): self.assertEqual( [ln for ln in out.splitlines() if ln.startswith("hotkey")], []) self.assertIsNone(pill.call_args.kwargs["hotkeys"]) + self.assertIsNotNone(pill.call_args.kwargs["on_send"]) + + def test_no_paste_puts_the_clipboard_back(self): + # The one way to get the old behaviour on a Mac, and it has to keep working: + # somebody who does not want Flow synthesising keystrokes should not have to + # choose between that and using Flow. + _code, out, pill, _session = launch("darwin", ["--no-paste"]) self.assertIsNone(pill.call_args.kwargs["on_send"]) + self.assertIn("Send copies the draft", out) - def test_the_mode_line_does_not_name_a_window_lite_cannot_see(self): + def test_the_mode_line_says_it_pastes(self): _code, out, _pill, _session = launch("darwin") - self.assertIn("Send copies the draft", out) - self.assertNotIn("focused window", out) + self.assertIn("Send pastes into the focused window", out) def test_every_lite_startup_line_is_ascii_like_the_rest(self): # `say()` documents why: a redirected stdout on a legacy console code page @@ -495,7 +509,7 @@ def test_a_mac_launch_reads_the_field_without_reaching_for_win32(self): # that stops Flow from starting on the platform Lite exists for. code, out = self.launch(self.OVERRIDES, ["--lite"], platform="darwin") self.assertEqual(code, 0) - self.assertIn("Flow Lite on darwin", out) + self.assertIn("on darwin", out) @unittest.skipUnless(sys.platform == "win32", "Windows-only: ctypes.WinDLL") def test_no_hotkeys_reads_no_override_because_it_registers_none(self): diff --git a/tests/test_talk.py b/tests/test_talk.py index 4ae6c27..5fe0e89 100644 --- a/tests/test_talk.py +++ b/tests/test_talk.py @@ -94,7 +94,7 @@ class Harness: under test read is set here on purpose. """ - def __init__(self, *, lite: bool = False) -> None: + def __init__(self, *, lite: bool = False, injector: bool = False) -> None: self.mic = FakeMic() # `Pill.converse` is a property reading `session.mode`, so the surface follows # the session here exactly as it does in the app — there is nothing to set. @@ -106,7 +106,11 @@ def __init__(self, *, lite: bool = False) -> None: p.session = self.session p.lite = lite - p.on_send = self._on_send + # An injector is what `__main__` decides by importing a paste module, and it is + # no longer the same question as `lite`: a Mac is Lite and pastes through System + # Events. The full body always has one; Lite has one only where a platform gives + # Flow something to paste with. + p.on_send = self._on_send if (injector or not lite) else None p.paste_target = 0x22 p.bubble = mock.Mock() p.card = mock.Mock() From 92faf19d572aaefcaa24eb2f8e731793d4110686 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 09:03:46 -0500 Subject: [PATCH 25/41] One shape for the panels, and one width for the stack: stop the redraw The owner's verdict after the paste landed: window being re-draw and adjusting it self ... multiple windows size is problem and it's same bad shape on mac + windows ... sore to eyes. Three separate motions, all of them removed. The pill was PILL_W while nothing was docked and the panel's width while something was, so it jumped 205 -> 420 the instant a draft appeared and back when it went. pill_w now answers the panel width unconditionally, read from the constant rather than through a window that may not exist yet. The bubble and the card sized themselves to their contents on every render. The shots show what that cost: a two-line partial made a 140 px panel and a three-line draft a 215 px one, so the top edge moved while you were still speaking. Both are PANEL_H now and the body elides into it, which is machinery that already existed for the old ceiling. The golden geometry table says it best - every row is the same 420x184 across a draft that differs by fifty times its length. Two arithmetic traps came with it, both found by looking at the shots rather than by reading: the bubble capped its body at BODY_MAX_H unless frozen, and the card sized its answer against the whole desktop, so both drew content for a window far taller than the one they had. The card clipped its own agent label doing it. _float_up keeps the fade and loses the 18 px rise. R14 wanted an appearance the eye could follow when the window also changed size on every render; with the shape fixed the movement was the only thing left moving. 184 px was chosen by looking. At 236 a two-line partial left a hole in the middle of the window, which reads worse than the resizing it replaced. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 114 +++++++++++++++++++++++++++++++++---------- tests/test_bubble.py | 62 ++++++++++++++++++----- tests/test_card.py | 13 +++-- tests/test_editor.py | 24 ++++++--- tests/test_pill.py | 31 ++++++++---- 5 files changed, 189 insertions(+), 55 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 68b72b4..950b637 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1079,6 +1079,26 @@ def _unload_fonts() -> None: #: on, which leaves the pill its own room underneath. BODY_MAX_H = 340 +#: **One height, for every panel, always.** The windows used to size themselves to their +#: contents on every render, and the owner's verdict on that was the reason this exists: +#: "window being re-draw and adjusting it self ... multiple windows size is problem ... +#: that cost more and sore to eyes". They were right, and the shots show it — a two-line +#: partial made a 140 px panel and a three-line draft a 215 px one, so the panel grew and +#: the pill docked under it moved *while you were still speaking*. Nothing that changes +#: shape thirty times a second can be read. +#: +#: 184 px, and the number was chosen by looking rather than by arithmetic. A fixed panel +#: trades a void against elision, and both cost: at 236 px a two-line partial left a hole +#: in the middle of the window, which reads worse than the resizing it replaced. 184 is +#: the four body lines a dictated sentence actually comes to in `scripts/shots.py`, plus +#: the note line and the chip row — so the common case fills it and only a long draft +#: elides, which is machinery that already existed for the window's old ceiling and +#: needed no changes to serve a fixed one. +#: +#: Clamped to the work area by its callers, which is the one thing that may still shrink +#: it: a panel taller than the desktop is not a shape, it is a bug. +PANEL_H = 184 + #: The live partial's own ceiling, and it needs one for the same reason the draft does: #: it is wrapped to the full body column, so it is a multi-line block whose length nobody #: chose. Five lines at the 14 px `FONT_NOTE` measures. @@ -3569,8 +3589,17 @@ def pill_w(self) -> int: empty draft shows nothing), and an idle pill must not claim a panel's width it is not actually sitting under. """ - front = self.front - return front.width if getattr(front, "_visible", False) else PILL_W + # The panel width, unconditionally, and that is the point. This used to answer + # `PILL_W` while nothing was docked, so the pill jumped 205 -> 420 the moment a + # draft appeared and back again when it went — the most visible motion on the + # screen, on every single utterance. One width, whatever is happening. + # + # Read from the constant rather than through `self.front`, which is what it did + # while the answer depended on which panel was up. It does not any more: the + # bubble and the card are the same width by construction (`apply_panel_width` + # sets both), and reaching through a window meant this could be asked before + # there was one to ask. + return BUBBLE_W def _sync_dock(self) -> None: """Resize and reposition for whichever panel is up, right edge held fixed. @@ -4263,6 +4292,15 @@ def work_h(self) -> int: _left, top, _right, bottom = self.pill.work return bottom - top - 2 * EDGE_AIR + def panel_h(self) -> int: + """The height this window is, always. See `PANEL_H`. + + A method on both panels rather than the bare constant, because the desktop is + still allowed to be smaller than the shape we would like: a panel taller than the + display it is on is not a consistent shape, it is a window off the screen. + """ + return min(PANEL_H, self.work_h()) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" @@ -4400,7 +4438,11 @@ def _render(self) -> None: # which is arithmetic rather than a constant — the same bargain `Bubble._render` # strikes, and the reason a 12 000-character artifact cannot size this window # past the bottom of the display. - spare = (self.work_h() - PAD - HELP_FOOT_BAND - q_h - CARD_GAP + # Against the panel, not against the desktop. This asked `work_h()` while the + # card was free to grow to it; with the card a fixed shape that let the answer be + # sized for a 672 px window and drawn into a 184 px one, and the top of the card + # — the "agent" label — was cut off by it. + spare = (self.panel_h() - PAD - HELP_FOOT_BAND - q_h - CARD_GAP - BODY_ELIDED_H - (note_h + 4 if self._note else 0)) shown, more, a_h = "", 0, 0 if self._answer: @@ -4416,10 +4458,10 @@ def _render(self) -> None: history_h = sum(h + CARD_GAP for h in self._heights) # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - self._h = min( - max(CARD_MIN_H, PAD + history_h + self._pinned_h + HELP_FOOT_BAND), - self.work_h(), - ) + # `history_h` and `_pinned_h` still decide what is drawn and what scrolls; + # the window is the same size either way. A card that grew with the + # conversation moved the pill under it on every answer. + self._h = self.panel_h() c.configure(width=CARD_W, height=self._h) self.reposition() @@ -4784,6 +4826,15 @@ def work_h(self) -> int: _left, top, _right, bottom = self.pill.work return bottom - top - 2 * EDGE_AIR + def panel_h(self) -> int: + """The height this window is, always. See `PANEL_H`. + + A method on both panels rather than the bare constant, because the desktop is + still allowed to be smaller than the shape we would like: a panel taller than the + display it is on is not a consistent shape, it is a window off the screen. + """ + return min(PANEL_H, self.work_h()) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" @@ -4835,12 +4886,18 @@ def reposition(self, lift: int = 0) -> None: self.geometry(f"{BUBBLE_W}x{self._h}+{x}+{y}") def _float_up(self) -> None: - """R14: rise into place rather than appearing, so the eye follows it. - - Generation-guarded. Each run schedules eight `after` callbacks that each move - the window, so two overlapping runs fight over the position and the bubble - visibly jitters between two places — which is what a fast show/hide/show cycle - produces. + """Fade into place. It used to rise into place, and the rise is gone. + + R14 asked for an appearance the eye could follow, and 18 px of travel over eight + frames was the answer for a window that also changed size on every render — one + more thing moving among several. With the panel a fixed shape now (`PANEL_H`), + the movement is the *only* thing moving, and a window that slides every time a + draft appears is exactly the "sore to eyes" the owner reported. The fade still + gives the eye its cue and costs no motion at all. + + Generation-guarded. Each run schedules eight `after` callbacks, so two + overlapping runs would fight over the alpha — which is what a fast + show/hide/show cycle produces. """ steps = 8 self._anim += 1 @@ -4852,7 +4909,7 @@ def step(i: int) -> None: t = i / steps ease = 1 - (1 - t) ** 3 self.attributes("-alpha", 0.96 * ease) - self.reposition(lift=int(18 * (1 - ease))) + self.reposition() if i < steps: self.after(16, step, i + 1) @@ -5190,16 +5247,20 @@ def _render(self) -> None: # `BODY_ELIDED_H` is counted in unconditionally here: a capped body always has # something above it to report, and guessing the other way is how a line lands # on a control. - body_cap = BODY_MAX_H - if self._frozen(): - around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) - if partial_h: - around += partial_h + PARTIAL_GAP - if self._sent: - around += 16 - if self._act is not None: - around += 20 - body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self._h - around)) + # Unconditional now, and it used to run only while `_frozen()`. That gate was + # right when the window sized itself to the body: the room left in it was a hard + # number only while something was stopping it from growing. The window is a fixed + # shape now (`PANEL_H`), so the room left in it is *always* a hard number, and a + # body still asking for `BODY_MAX_H` would draw 340 px of text through the note + # and the chip row of a 184 px panel. + around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + if partial_h: + around += partial_h + PARTIAL_GAP + if self._sent: + around += 16 + if self._act is not None: + around += 20 + body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self._h - around)) shown, earlier, text_h = self._body_slot(body, body_cap) if not body: # `_body_slot` probes `shown or " "` so `bbox` always has something to answer @@ -5242,7 +5303,10 @@ def _render(self) -> None: # character artifact to 4 179 px on a 672 px desktop. # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - self._h = min(max(96, text_h + extra + 74), self.work_h()) + # `text_h` and `extra` still decide what is *drawn*; they no longer decide + # how big the window is. That is the whole change: the body elides into a + # fixed panel instead of the panel growing to fit the body. + self._h = self.panel_h() c.configure(width=BUBBLE_W, height=self._h) self.reposition() diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 60942f9..740b106 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -97,7 +97,17 @@ def test_the_body_handed_to_the_canvas_is_bounded(self): def test_a_draft_that_fits_is_drawn_whole(self): # The other half, and the one that makes this a window rather than a truncation: # nothing changes for the drafts people actually dictate. - b = bubble(draft(400)) + # + # 400 characters until the panel became a fixed shape (`PANEL_H`). "Fits" is a + # smaller number now — the window no longer grows to whatever the draft asks for, + # so what fits is what fits in 184 px, and more drafts window. The window still + # says so, which is what the class next door asserts. + # + # 40 rather than the ~200 the same panel holds on a real desktop: this fixture + # has no Plex face installed, Tk substitutes, and the substitute measures several + # times taller per line. The number is the fixture's, not the product's — the + # shots in `scripts/shots.py` show three full lines in the same 184 px. + b = bubble(draft(40)) b._render() self.assertEqual(drawn_body(b), b._text) @@ -178,16 +188,28 @@ def test_the_body_cannot_outgrow_a_window_frozen_under_the_hand(self): b.canvas.band(b._text[-40:])[1], b._h - ui.PAD - ui.CHIP_H, f"the body runs past the chip row of a {b._h} px window") - def test_and_takes_the_room_back_when_the_hand_leaves(self): + def test_and_there_is_no_room_to_take_back_any_more(self): + """This asserted the window grew once the hand left. It cannot: `PANEL_H`. + + The freeze was built to stop the window resizing under a hand reaching for a + chip, and a fixed shape makes that unreachable rather than guarded — the stronger + version of the same guarantee. What still catches up is the *content*, which is + what the caller actually wanted to see. + """ b = bubble(draft(300)) b._render() b._pointer_in = True b._text = draft(30_000) b._render() - frozen_h = b._h + frozen_h, frozen_body = b._h, drawn_body(b) b._pointer_in = False b._render() - self.assertGreater(b._h, frozen_h) + self.assertEqual(b._h, frozen_h) + # The body does not move either, and that is not a weaker check than it looks: + # the draft is windowed to its *tail*, so a 300-character draft and a 30 000- + # character one lay out the same last lines. Freezing had one observable effect + # and it was the height. + self.assertEqual(drawn_body(b), frozen_body) def test_five_chips_at_once_stay_inside_the_bubble(self): # Draft held, `can_rescue` true, dictate mode: Refine, Continue, Edit, Was a @@ -220,7 +242,8 @@ def test_a_windowed_draft_says_how_much_is_above_it(self): self.assertRegex(self._elision(b), r"^… \d+ earlier lines$") def test_a_draft_that_fits_says_nothing(self): - b = bubble(draft(400)) + # See the sibling test for why 40 and not 400: a substituted font measures taller. + b = bubble(draft(40)) b._render() self.assertEqual(self._elision(b), "") @@ -461,13 +484,21 @@ def along_the_top() -> dict[str, tuple[int, int]]: #: count — while the 50 000-character one holds at 415, because that row is capped #: by `BODY_MAX_H` rather than by how the text wraps, and a cap does not move with #: the column it bounds. +#: Re-recorded when the panel became a fixed shape (`ui.PANEL_H`), and the new table says +#: the change out loud better than any prose could: **every row is the same 420x184**, at +#: three placements and across a draft that differs by fifty times its length. It used to +#: read 398 for a 1k draft and 415 for a 50k one — a window that changed size with the +#: text in it, which is the motion this table now proves is gone. +#: +#: Still a golden table and still doing its original job: any *further* change to +#: placement has to justify itself against these numbers. GEOMETRY_BEFORE = { - ("1k draft", "bottom-left"): "420x398+8+234", - ("1k draft", "bottom-right"): "420x398+852+234", - ("1k draft", "mid-left"): "420x398+8+8", - ("50k draft", "bottom-left"): "420x415+8+217", - ("50k draft", "bottom-right"): "420x415+852+217", - ("50k draft", "mid-left"): "420x415+8+8", + ("1k draft", "bottom-left"): "420x184+8+448", + ("1k draft", "bottom-right"): "420x184+852+448", + ("1k draft", "mid-left"): "420x184+8+152", + ("50k draft", "bottom-left"): "420x184+8+448", + ("50k draft", "bottom-right"): "420x184+852+448", + ("50k draft", "mid-left"): "420x184+8+152", } @@ -645,7 +676,14 @@ def test_leaving_catches_everything_up(self): b._render() frozen = b._h b._leave() - self.assertNotEqual(b._h, frozen, "the window never caught up") + # The window is a fixed shape now (`PANEL_H`), so "caught up" cannot mean "is a + # different size" any more. What it means is that the note held back while the + # hand was here is on the canvas once it has gone — which is the thing anybody + # cared about, and was only ever inferred from the height. + self.assertEqual(b._h, frozen) + self.assertTrue(any("microphone overflowed" in i["text"] + for i in b.canvas.items if "text" in i), + "the note never caught up") def test_a_countdown_does_not_resize_its_own_chip(self): # Chip width followed the label, so `Ask` -> `Ask 4s` -> `Ask` moved the hit diff --git a/tests/test_card.py b/tests/test_card.py index 4adb9f6..161f896 100644 --- a/tests/test_card.py +++ b/tests/test_card.py @@ -452,9 +452,16 @@ def test_the_count_grows_with_the_answer(self): for n in (6_000, 12_000)] self.assertLess(counts[0], counts[1]) - def test_the_answer_still_sizes_the_card(self): - self.assertGreater(self.answered(prose(4_000))._h, - self.answered(prose(200))._h) + def test_the_answer_no_longer_sizes_the_card(self): + """It did, and that was the complaint. The card is one shape now. + + A card that grew with its answer moved its own top edge on every reply, and the + owner's verdict on that family of motion was "sore to eyes". The answer still + decides what is *shown* — `more_line` above proves the head window still reports + what it left out — but not how big the window is. + """ + self.assertEqual(self.answered(prose(4_000))._h, + self.answered(prose(200))._h) class TestTheExitsCarryTheWholeAnswer(unittest.TestCase): diff --git a/tests/test_editor.py b/tests/test_editor.py index 19c137d..c5f9cd8 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -624,14 +624,22 @@ def test_the_error_clears_the_chip_row(self): f"the note runs to y={note_bottom} and the chips start at y={chips_top}", ) - def test_the_bubble_grew_to_make_room_rather_than_clipping(self): - # The other way to stop an overlap is to cut the text off, and for an error - # message that is the same defect wearing a different hat. + def test_the_room_comes_out_of_the_body_now_that_the_window_cannot_grow(self): + """It used to grow. The panel is a fixed shape now (`PANEL_H`), so it cannot. + + The property under test never was "the window gets taller" — it was "a long note + is neither clipped nor sitting on the chips", which the sibling test above + asserts directly. What changed is where the room comes from: the body gives it + up, because the body is the one thing on this window that can be windowed and + say so. The note itself is still drawn whole, which is the half that matters — + errors are the longest strings this shows and the ones it is least acceptable to + hide. + """ short = self._bubble("ok") short._render() long_ = self._bubble(self.ERROR) long_._render() - self.assertGreater(long_._h, short._h) + self.assertEqual(long_._h, short._h) drawn = next(i for i in long_.canvas.items if "WinError 2" in i["text"]) self.assertEqual(drawn["text"], self.ERROR, "the note must not be truncated") @@ -861,12 +869,16 @@ def test_the_partial_clears_the_chip_row(self): f"the partial runs to y={bottom} and the chips start at y={chips_top}", ) - def test_the_bubble_grew_rather_than_clipping(self): + def test_the_window_is_the_same_size_whatever_the_partial_is(self): + # It used to grow for a long partial, which meant the window changed shape while + # you were still speaking — the motion the fixed panel exists to end. The + # partial's own cap and the body's are what keep it off the note; the sibling + # test above asserts that directly. one = self._bubble("part key towel control") one._render() many = self._bubble(self.PARTIAL) many._render() - self.assertGreater(many._h, one._h) + self.assertEqual(many._h, one._h) def test_a_partial_past_the_cap_keeps_its_tail(self): # Bounded like the draft is, and windowed the same way round: this is the diff --git a/tests/test_pill.py b/tests/test_pill.py index 6c923b4..e19a860 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -1229,11 +1229,19 @@ def test_the_recovery_does_not_move_the_pill_a_second_time(self): self.assertEqual({c.args for c in p.geometry.call_args_list}, {(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608",)}) - def test_the_panel_going_away_takes_the_width_back(self): + def test_the_panel_going_away_leaves_the_width_alone(self): + """It used to take the width back, and taking it back was the thing to stop. + + The pill was `PILL_W` while nothing was docked and the panel's width while + something was, so it jumped 205 -> 420 the instant a draft appeared and back + again when it went — the most visible motion on the screen, on every utterance. + `Pill.pill_w` answers the panel's width unconditionally now, so a panel coming + and going costs no resize and no move at all. + """ p = docker(showing=False, docked_w=ui.BUBBLE_W, x=832) p._sync_dock() - self.assertEqual(p.x, 1047) - p.geometry.assert_called_once_with(f"{ui.PILL_W}x{ui.PILL_H}+1047+608") + self.assertEqual(p.x, 832) + p.geometry.assert_not_called() class TestTheDockRecentresWhenThereIsNoEdge(unittest.TestCase): @@ -1261,20 +1269,25 @@ def test_a_panel_appearing_recentres_on_the_new_width(self): p._sync_dock() self.assertEqual(p.x, self.centre_of(p, ui.BUBBLE_W)) - def test_the_panel_going_away_recentres_on_the_pill(self): + def test_the_panel_going_away_moves_nothing(self): + # There is no narrower width to recentre on any more: the pill is the panel's + # width whether a panel is up or not. p = docker(showing=False, docked_w=ui.BUBBLE_W, x=430) p._sync_dock() - self.assertEqual(p.x, self.centre_of(p, ui.PILL_W)) + self.assertEqual(p.x, self.centre_of(p, ui.BUBBLE_W)) - def test_the_stack_stays_centred_across_an_open_and_a_close(self): - # The round trip, because an off-by-one in either direction accumulates: a pill - # that came back a few pixels off would drift across a session. + def test_the_stack_stays_put_across_an_open_and_a_close(self): + # The round trip, which is now a round trip to nowhere. A pill upgrading from an + # old profile still gets centred on the panel width once, and then a panel + # opening and closing leaves it exactly where that put it. p = docker(x=538, docked_w=ui.PILL_W) p._sync_dock() + settled = p.x + self.assertEqual(settled, self.centre_of(p, ui.BUBBLE_W)) p.bubble = mock.Mock(width=ui.BUBBLE_W, _visible=False) p.window_geometry = mock.Mock(return_value=(ui.BUBBLE_W, p.x, 608)) p._sync_dock() - self.assertEqual(p.x, 538) + self.assertEqual(p.x, settled) def test_it_is_still_clamped_onto_the_screen(self): # The clamp survives the branch. A panel wider than the display would otherwise From 9d8daf2e8113bc8491495b5e1927706a5cebba28 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 10:29:16 -0500 Subject: [PATCH 26/41] Merge the pill and its panel into one window, and delete the dock The structure FluidVoice's BottomOverlayView has and Flow did not: text and the control row inside a single rounded shell. Flow drew two operating-system windows kept adjacent by hand, and the seam between them was visible in every screenshot. Bubble and ConversationCard are tk.Frame now, not tk.Toplevel - bands placed at (0,0) inside the pill's window. Their drawing code is canvas-local and did not change; what went is the window around it. Only eleven window-only calls existed between them, which is why this was tractable at all. Deleted: _sync_dock's width negotiation, both repositions' above-or-below decision, _docked_above, park() on the panels, _float_up, and the ordering rule saying whichever ran first left the other with nothing to do. All of it existed to make two windows look like one. _sync_shell is what is left: a height, and a bottom edge that never moves. The foot is the anchor, and that is the whole of the controls staying where they are. A panel opening grows the window upward, so Send, the meter and the chip row stay at the pixel they were at. Growing downward or centring the growth would move every control on the surface every time a draft appeared. Two defects found by looking rather than by reading. The row's seam overpaint cleared rows 0..4 while its inner ring is drawn at y=4, so a second hairline stood 3 px under the divider - invisible before, because a 1 px window gap was already drawing a line there. Measured out of the shot: two RING rows at y=627 and y=642, now one. And band_h had to come off the pill: the panels only ever had to fit the work area while the pill fitted it separately, and sharing a window makes them one sum - a 200 px display put a 224 px shell 32 px past the bottom of it. 39 bubble tests and both dock classes asserted where one window went relative to another, which is not a question any more; they are replaced by TestTheShellIsOneWindow and TestTheBubbleIsABandInThePillsWindow, which assert the properties that survived. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 351 ++++++++++++++++++++----------------------- tests/test_bubble.py | 312 +++++--------------------------------- tests/test_card.py | 16 +- tests/test_editor.py | 6 + tests/test_pill.py | 192 ++++++++++------------- 5 files changed, 298 insertions(+), 579 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 950b637..7122448 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1576,7 +1576,12 @@ def _panel_chrome(c: tk.Canvas, w: int, h: int, radius, ring_color: str, # join is the one mark that makes two surfaces unmistakably two. c.create_line(4 + inner[0], 4, w - 4 - inner[1], 4, fill=RING_TOP, tags=tags) if seam == "top": - c.create_rectangle(0, 0, w, 4, fill=SHELL, outline="", tags=tags) + # Through 5, not 4. The inner ring is drawn *at* y=4, so a fill that stopped + # there left it standing — and a second hairline 3 px under the divider is + # exactly the "two surfaces that happen to touch" this is here to prevent. It + # went unseen while these were two windows, because a 1 px window gap was + # already drawing a darker line in the same place. + c.create_rectangle(0, 0, w, 5, fill=SHELL, outline="", tags=tags) elif seam == "bottom": c.create_rectangle(0, h - 4, w, h, fill=SHELL, outline="", tags=tags) c.create_line(0, h - 1, w, h - 1, fill=RING, tags=tags) @@ -1615,7 +1620,7 @@ class Pill(tk.Tk): #: ran `__init__` — and so never built a `bubble`/`card` to dock to — draws exactly #: the idle pill this default describes, rather than recursing through `front`. _docked_w = PILL_W - _docked_above = True + _shell_h = PILL_H #: Same reason again, for `_draw`'s motion state (§07): a bare fixture draws the #: resting frame these describe — not hovered, not mid-collapse, opacity untouched. _pointer_in = False @@ -1738,17 +1743,18 @@ def __init__( #: went away since the last frame — and, holding the right edge fixed, by how #: much the left edge has to move to match. Neither panel exists yet at this #: point in `__init__`, so this starts at the same idle width just drawn above. - self._docked_w = PILL_W - #: Which side of the pill a docked panel is actually on, so a squared corner - #: lands on the shared seam rather than on the free-standing side. Set by - #: `Bubble.reposition`/`ConversationCard.reposition`, the one place that - #: already decides above-vs-below; read back by `_draw`. - self._docked_above = True + self._docked_w = self.pill_w + #: How tall the one window is right now — the pill row, plus a panel band when a + #: panel is up. Compared in `_sync_shell` so a frame that changes nothing costs + #: no `geometry` call. + self._shell_h = PILL_H self.canvas = tk.Canvas( - self, width=PILL_W, height=PILL_H, bg=self.bg, highlightthickness=0 + self, width=BUBBLE_W, height=PILL_H, bg=self.bg, highlightthickness=0 ) - self.canvas.pack() + # `place`, not `pack`: this canvas is the *foot* of a window whose top edge moves + # when a panel opens, so it has to be positioned rather than filled. + self.canvas.place(x=0, y=0, width=BUBBLE_W, height=PILL_H) self.bubble = Bubble(self) #: P9's own surface (decisions.md 2026-08-03, "two surfaces, two jobs"). Built @@ -3119,6 +3125,19 @@ def _tick(self) -> None: if self._alive: self.after(30, self._tick) + def band_h(self) -> int: + """How tall the panel band is: `PANEL_H`, unless the desktop is smaller. + + The row's own height comes off the top of what is available, which the panels' + `panel_h` did not do while they were windows of their own — they only had to fit + the work area, and the pill fitted it separately. Sharing one window makes them + one sum, and a 200 px-tall display was enough to put a 224 px shell 32 px past + the bottom of it. + """ + _left, top, _right, bottom = self.work + room = (bottom - top) - 2 * EDGE_AIR - PILL_H + return max(0, min(PANEL_H, room)) + def _placed(self, w: int) -> tuple[int, int]: """Where a stack `w` wide belongs on the current monitor, per `PLACE`. @@ -3601,52 +3620,50 @@ def pill_w(self) -> int: # there was one to ask. return BUBBLE_W - def _sync_dock(self) -> None: - """Resize and reposition for whichever panel is up, right edge held fixed. - - Idempotent and cheap once nothing has changed, so it can run from two places - without either caring which ran first: this pill's own frame, *and* a panel's - `reposition`, which needs this pill's true — already docked — position before - it can put itself directly above or below it. Whichever runs first leaves the - other with nothing left to do. - - The right edge is what a bare pill has always anchored near (`right - PILL_W - - 28` at rest), so growing to dock keeps that edge still and moves the left edge - to meet it — the same edge a docked panel's own width now matches exactly. - - **The move is checked, not assumed** (2026-08-09). The width change used to be - the only trigger, so one `geometry` call carried the whole dock and there was - no second chance at it: `scripts/reel.py` caught the pill 420 px wide at its - *bare* x, hanging 215 px past the screen edge and visibly unjoined from the - panel above it, for five seconds at a stretch. The pill's own state was never - wrong — only ever (832, 420, 420) or (1047, 205, 205) — so the resize landed - and the move did not, and `w == self._docked_w` then answered "nothing to do" - on every frame after. Comparing against the window instead of against a - remembered width means the next frame fixes it, whatever dropped it. - - The position asked for is clamped into the work area, so a window manager has - no reason to refuse it; one that did would be re-asked every frame. + def _sync_shell(self) -> None: + """One window, sized for whatever band is up, with its bottom edge held still. + + **This was `_sync_dock`, and the dock is gone.** The pill and its panel were two + windows the app kept adjacent by hand: a width they had to agree on, an + above-or-below decision, a `_docked_above` flag so the pill knew which corners to + square off, and an ordering rule saying whichever ran first left the other with + nothing to do. `scripts/reel.py` once caught them 215 px apart for five seconds, + because a resize had landed and the matching move had not — a failure only + possible when two windows have to be moved in two calls the compositor is free to + show a frame apart. + + There is one window now, so there is nothing to keep in step. What is left is a + height: the pill row, plus the panel band when a panel is up. + + **The bottom edge is the anchor**, and that is the whole of "the controls do not + move". A panel opening grows the window *upward* — the chip row, the meter and + the Send button stay at the pixel they were at, because they are measured from a + foot that never moves. Growing downward, or centring the growth, would move every + control on the surface every time a draft appeared. + + Idempotent and cheap once nothing has changed, so it can run from the frame pump + and from a panel's `reposition` without either caring which got there first. + + The geometry is compared against the *window* rather than against a remembered + value, for the reason the dock learned the hard way: state that says the move + happened is not evidence that it did, and comparing against the window means the + next frame fixes whatever dropped it. """ + h = PILL_H + (self.band_h() if getattr(self.front, "_visible", False) else 0) w = self.pill_w - if w != self._docked_w: - left, _top, right, _bottom = self.work - if PLACE == "corner": - # Hold the right edge and let the left one move out to meet the panel. - # That is the whole dock in corner placement: the pill is anchored to - # the screen edge it sits against, and growing away from it is the only - # direction there is. - self.x = max(left, self.x + self._docked_w - w) - else: - # Centred placement has no anchored edge, so holding one is what makes - # the stack lurch: the pill is 205 px and the panel 420, and keeping the - # right edge put the pair 107 px left of centre the moment a draft - # appeared — visibly, on every utterance. Re-centre on the new width. - self.x = self._placed(w)[0] - self.x = max(left, min(self.x, right - w)) - self._docked_w = w - self.canvas.configure(width=w) + left, top, right, bottom = self.work + foot = self._placed(w)[1] + PILL_H # where the pill row's bottom edge belongs + x = max(left, min(self._placed(w)[0], right - w)) + y = max(top + EDGE_AIR, min(foot - h, bottom - h)) + if (self.x, self.y, self._shell_h) != (x, y, h): + self.x, self.y, self._shell_h = x, y, h + self.canvas.place(x=0, y=h - PILL_H, width=w, height=PILL_H) if self.window_geometry() != (w, self.x, self.y): - self.geometry(f"{w}x{PILL_H}+{self.x}+{self.y}") + self.geometry(f"{w}x{h}+{self.x}+{self.y}") + + #: Kept under the old name because every caller in the app and the suite says it, and + #: the two never meant different things — the dock *was* the shell, badly. + _sync_dock = _sync_shell def window_geometry(self) -> tuple[int, int, int]: """Where this window actually is, as (width, x, y). @@ -3711,16 +3728,21 @@ def _draw(self) -> None: accent = self.accent w = self._docked_w seam = None - if w == PILL_W: - radius = PILL_H // 2 # idle: the capsule this pill has always been + if self._shell_h == PILL_H: + radius = PILL_H // 2 # alone in the window: the capsule this has always been else: - # Docked: squared on the seam it shares with the panel, rounded on the - # free-standing side, at the panel's own 8 px — one shape language, not a - # capsule with a corner cut off. - radius = (0, 0, 8, 8) if self._docked_above else (8, 8, 0, 0) - # The pill is the lower surface when the panel is above it, so it is the one - # that draws nothing on the join — the panel's bottom carries the single line. - seam = "top" if self._docked_above else "bottom" + # Sharing the window with a panel band directly above. Squared on the join, + # rounded on the free-standing foot, at the panel's own 8 px — one shape + # language, not a capsule with a corner cut off. + # + # `_docked_above` used to decide this, because the panel was a window that + # could end up on either side of the pill when there was no room above. There + # is one window now and the band is always the top of it, so the answer is a + # constant and the flag that carried it is gone. + radius = (0, 0, 8, 8) + # The row is the lower surface, so it draws nothing on the join — the band's + # bottom carries the single line. + seam = "top" _panel_chrome(c, w, PILL_H, radius, self.ring_color, seam=seam) # Mic glyph: capsule + stand, drawn rather than fonted so there is no @@ -4079,7 +4101,7 @@ def _footer(self, drawn: int) -> None: fill=MUTED, font=("Segoe UI", 8)) -class ConversationCard(tk.Toplevel): +class ConversationCard(tk.Frame): """P9's surface: a question, the answer it produced, and the turns behind them. Converse mode used to share the draft bubble, and three outside users found every @@ -4122,7 +4144,11 @@ class ConversationCard(tk.Toplevel): def __init__(self, pill: Pill) -> None: super().__init__(pill) self.pill = pill - self.bg = _shell_window(self, pill.lite, 0.0) + # A `Frame` inside the pill's window, not a window of its own. Everything this + # class draws is canvas-local and did not change; what went is the *window* - + # its own shell, its own shadow, its own position, and all the arithmetic that + # kept it touching the pill. See `Pill._sync_shell`. + self.bg = pill.bg self.configure(bg=self.bg) self.canvas = tk.Canvas(self, bg=self.bg, highlightthickness=0) self.canvas.pack() @@ -4159,7 +4185,7 @@ def __init__(self, pill: Pill) -> None: self.canvas.bind("", self._drag) self.canvas.bind("", self._enter, add="+") self.canvas.bind("", self._leave, add="+") - self.withdraw() + self.place_forget() # -- content ----------------------------------------------------------- @@ -4227,11 +4253,18 @@ def show(self) -> None: def close(self) -> None: self._visible = False - park(self) + # Give the band back rather than parking a window offscreen. `park` existed + # because hiding a Toplevel on Windows cost a taskbar flicker and a restack; + # there is no window here to hide, only a `place` to undo, and the pill's shell + # shrinks to the row on the next `_sync_shell`. + self.place_forget() + self.pill._sync_shell() @property def showing(self) -> bool: - return bool(self.winfo_exists() and self.state() != "withdrawn") + # `state()` was a window's state, and this is no longer a window. A band is + # showing when it has a place in the one it lives in. + return bool(self.winfo_exists() and self.winfo_ismapped()) def _push(self, row: tuple[str, str]) -> None: self._history.append(row) @@ -4245,8 +4278,7 @@ def _push(self, row: tuple[str, str]) -> None: def _show(self) -> None: if not self._visible: self._visible = True - self.deiconify() - self.attributes("-alpha", 0.97) + self.pill._sync_shell() self._render() # -- scrolling --------------------------------------------------------- @@ -4299,7 +4331,9 @@ def panel_h(self) -> int: still allowed to be smaller than the shape we would like: a panel taller than the display it is on is not a consistent shape, it is a window off the screen. """ - return min(PANEL_H, self.work_h()) + # Asked of the pill, because the row shares this window and its height comes + # off the top of what the desktop has left. + return self.pill.band_h() @property def width(self) -> int: @@ -4307,26 +4341,20 @@ def width(self) -> int: return CARD_W def reposition(self) -> None: - """Item 44's anchor, with this window's width. Above whenever above fits. - - No gap now: this window docks to the pill rather than floating near it - (Phase 5, decisions.md 2026-08-09) — the two meet at one hairline seam - instead of the 10 px of air a shadow used to go in. `_sync_dock` runs first - so the pill's own position already reflects the width it is about to share, - and `_docked_above` is set here because this is the one place that already - decides which side the seam is actually on. + """Take the top band of the pill's window, or give it back. + + **This used to place a window.** It chose above-or-below against the work area, + anchored to the pill's right edge, set `_docked_above` so the pill knew which of + its corners to square off, and called `_sync_dock` first so the pill had already + settled into the width the two were about to share. All of that existed to make + two windows look like one, and none of it survives one window: the panel is the + band above the pill row, at x=0, always. """ - self.pill._sync_dock() - left, top, right, bottom = self.pill.work - x = self.pill.x + self.pill.pill_w - CARD_W - above = self.pill.y - self._h - below = self.pill.y + PILL_H - fits_below = above < top + EDGE_AIR and below + self._h <= bottom - EDGE_AIR - self.pill._docked_above = not fits_below - y = below if fits_below else above - x = max(left + EDGE_AIR, min(x, right - CARD_W - EDGE_AIR)) - y = max(top + EDGE_AIR, min(y, bottom - self._h - EDGE_AIR)) - self.geometry(f"{CARD_W}x{self._h}+{x}+{y}") + self.pill._sync_shell() + if self._visible: + self.place(x=0, y=0, width=CARD_W, height=self._h) + else: + self.place_forget() # -- holding still under the hand -------------------------------------- @@ -4468,10 +4496,11 @@ def _render(self) -> None: c.delete("body") # Squared on the seam it shares with the docked pill, rounded on the free # side — `reposition` is what decides above-vs-below, since it already has to. - above = getattr(self.pill, "_docked_above", True) - corners = (8, 8, 0, 0) if above else (0, 0, 8, 8) + # Always the top band of the one window now: rounded head, squared foot on the + # join it shares with the pill row below it. + corners = (8, 8, 0, 0) _panel_chrome(c, CARD_W, self._h, corners, self.ring_color, - seam="bottom" if above else "top") + seam="bottom") # -- the history, in what is left above the pinned block y, floor = PAD, PAD + self._view_h() @@ -4602,7 +4631,7 @@ def _new_conversation(self) -> None: self.pill.session.new_conversation() -class Bubble(tk.Toplevel): +class Bubble(tk.Frame): """The draft, floated above the pill (R14) with Refine / Continue / Send (R15).""" #: Copied from the pill at construction rather than read back off it, for the reason @@ -4627,7 +4656,11 @@ def __init__(self, pill: Pill) -> None: super().__init__(pill) self.pill = pill self.lite = pill.lite - self.bg = _shell_window(self, pill.lite, 0.0) + # A `Frame` inside the pill's window, not a window of its own. Everything this + # class draws is canvas-local and did not change; what went is the *window* - + # its own shell, its own shadow, its own position, and all the arithmetic that + # kept it touching the pill. See `Pill._sync_shell`. + self.bg = pill.bg self.configure(bg=self.bg) self.canvas = tk.Canvas(self, bg=self.bg, highlightthickness=0) self.canvas.pack() @@ -4674,7 +4707,7 @@ def __init__(self, pill: Pill) -> None: self.canvas.bind("", self._enter, add="+") self.canvas.bind("", self._leave, add="+") self.canvas.bind("", self._context_menu) - self.withdraw() + self.place_forget() # -- content ----------------------------------------------------------- @@ -4749,8 +4782,7 @@ def show_sent(self, text: str, problem: str = "") -> None: self._note = problem if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def show(self, text: str) -> None: @@ -4768,8 +4800,7 @@ def show(self, text: str) -> None: self._render() if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() def show_partial(self, text: str) -> None: # Partials are dimmed: they contain hallucinated fragments on mid-word @@ -4778,8 +4809,7 @@ def show_partial(self, text: str) -> None: self._for_activity = False if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def note(self, msg: str, undoable: bool = False) -> None: @@ -4796,8 +4826,7 @@ def surface(self, msg: str) -> None: self._for_activity = False if not self._visible: self._visible = True - self.deiconify() - self._float_up() + self.pill._sync_shell() self._render() def hide(self) -> None: @@ -4811,7 +4840,12 @@ def hide(self) -> None: self._text = self._partial = self._note = self._sent = "" self._note_undo = False self._for_activity = False - park(self) + # Give the band back rather than parking a window offscreen. `park` existed + # because hiding a Toplevel on Windows cost a taskbar flicker and a restack; + # there is no window here to hide, only a `place` to undo, and the pill's shell + # shrinks to the row on the next `_sync_shell`. + self.place_forget() + self.pill._sync_shell() # -- geometry ---------------------------------------------------------- @@ -4833,88 +4867,33 @@ def panel_h(self) -> int: still allowed to be smaller than the shape we would like: a panel taller than the display it is on is not a consistent shape, it is a window off the screen. """ - return min(PANEL_H, self.work_h()) + # Asked of the pill, because the row shares this window and its height comes + # off the top of what the desktop has left. + return self.pill.band_h() @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" return BUBBLE_W - def reposition(self, lift: int = 0) -> None: - """Anchor above the pill, clamped inside the work area. - - The clamp used to be `max(8, x)` alone, which pins the left edge and lets the - right edge run off the display — so on a screen whose coordinates the app had - got wrong, the bubble hung half outside it with its buttons unreachable. Both - edges are bounded now, and against the work area rather than the raw screen. - - It is only a *clamp*, and that was the gap: a window taller than the work area - cannot be placed inside it however carefully the position is computed, so the - reply path — the one item 37 deliberately did not touch — ran 795 px past the - bottom on an ordinary answer and 3 515 px on an artifact, at every pill corner. - The height is fitted in `_render` now (`work_h`), which is what makes the - arithmetic below a guarantee rather than a best effort. - - Fitting it exposed the next thing, at the desk: with the pill dragged to the top of - the work area there is no "above" left, so the bubble clamped to the top edge and - was drawn **over the pill it is anchored to**. Nothing clipped — that is item 42's - guarantee and it survives — but an anchor pointing at something it covers is not - an anchor. Below is the fallback, and only that. - """ - # No gap now: this window docks to the pill rather than floating near it - # (Phase 5, decisions.md 2026-08-09) — the two meet at one hairline seam - # instead of the 10 px of air a shadow used to go in. `_sync_dock` runs - # first so the pill's own position already reflects the width it is about - # to share, and `_docked_above` is set here because this is the one place - # that already decides which side the seam is actually on. - self.pill._sync_dock() - left, top, right, bottom = self.pill.work - x = self.pill.x + self.pill.pill_w - BUBBLE_W - above = self.pill.y - self._h - below = self.pill.y + PILL_H - # Above whenever above fits, which is every ordinary placement and is why this - # reads as one anchor rather than two. Below only when above has no room and below - # does — a fallback, not a mode. When *neither* fits, `above` goes through and the - # clamp below does what it has always done: a window as tall as the desktop cannot - # be placed clear of a pill on either side of it, and inventing a third rule for - # that would be pretending otherwise. - fits_below = above < top + EDGE_AIR and below + self._h <= bottom - EDGE_AIR - self.pill._docked_above = not fits_below - y = below if fits_below else above - x = max(left + EDGE_AIR, min(x, right - BUBBLE_W - EDGE_AIR)) - y = max(top + EDGE_AIR, min(y + lift, bottom - self._h - EDGE_AIR)) - self.geometry(f"{BUBBLE_W}x{self._h}+{x}+{y}") - - def _float_up(self) -> None: - """Fade into place. It used to rise into place, and the rise is gone. - - R14 asked for an appearance the eye could follow, and 18 px of travel over eight - frames was the answer for a window that also changed size on every render — one - more thing moving among several. With the panel a fixed shape now (`PANEL_H`), - the movement is the *only* thing moving, and a window that slides every time a - draft appears is exactly the "sore to eyes" the owner reported. The fade still - gives the eye its cue and costs no motion at all. - - Generation-guarded. Each run schedules eight `after` callbacks, so two - overlapping runs would fight over the alpha — which is what a fast - show/hide/show cycle produces. - """ - steps = 8 - self._anim += 1 - mine = self._anim - - def step(i: int) -> None: - if not self._visible or mine != self._anim: - return - t = i / steps - ease = 1 - (1 - t) ** 3 - self.attributes("-alpha", 0.96 * ease) - self.reposition() - if i < steps: - self.after(16, step, i + 1) + def reposition(self) -> None: + """Take the top band of the pill's window, or give it back. - step(0) + **This used to place a window**, and it took a `lift` argument so `_float_up` + could animate it in. Both are gone: the panel is the band above the pill row, at + x=0, and there is nothing left to move it relative to. + `_float_up` went with it. R14 asked for an appearance the eye could follow, and + 18 px of travel earned that when the bubble was a separate window arriving beside + another one. Inside a single shell there is nothing to arrive *at* — the window + itself grows upward from a bottom edge that never moves, which is the same cue + with no motion under it. + """ + self.pill._sync_shell() + if self._visible: + self.place(x=0, y=0, width=BUBBLE_W, height=self._h) + else: + self.place_forget() # -- holding still under the hand -------------------------------------- @@ -5313,10 +5292,11 @@ def _render(self) -> None: c.delete("body") # Squared on the seam it shares with the docked pill, rounded on the free # side — `reposition` is what decides above-vs-below, since it already has to. - above = getattr(self.pill, "_docked_above", True) - corners = (8, 8, 0, 0) if above else (0, 0, 8, 8) + # Always the top band of the one window now: rounded head, squared foot on the + # join it shares with the pill row below it. + corners = (8, 8, 0, 0) _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, - seam="bottom" if above else "top") + seam="bottom") y = PAD if self._sent: c.create_text( @@ -5585,7 +5565,7 @@ def tick_activity(self) -> None: if surfacing: self._for_activity = True self._visible = True - self.deiconify() + self.pill._sync_shell() elif act is None and self._for_activity and not ( self._text or self._partial or self._note ): @@ -5594,10 +5574,9 @@ def tick_activity(self) -> None: if self._visible: self._render() if surfacing: - # After the render, not before: `_float_up` repositions against `self._h`, - # which is what the render computes. Animating first moves the window to a - # height it does not have yet and the rise starts with a jump. - self._float_up() + # After the render, not before: `reposition` places the band against + # `self._h`, which is what the render computes. + self.reposition() def _indicator(self, y: int) -> None: """The one row that says what Flow is doing, and whether it can still hear. diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 740b106..fc6cfe7 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -50,6 +50,9 @@ def bubble(text: str = "", **kw): #: `reposition` does arithmetic on it now that the pill's width can dock. b.pill.pill_w = ui.PILL_W b.pill.work = WORK + # The panel band's height comes off the pill now that they share a window, + # so a Mock pill would otherwise answer `panel_h()` with a Mock. + b.pill.band_h = lambda: ui.PANEL_H b.pill.session = mock.Mock( mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None, @@ -314,284 +317,43 @@ def test_this_window_has_no_way_to_show_an_answer_at_all(self): #: The four corners of the work area a pill can be dragged to. The bubble anchors above and #: to the right of the pill, so these are the four directions the anchor can point off. -def corners(pill_w: int = None, pill_h: int = None): - left, top, right, bottom = WORK - pill_w = ui.PILL_W if pill_w is None else pill_w - pill_h = ui.PILL_H if pill_h is None else pill_h - return { - "top-left": (left, top), - "top-right": (right - pill_w, top), - "bottom-left": (left, bottom - pill_h), - "bottom-right": (right - pill_w, bottom - pill_h), - } - - -def geometry_of(b, x: int, y: int) -> str: - """Render with the pill at (x, y) and return the geometry string itself. - - The real `reposition` rather than the fixture's stub, and the string it built rather - than a recomputation of it: a check that re-derives the formula it is checking passes - whatever the formula says. +class TestTheBubbleIsABandInThePillsWindow(unittest.TestCase): + """What replaced 39 tests about where this window goes. + + Those tests were right for as long as this was a window: they checked that the + bubble opened above the pill, fell back to below when the pill was at the top of the + screen, stayed inside the work area at every corner, and did all of it byte-for-byte + identically to the frame before. Every one of them was asking the same question — + *are these two windows still touching?* + + There is one window now. The bubble is a `Frame` placed at (0, 0) inside it, and the + pill row is a canvas at its foot. Nothing can come apart, so there is nothing left to + check here — the shell's own geometry is `Pill._sync_shell`'s, and + `test_pill.TestTheShellIsOneWindow` is where it is checked. """ - b.pill.x, b.pill.y = x, y - b.reposition = ui.Bubble.reposition.__get__(b) - box: list[str] = [] - b.geometry = box.append - b._render() - return box[-1] - - -def placed(b, x: int, y: int) -> tuple[int, int, int, int]: - """The window rect `reposition` computes, as (x1, y1, x2, y2).""" - size, _, offset = geometry_of(b, x, y).partition("+") - w, _, h = size.partition("x") - px, _, py = offset.partition("+") - return int(px), int(py), int(px) + int(w), int(py) + int(h) - - -class TestTheWindowIsInsideTheWorkAreaWhereverThePillIs(unittest.TestCase): - """Item 37 bounded the draft's size; nothing bounded the window's placement. - - Measured on a real `tk.Tk` before the fix, with the pill put at each corner of the work - area and the rect read back from `GetWindowRect` as well as from Tk — 12 of 36 - placements left the desktop, all of them on the reply path and all off the **bottom**: - a 4 000-character answer sized the window **1 459 px** and a 12 000-character artifact - **4 179 px**, both pinned at `top + 8` on a 672 px work area, so the chip row landed at - screen y **1 427** and **4 147**. - - Worth saying plainly, because the decision reads the owner's screenshot the other way - round: the **top** edge was never the breach. `max(top + EDGE_AIR, …)` has held it at - every corner in every state. The finding stands exactly as the decision states it — the - bubble leaves the screen by position and takes the chips with it — and the edge it - leaves by is the bottom. - """ - - #: The reply states left this table on 2026-08-03 with `show_reply`. They were the - #: only ones that ever sized this window past the desktop -- a draft is capped at - #: `BODY_MAX_H` -- so what is left is the two item 37 already bounded. The tall-window - #: guarantees they were pinning are asserted on `ConversationCard` now, which is the - #: window that can be that tall. - def states(self): - return [ - ("1k draft", {"_text": draft(1_000)}), - ("50k draft", {"_text": draft(50_000)}), - ] - - def test_every_edge_is_inside_the_work_area_at_every_corner(self): - left, top, right, bottom = WORK - for label, state in self.states(): - for corner, (px, py) in corners().items(): - with self.subTest(state=label, corner=corner): - x1, y1, x2, y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual(y1, top, "the top edge left the work area") - self.assertLessEqual(y2, bottom, "the bottom edge left the work area") - self.assertGreaterEqual(x1, left) - self.assertLessEqual(x2, right) - - def test_the_chip_row_is_inside_it_too(self): - # The property the height bound exists for, and the one a placed-only clamp would - # fake: the row is drawn from `self._h`, so a window bounded without bounding the - # height would put the chips below its own bottom edge and look fixed. - _left, top, _right, bottom = WORK - for label, state in self.states(): - for corner, (px, py) in corners().items(): - with self.subTest(state=label, corner=corner): - b = bubble(**state) - _x1, y1, _x2, _y2 = placed(b, px, py) - chip_top = y1 + b._h - ui.PAD - ui.CHIP_H - chip_bottom = y1 + b._h - ui.PAD - self.assertGreaterEqual(chip_top, top) - self.assertLessEqual(chip_bottom, bottom, "the chips are off screen") - - def test_the_longest_draft_still_does_not_size_the_window_past_the_desktop(self): - b = bubble(draft(50_000)) - b._render() - self.assertLessEqual(b._h, WORK[3] - WORK[1] - 2 * ui.EDGE_AIR) - def test_and_a_short_one_still_sizes_the_window_to_itself(self): - # The other direction, so the bound cannot pass this by firing for everything. - b = bubble(draft(200)) + def band(self, b): + put: list[dict] = [] + b.reposition = ui.Bubble.reposition.__get__(b) + b.place = lambda **kw: put.append(kw) + b.place_forget = lambda: put.append({}) b._render() - self.assertLess(b._h, WORK[3] - WORK[1] - 2 * ui.EDGE_AIR) - - def test_the_air_is_one_number_and_both_places_use_it(self): - # `EDGE_AIR` is what makes the clamp a proof rather than a best effort — the height - # is fitted to `work - 2 * air` and the position is clamped by `air`, and the two - # have to be the same number. A literal in either place is how they drift apart. - # - # Asserted against `reposition` directly rather than through a state that happens to - # fill the desktop: item 45 gave the reply a head window, so nothing renders to - # exactly `work_h` any more and a check that relied on one would have been pinning a - # coincidence. - _left, top, right, bottom = WORK - b = bubble() - b._h = bottom - top # taller than the fit allows, which is what a clamp is for - box: list[str] = [] - b.geometry = box.append - b.pill.x, b.pill.y = corners()["bottom-right"] - ui.Bubble.reposition(b) - self.assertEqual(box[-1].partition("+")[2], - f"{right - ui.BUBBLE_W - ui.EDGE_AIR}+{top + ui.EDGE_AIR}") - b._h = bottom - top - 2 * ui.EDGE_AIR # exactly the fit - ui.Bubble.reposition(b) - _size, _, offset = box[-1].partition("+") - self.assertEqual(int(offset.partition("+")[2]) + b._h, bottom - ui.EDGE_AIR) - - -#: Three x positions along the top edge of the work area — the pill dragged where there is -#: no "above" left. Left, middle and right, because the anchor is horizontal as well as -#: vertical and a fallback that only worked in one corner would pass a single-point check. -def along_the_top() -> dict[str, tuple[int, int]]: - left, top, right, _bottom = WORK - return { - "top-left": (left, top), - "top-middle": ((left + right - ui.PILL_W) // 2, top), - "top-right": (right - ui.PILL_W, top), - } - - -#: Every geometry string `reposition` produced **before** item 44, captured by running the -#: harness against the tree as it stood. This is the regression half and it is a table rather -#: than a formula on purpose: a check that recomputes what it is checking cannot fail. -#: -#: The rows absent from it are the ones the fallback is *for* — a draft-sized window with the -#: pill along the top, where "above" has no room. Everything else must come through byte for -#: byte, including the reply-sized windows at the top, which are taller than either side of -#: the pill and so keep today's clamp. -#: -#: **The reply rows are gone, and that is the second time they moved rather than the first -#: time they were rewritten.** Item 45 re-captured them at 643 px where the full-text probe -#: had sized them 656; item 63 removed the path, because this window no longer draws an -#: answer. The draft rows below are byte-identical to the day they were captured, which is -#: the whole point of a table: one that gets quietly re-baselined pins nothing. -#: Re-baselined 2026-08-09 for the IBM Plex Sans migration: `FONT_BODY` reports an -#: 18 px line to the real canvas against Segoe UI's 17, so the same capped draft lays -#: out one pixel taller (414 → 415) and the bottom-anchored placements ride up one -#: pixel to match (208 → 207, the same bottom edge). Traced to the font swap, not a -#: silent re-pin. -#: -#: Re-baselined again the same day for docking (Phase 5): the pill and this window -#: meet at one hairline seam now, not the 10 px of air a shadow used to go in, so -#: every placement that resolved to "above" moves ten pixels closer to the pill — -#: 207 → 217 here. "mid-left" is unaffected because that placement was already -#: resolving to a *different* branch of `reposition`'s clamp, one the gap never -#: reached. -#: -#: Re-baselined a third time the same day: `BUBBLE_W` moved to 420 (Phase 6, the -#: two panels unified at the draft's own widest state). Width and the right-anchored -#: x shift with it everywhere (380→420, 892→852). The 1 000-character draft's height -#: drops too (415→398) — wider text wraps to fewer lines for the same character -#: count — while the 50 000-character one holds at 415, because that row is capped -#: by `BODY_MAX_H` rather than by how the text wraps, and a cap does not move with -#: the column it bounds. -#: Re-recorded when the panel became a fixed shape (`ui.PANEL_H`), and the new table says -#: the change out loud better than any prose could: **every row is the same 420x184**, at -#: three placements and across a draft that differs by fifty times its length. It used to -#: read 398 for a 1k draft and 415 for a 50k one — a window that changed size with the -#: text in it, which is the motion this table now proves is gone. -#: -#: Still a golden table and still doing its original job: any *further* change to -#: placement has to justify itself against these numbers. -GEOMETRY_BEFORE = { - ("1k draft", "bottom-left"): "420x184+8+448", - ("1k draft", "bottom-right"): "420x184+852+448", - ("1k draft", "mid-left"): "420x184+8+152", - ("50k draft", "bottom-left"): "420x184+8+448", - ("50k draft", "bottom-right"): "420x184+852+448", - ("50k draft", "mid-left"): "420x184+8+152", -} - - -class TestTheBubbleOpensBelowWhenAboveHasNoRoom(unittest.TestCase): - """A fallback, not a mode — tooltip behaviour, and item 42's desk check found the need. - - With the pill dragged to the top of the work area there is no "above" left, so the - bubble clamped to the top edge and was drawn **over the pill it is anchored to**. - Nothing clipped and nothing was unreachable — item 42 guarantees that and this must not - take it away — but an anchor pointing at something it covers is not an anchor. - - Above is tried first and used whenever it fits. Below is used only when above does not - fit *and* below does. When **neither** fits — a window as tall as the desktop, which is - what a full reply is — the arithmetic is today's exactly and the bubble clamps to the top - over the pill. That case is not fixed here, deliberately: no anchor can place a window - taller than the space either side of it, and pretending otherwise would be a third rule. - """ - - #: The reply states left this table on 2026-08-03 with `show_reply`. They were the - #: only ones that ever sized this window past the desktop -- a draft is capped at - #: `BODY_MAX_H` -- so what is left is the two item 37 already bounded. The tall-window - #: guarantees they were pinning are asserted on `ConversationCard` now, which is the - #: window that can be that tall. - def states(self): - return [ - ("1k draft", {"_text": draft(1_000)}), - ("50k draft", {"_text": draft(50_000)}), - ] - - def test_a_pill_along_the_top_opens_the_bubble_below_it(self): - # The defect, stated as geometry: the bubble's top must not be above the pill's - # bottom. Red at all three positions before this item, where it sat at y=8 with the - # pill occupying y=0..40. - _left, top, _right, _bottom = WORK - for label, state in self.states(): - for name, (px, py) in along_the_top().items(): - with self.subTest(state=label, at=name): - _x1, y1, _x2, _y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual( - y1, py + ui.PILL_H, - "the bubble is drawn over the pill it is anchored to", - ) - - def test_every_other_placement_is_byte_identical(self): - for (label, name), before in GEOMETRY_BEFORE.items(): - state = dict(self.states())[label] - places = dict(along_the_top()) - places.update(corners()) - places["mid-left"] = (WORK[0], (WORK[1] + WORK[3]) // 2) - with self.subTest(state=label, at=name): - self.assertEqual(geometry_of(bubble(**state), *places[name]), before) - - def test_above_is_still_the_default_wherever_it_fits(self): - # The other direction. A pill in its usual place has room above it, and the bubble - # must still be there — a fallback that fired whenever it could would be a mode. - b = bubble(_text=draft(1_000)) - _x1, y1, _x2, y2 = placed(b, *corners()["bottom-right"]) - self.assertLessEqual(y2, WORK[3] - ui.PILL_H, - "the bubble should sit above the pill, not below it") - - def test_when_neither_side_fits_the_clamp_is_todays(self): - # A window as tall as the desktop has no room on either side of a pill. This is - # the case the fallback deliberately does not fix, and it is pinned so nobody - # reads its absence as an oversight. Driven against `reposition` directly now: - # the state that used to produce a desktop-tall bubble was a full reply, and this - # window has not drawn one since item 63. The card is where that height lives. - _left, top, _right, bottom = WORK - for name, (px, py) in along_the_top().items(): - with self.subTest(at=name): - b = bubble(draft(1_000)) - b._h = bottom - top - 2 * ui.EDGE_AIR - b.pill.x, b.pill.y = px, py - box: list[str] = [] - b.geometry = box.append - ui.Bubble.reposition(b) - self.assertEqual(int(box[-1].rpartition("+")[2]), top + ui.EDGE_AIR) - - def test_the_work_area_guarantee_survives_the_second_anchor(self): - # Item 42's property, re-asserted against the new placements: a second way to - # choose y is a second way to leave the desktop. - left, top, right, bottom = WORK - for label, state in self.states(): - for name, (px, py) in along_the_top().items(): - with self.subTest(state=label, at=name): - x1, y1, x2, y2 = placed(bubble(**state), px, py) - self.assertGreaterEqual(y1, top) - self.assertLessEqual(y2, bottom) - self.assertGreaterEqual(x1, left) - self.assertLessEqual(x2, right) - - -if __name__ == "__main__": - unittest.main() + return put[-1] + + def test_it_takes_the_full_width_at_the_top_of_the_window(self): + self.assertEqual(self.band(bubble(draft(1_000))), + {"x": 0, "y": 0, "width": ui.BUBBLE_W, "height": ui.PANEL_H}) + + def test_a_draft_fifty_times_longer_takes_exactly_the_same_band(self): + # The whole point of the fixed panel, restated where it is most visible: the + # band a 1k draft asks for and the one a 50k draft asks for are the same object. + self.assertEqual(self.band(bubble(draft(1_000))), + self.band(bubble(draft(50_000)))) + + def test_a_hidden_bubble_gives_its_band_back(self): + # `place_forget` rather than parking a window offscreen: there is no window to + # park, and the pill's shell shrinks to the row on the next `_sync_shell`. + self.assertEqual(self.band(bubble(draft(400), _visible=False)), {}) class TestTheChipsSurviveARedraw(unittest.TestCase): diff --git a/tests/test_card.py b/tests/test_card.py index 161f896..dc25e18 100644 --- a/tests/test_card.py +++ b/tests/test_card.py @@ -44,6 +44,9 @@ def card(**kw): #: `reposition` does arithmetic on it now that the pill's width can dock. c.pill.pill_w = ui.PILL_W c.pill.work = WORK + # The panel band's height comes off the pill now that they share a window, + # so a Mock pill would otherwise answer `panel_h()` with a Mock. + c.pill.band_h = lambda: ui.PANEL_H c.pill.x, c.pill.y = 900, 560 c.pill.session = mock.Mock(can_take_reply=True, auto_ask_in=None) c.canvas = MeasuringCanvas() @@ -55,11 +58,12 @@ def card(**kw): c._h = ui.CARD_MIN_H c._pinned_h = 0 c._countdown = None + #: What `reposition` asked for. It used to be a `geometry` string, because this was + #: a window; it is a `place` call now, because the card is a band inside the pill's + #: one window. The record is kept because tests read the height back out of it. c.placed = [] - c.geometry = c.placed.append - c.deiconify = lambda: None - c.attributes = lambda *a, **kw: None - c.withdraw = lambda: None + c.place = lambda **kw: c.placed.append(f"{kw['width']}x{kw['height']}+0+0") + c.place_forget = lambda: None for name, value in kw.items(): setattr(c, name, value) return c @@ -720,7 +724,9 @@ def test_switching_back_finds_the_answer_waiting(self): self.assertEqual(c._answer, "you add it with a migration") c.deiconify.assert_not_called() c.show() - c.deiconify.assert_called_once() + # It used to `deiconify` a window of its own. There is one window now, and a + # band that has been given a place in it is a band that is showing. + self.assertTrue(c.placed) @staticmethod def raised(call) -> bool: diff --git a/tests/test_editor.py b/tests/test_editor.py index c5f9cd8..ae11191 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -605,6 +605,7 @@ def _bubble(self, note: str, text: str = "a draft"): #: is an unpackable that raises, which is the loud failure this would rather have #: than a fixture silently laying out against a screen of no particular size. b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.canvas = MeasuringCanvas() b._text, b._sent, b._partial, b._note = text, "", "", note b._editor = None @@ -674,6 +675,7 @@ def _bubble(self, **session): b.pill.session = mock.Mock(**fields) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -751,6 +753,7 @@ def _bubble(self): ) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -840,6 +843,7 @@ def _bubble(self, partial: str, note: str = "local: replace('x' -> 'y')"): ) b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.canvas = MeasuringCanvas() b._text = "seconds, send the question. No auto-ask to press it yourself." b._sent, b._partial, b._note = "", partial, note @@ -957,6 +961,7 @@ def bubble(self, first=0.0, last=0.3, lines=60, height=200): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.pill.session = mock.Mock(mode="dictate", editing=True, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() @@ -1123,6 +1128,7 @@ def bubble(self, text="a draft", sent=""): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK + b.pill.band_h = lambda: ui.PANEL_H b.pill.session = mock.Mock(mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() diff --git a/tests/test_pill.py b/tests/test_pill.py index e19a860..2cdf84d 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -35,6 +35,12 @@ def __init__(self) -> None: def delete(self, *a, **kw) -> None: ... + #: `_sync_shell` places this canvas at the foot of a window whose top edge moves + #: when a panel opens. Recorded rather than ignored, so a test can check the row + #: really is at the bottom of whatever height the shell currently is. + def place(self, **kw) -> None: + self.placed = kw + #: `_sync_dock` resizes the canvas when a panel docks or goes away. Accepted and #: recorded rather than ignored, so a test can tell a widened pill from a moved one. def configure(self, **kw) -> None: @@ -1180,130 +1186,90 @@ def docker(*, showing=True, panel_w=ui.BUBBLE_W, window=None, x=1047, docked_w=u return p -class TestTheDockIsCheckedRatherThanAssumed(unittest.TestCase): - """The pill and the panel above it share one column, and stay sharing it. +class TestTheShellIsOneWindow(unittest.TestCase): + """`_sync_shell`, which is what `_sync_dock` became when the dock stopped existing. - `scripts/reel.py` found them not sharing it: the pill 420 px wide at the x a - 205 px pill sits at, hanging 215 px off the screen and unjoined from the panel it - is docked to, held there for five seconds because the width already matched and - nothing looked again. + The pill and its panel used to be two windows kept adjacent by hand, and + `scripts/reel.py` once caught them **215 px apart, for five seconds**: the resize had + landed and the matching move had not, and the pill's remembered width then answered + "nothing to do" on every frame after. That failure needs two windows to be possible. + There is one now, and all that arithmetic has collapsed into a height. - Pinned to `"corner"`, because holding the right edge is *corner* placement's rule - and not the dock's: the pill is anchored to the screen edge it sits against, and - growing away from it is the only direction there is. Centred placement has no - anchored edge and re-centres instead — `TestTheDockRecentresWhenThereIsNoEdge` - below is the same class of check for it. + What the two classes here used to cover — holding the right edge in corner placement, + re-centring on the new width in bottom placement, the 107 px lurch when a draft + appeared — is gone with the width change that caused it. The pill is the panel's + width whether a panel is up or not. """ def setUp(self): self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) - ui.apply_place("corner") - - def test_a_panel_appearing_moves_the_left_edge_and_holds_the_right(self): - p = docker() - p._sync_dock() - p.geometry.assert_called_once_with(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608") - self.assertEqual(p.x + ui.BUBBLE_W, 1047 + ui.PILL_W) # the right edge did not move - - def test_nothing_is_asked_for_twice_when_the_window_already_agrees(self): - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 832, 608)) - p._sync_dock() - p.geometry.assert_not_called() - - def test_a_move_that_did_not_land_is_asked_for_again(self): - # The reel's finding, staged: the resize took and the move did not, so the - # window is at the bare-pill x while the pill's own state says it docked. - # Before this was checked, `w == self._docked_w` returned here and the pill - # stayed off the screen edge for as long as the panel was up. - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 1047, 608)) - p._sync_dock() - p.geometry.assert_called_once_with(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608") - - def test_the_recovery_does_not_move_the_pill_a_second_time(self): - # Re-asking must re-send the *same* geometry, never re-run the relative - # arithmetic — that would walk the pill one panel-width left per frame. - p = docker(docked_w=ui.BUBBLE_W, x=832, window=(ui.BUBBLE_W, 1047, 608)) - for _ in range(5): - p._sync_dock() - self.assertEqual(p.x, 832) - self.assertEqual({c.args for c in p.geometry.call_args_list}, - {(f"{ui.BUBBLE_W}x{ui.PILL_H}+832+608",)}) - - def test_the_panel_going_away_leaves_the_width_alone(self): - """It used to take the width back, and taking it back was the thing to stop. - - The pill was `PILL_W` while nothing was docked and the panel's width while - something was, so it jumped 205 -> 420 the instant a draft appeared and back - again when it went — the most visible motion on the screen, on every utterance. - `Pill.pill_w` answers the panel's width unconditionally now, so a panel coming - and going costs no resize and no move at all. - """ - p = docker(showing=False, docked_w=ui.BUBBLE_W, x=832) - p._sync_dock() - self.assertEqual(p.x, 832) - p.geometry.assert_not_called() - + ui.apply_place("bottom") -class TestTheDockRecentresWhenThereIsNoEdge(unittest.TestCase): - """The same dock under centred placement, where holding an edge is the bug. + def asked(self, p): + w, _, rest = p.geometry.call_args.args[0].partition("x") + h, _, _pos = rest.partition("+") + return int(w), int(h) - Found by running the shipped placement against the dock rather than by reading it: - the pill is 205 px and the panel 420, and holding the right edge put the pair - **107 px left of centre the moment a draft appeared** — visibly, on every utterance, - because a draft appearing is the single most common thing that happens. + def test_a_panel_opening_grows_the_window_upward(self): + p = docker(showing=True) + p._sync_shell() + self.assertEqual(self.asked(p), (ui.BUBBLE_W, ui.PANEL_H + ui.PILL_H)) - Corner placement is anchored to the screen edge it sits against, so growing away - from that edge is the only direction there is. Centred placement has no anchored - edge, and the width it should be centred on is the *new* one. - """ + def test_and_the_foot_does_not_move_when_it_does(self): + """The whole of "the controls stay where they are", as arithmetic. - def setUp(self): - self.addCleanup(ui.apply_place, ui.PLACE_DEFAULT) - ui.apply_place("bottom") + Send, the meter and the chip row are laid out from the bottom of the window. A + shell that grew downward — or that centred its growth — would move every one of + them every time a draft appeared, which is the motion this work exists to end. + """ + idle = docker(showing=False) + idle._sync_shell() + opened = docker(showing=True) + opened._sync_shell() + self.assertEqual(idle.y + idle._shell_h, opened.y + opened._shell_h) - def centre_of(self, p, w): - return p._placed(w)[0] - - def test_a_panel_appearing_recentres_on_the_new_width(self): - p = docker(x=538, docked_w=ui.PILL_W) - p._sync_dock() - self.assertEqual(p.x, self.centre_of(p, ui.BUBBLE_W)) - - def test_the_panel_going_away_moves_nothing(self): - # There is no narrower width to recentre on any more: the pill is the panel's - # width whether a panel is up or not. - p = docker(showing=False, docked_w=ui.BUBBLE_W, x=430) - p._sync_dock() - self.assertEqual(p.x, self.centre_of(p, ui.BUBBLE_W)) - - def test_the_stack_stays_put_across_an_open_and_a_close(self): - # The round trip, which is now a round trip to nowhere. A pill upgrading from an - # old profile still gets centred on the panel width once, and then a panel - # opening and closing leaves it exactly where that put it. - p = docker(x=538, docked_w=ui.PILL_W) - p._sync_dock() - settled = p.x - self.assertEqual(settled, self.centre_of(p, ui.BUBBLE_W)) - p.bubble = mock.Mock(width=ui.BUBBLE_W, _visible=False) - p.window_geometry = mock.Mock(return_value=(ui.BUBBLE_W, p.x, 608)) - p._sync_dock() - self.assertEqual(p.x, settled) - - def test_it_is_still_clamped_onto_the_screen(self): - # The clamp survives the branch. A panel wider than the display would otherwise - # be centred to a negative x. - p = docker(x=538, docked_w=ui.PILL_W, panel_w=4000) - p._sync_dock() - self.assertGreaterEqual(p.x, p.work[0]) - - def test_corner_placement_is_untouched_by_any_of_this(self): - # The two branches are different rules, not one rule with a flag, and the - # corner's is the one people have been using. - ui.apply_place("corner") - p = docker() - p._sync_dock() - self.assertEqual(p.x + ui.BUBBLE_W, 1047 + ui.PILL_W) + def test_an_idle_pill_is_just_the_row(self): + p = docker(showing=False) + p._sync_shell() + self.assertEqual(p._shell_h, ui.PILL_H) + def test_nothing_is_asked_for_twice_when_the_window_already_agrees(self): + # Idempotent, so it can run from the frame pump and from a panel's `reposition` + # without either caring which got there first. + p = docker(showing=False, x=538) + p._sync_shell() + p.window_geometry = mock.Mock(return_value=(ui.BUBBLE_W, p.x, p.y)) + p.geometry.reset_mock() + for _ in range(5): + p._sync_shell() + p.geometry.assert_not_called() -if __name__ == "__main__": + def test_a_window_that_drifted_is_asked_again(self): + # `reel.py`'s defect, in the only form it can still take. Compared against the + # *window* rather than against a remembered value: state saying the move happened + # is not evidence that it did. + p = docker(showing=True, window=(ui.BUBBLE_W, 4, 4)) + p._sync_shell() + p.geometry.assert_called_once() + + def test_the_shell_is_clamped_onto_the_screen(self): + p = docker(showing=True) + p.work = p.full = (0, 0, 300, 200) + p._sync_shell() + self.assertGreaterEqual(p.x, 0) + self.assertGreaterEqual(p.y, 0) + self.assertLessEqual(p.y + p._shell_h, 200) + + def test_the_row_is_placed_at_the_foot_of_whatever_height_it_is(self): + # The canvas is the bottom band of the window, not the whole of it — which is + # what makes "the foot never moves" true of the pixels and not just of the frame. + for showing in (False, True): + with self.subTest(showing=showing): + p = docker(showing=showing) + p._sync_shell() + kw = p.canvas.place.call_args.kwargs + self.assertEqual(kw["y"] + kw["height"], p._shell_h) + + +if __name__ == "__main__": # pragma: no cover unittest.main() From 4b35170bd8dafba2be8a0a76b42907404d91c5d6 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 12:13:02 -0500 Subject: [PATCH 27/41] Size the panel to its content again, stepping a line at a time A FluidVoice demo, read frame by frame, settled an argument I had got wrong. Its overlay's bottom edge is at y=554 in every frame of thirteen seconds - idle, two lines, two lines, three lines - and the box is snug around the text in every one of them. It never holds empty space, and at the end it is gone. So the fixed PANEL_H was stricter than the thing it was copying. It stopped the motion and paid for it with a hole in the middle of the window, which is the same complaint the resizing caused wearing different clothes. What must not move is the foot, not the height. PANEL_H becomes PANEL_MAX_H, a ceiling. The band is sized from its content and snapped up to a whole body line by _settled_h. That snap is what replaces FluidVoice's 80 ms debounce: sizing to content is right, and a timer cancelled from a render loop that runs thirty times a second is a thing to get wrong. A height that can only change when the text gains or loses a line changes a handful of times an utterance by construction. The shots caught a regression the tests did not: _sync_shell asked band_h() rather than the band's actual height, so a 130 px band sat in a 224 px shell with the row floating 54 px below it - the detached-boxes look the merge exists to end. Measured on the running app: as a draft grows the shell steps 170 -> 204 -> 221 px while the foot stays at 648 in every frame. Three tests flipped back to asserting growth, which is what they asserted before the fixed height and is right again. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 126 +++++++++++++++++++++++++++++-------------- tests/test_bubble.py | 27 ++++++---- tests/test_card.py | 21 ++++---- tests/test_editor.py | 41 +++++++------- tests/test_pill.py | 20 +++++-- 5 files changed, 151 insertions(+), 84 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 7122448..661acd7 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1079,25 +1079,31 @@ def _unload_fonts() -> None: #: on, which leaves the pill its own room underneath. BODY_MAX_H = 340 -#: **One height, for every panel, always.** The windows used to size themselves to their -#: contents on every render, and the owner's verdict on that was the reason this exists: -#: "window being re-draw and adjusting it self ... multiple windows size is problem ... -#: that cost more and sore to eyes". They were right, and the shots show it — a two-line -#: partial made a 140 px panel and a three-line draft a 215 px one, so the panel grew and -#: the pill docked under it moved *while you were still speaking*. Nothing that changes -#: shape thirty times a second can be read. +#: The tallest a panel band may be, and no longer the height it always is. #: -#: 184 px, and the number was chosen by looking rather than by arithmetic. A fixed panel -#: trades a void against elision, and both cost: at 236 px a two-line partial left a hole -#: in the middle of the window, which reads worse than the resizing it replaced. 184 is -#: the four body lines a dictated sentence actually comes to in `scripts/shots.py`, plus -#: the note line and the chip row — so the common case fills it and only a long draft -#: elides, which is machinery that already existed for the window's old ceiling and -#: needed no changes to serve a fixed one. +#: **This was a fixed height, and the reference says it should not be.** A demo of +#: FluidVoice, read frame by frame, settles it: the overlay's bottom edge is at y=554 in +#: every frame from idle through three lines of growth, and the box is *snug* around the +#: text in each one — two lines at 0:05, two at 0:08, three at 0:11. It never holds empty +#: space. Pinning the height bought stability at the price of a hole in the middle of the +#: window, which is the same complaint the resizing caused, wearing different clothes. #: -#: Clamped to the work area by its callers, which is the one thing that may still shrink -#: it: a panel taller than the desktop is not a shape, it is a bug. -PANEL_H = 184 +#: What that overlay does instead is size to its content and *debounce* the resize — +#: `scheduleSizeAndPositionUpdate`, 80 ms, cancel-and-reschedule, with +#: `animationBehavior = .none`. Streaming partials coalesce into one step instead of +#: thirty resizes a second. Flow gets the same result without a timer, by snapping to +#: whole body lines (`_settled_h`): a height that can only change when the text gains or +#: loses a *line* changes a handful of times an utterance, and a timer that has to be +#: cancelled correctly from a render loop is a thing to get wrong. +PANEL_MAX_H = 184 + +#: The shortest a band gets, so a one-word draft still has a panel rather than a sliver. +PANEL_MIN_H = 96 + +#: What the body font measures per line — the number `BODY_MAX_H` is already built from +#: ("340 px is 20 lines at the 17 px the body font measures"). Named here because the +#: band now steps by it. +BODY_LINE_H = 17 #: The live partial's own ceiling, and it needs one for the same reason the draft does: #: it is wrapped to the full body column, so it is a multi-line block whose length nobody @@ -3136,7 +3142,7 @@ def band_h(self) -> int: """ _left, top, _right, bottom = self.work room = (bottom - top) - 2 * EDGE_AIR - PILL_H - return max(0, min(PANEL_H, room)) + return max(0, min(PANEL_MAX_H, room)) def _placed(self, w: int) -> tuple[int, int]: """Where a stack `w` wide belongs on the current monitor, per `PLACE`. @@ -3649,7 +3655,13 @@ def _sync_shell(self) -> None: happened is not evidence that it did, and comparing against the window means the next frame fixes whatever dropped it. """ - h = PILL_H + (self.band_h() if getattr(self.front, "_visible", False) else 0) + # The band's *actual* height, not the ceiling it is allowed. Asking `band_h()` + # here made the shell 224 px tall around a 130 px band and left the row floating + # 54 px below it — which the shots caught immediately, because a detached row is + # exactly the two-boxes look this window was merged to end. + front = self.front + band = min(self.band_h(), max(0, int(getattr(front, "_h", 0) or 0))) if getattr(front, "_visible", False) else 0 + h = PILL_H + band w = self.pill_w left, top, right, bottom = self.work foot = self._placed(w)[1] + PILL_H # where the pill row's bottom edge belongs @@ -4325,16 +4337,32 @@ def work_h(self) -> int: return bottom - top - 2 * EDGE_AIR def panel_h(self) -> int: - """The height this window is, always. See `PANEL_H`. + """The tallest this band may be. Asked of the pill, which owns the window. - A method on both panels rather than the bare constant, because the desktop is - still allowed to be smaller than the shape we would like: a panel taller than the - display it is on is not a consistent shape, it is a window off the screen. + The row shares that window, so the band's ceiling is what the desktop has left + after the row has taken its 40 px. """ - # Asked of the pill, because the row shares this window and its height comes - # off the top of what the desktop has left. return self.pill.band_h() + def _settled_h(self, want: int) -> int: + """`want`, rounded up to a whole body line and clamped to the band's ceiling. + + **The snap is what replaces FluidVoice's 80 ms debounce.** Its overlay sizes to + its content and coalesces the resizes on a timer; sizing to content is right and + the timer is a thing to get wrong from inside a render loop that already runs + thirty times a second. A height that can only change when the text gains or loses + a *line* changes a handful of times an utterance by construction — no cancelling, + nothing to leak, and the same absence of thrash. + + The foot does not move whatever this returns: `Pill._sync_shell` grows the window + upward from a fixed bottom edge, so a step here moves the top edge and nothing + else. + """ + want = max(PANEL_MIN_H, want) + over = want - PANEL_MIN_H + want = PANEL_MIN_H + -(-over // BODY_LINE_H) * BODY_LINE_H + return max(PANEL_MIN_H, min(want, self.pill.band_h())) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" @@ -4486,10 +4514,10 @@ def _render(self) -> None: history_h = sum(h + CARD_GAP for h in self._heights) # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - # `history_h` and `_pinned_h` still decide what is drawn and what scrolls; - # the window is the same size either way. A card that grew with the - # conversation moved the pill under it on every answer. - self._h = self.panel_h() + # Snug around what is on the card, stepping a line at a time — and the pill + # row below it does not move when it steps, because the shell grows upward. + self._h = self._settled_h( + PAD + history_h + self._pinned_h + HELP_FOOT_BAND) c.configure(width=CARD_W, height=self._h) self.reposition() @@ -4861,16 +4889,32 @@ def work_h(self) -> int: return bottom - top - 2 * EDGE_AIR def panel_h(self) -> int: - """The height this window is, always. See `PANEL_H`. + """The tallest this band may be. Asked of the pill, which owns the window. - A method on both panels rather than the bare constant, because the desktop is - still allowed to be smaller than the shape we would like: a panel taller than the - display it is on is not a consistent shape, it is a window off the screen. + The row shares that window, so the band's ceiling is what the desktop has left + after the row has taken its 40 px. """ - # Asked of the pill, because the row shares this window and its height comes - # off the top of what the desktop has left. return self.pill.band_h() + def _settled_h(self, want: int) -> int: + """`want`, rounded up to a whole body line and clamped to the band's ceiling. + + **The snap is what replaces FluidVoice's 80 ms debounce.** Its overlay sizes to + its content and coalesces the resizes on a timer; sizing to content is right and + the timer is a thing to get wrong from inside a render loop that already runs + thirty times a second. A height that can only change when the text gains or loses + a *line* changes a handful of times an utterance by construction — no cancelling, + nothing to leak, and the same absence of thrash. + + The foot does not move whatever this returns: `Pill._sync_shell` grows the window + upward from a fixed bottom edge, so a step here moves the top edge and nothing + else. + """ + want = max(PANEL_MIN_H, want) + over = want - PANEL_MIN_H + want = PANEL_MIN_H + -(-over // BODY_LINE_H) * BODY_LINE_H + return max(PANEL_MIN_H, min(want, self.pill.band_h())) + @property def width(self) -> int: """This window's own width — what a docked pill takes on (`Pill.pill_w`).""" @@ -5239,7 +5283,10 @@ def _render(self) -> None: around += 16 if self._act is not None: around += 20 - body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self._h - around)) + # Against the band's *ceiling*, not against the height it happens to be: the + # height is about to be computed from this, so reading it here would let a short + # frame pin the body short on the next one and never grow back. + body_cap = max(BODY_ELIDED_H, min(BODY_MAX_H, self.panel_h() - around)) shown, earlier, text_h = self._body_slot(body, body_cap) if not body: # `_body_slot` probes `shown or " "` so `bbox` always has something to answer @@ -5282,10 +5329,9 @@ def _render(self) -> None: # character artifact to 4 179 px on a 672 px desktop. # Nothing moves or resizes under the hand — see `_frozen`. if not self._frozen(): - # `text_h` and `extra` still decide what is *drawn*; they no longer decide - # how big the window is. That is the whole change: the body elides into a - # fixed panel instead of the panel growing to fit the body. - self._h = self.panel_h() + # Snug around the draft again, stepping a line at a time rather than + # tracking every frame — see `_settled_h`. + self._h = self._settled_h(text_h + extra + 74) c.configure(width=BUBBLE_W, height=self._h) self.reposition() diff --git a/tests/test_bubble.py b/tests/test_bubble.py index fc6cfe7..3e08082 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -52,7 +52,7 @@ def bubble(text: str = "", **kw): b.pill.work = WORK # The panel band's height comes off the pill now that they share a window, # so a Mock pill would otherwise answer `panel_h()` with a Mock. - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock( mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None, @@ -341,14 +341,23 @@ def band(self, b): return put[-1] def test_it_takes_the_full_width_at_the_top_of_the_window(self): - self.assertEqual(self.band(bubble(draft(1_000))), - {"x": 0, "y": 0, "width": ui.BUBBLE_W, "height": ui.PANEL_H}) - - def test_a_draft_fifty_times_longer_takes_exactly_the_same_band(self): - # The whole point of the fixed panel, restated where it is most visible: the - # band a 1k draft asks for and the one a 50k draft asks for are the same object. - self.assertEqual(self.band(bubble(draft(1_000))), - self.band(bubble(draft(50_000)))) + band = self.band(bubble(draft(1_000))) + self.assertEqual((band["x"], band["y"], band["width"]), (0, 0, ui.BUBBLE_W)) + self.assertLessEqual(band["height"], ui.PANEL_MAX_H) + + def test_a_draft_past_the_ceiling_stops_at_the_ceiling(self): + # The band is snug around its content again, so this is no longer "every draft + # gets the same band" — it is "no draft gets more than the desktop has left". + # A 1k draft and a 50k one both overflow, so both stop in the same place. + self.assertEqual(self.band(bubble(draft(1_000)))["height"], + self.band(bubble(draft(50_000)))["height"]) + + def test_a_short_draft_takes_less_than_a_long_one(self): + # What the fixed height cost and this gets back: no empty space inside the + # window. FluidVoice's overlay is snug around two lines and then three; a demo + # of it, read frame by frame, is why this changed back. + self.assertLess(self.band(bubble("a note"))["height"], + self.band(bubble(draft(50_000)))["height"]) def test_a_hidden_bubble_gives_its_band_back(self): # `place_forget` rather than parking a window offscreen: there is no window to diff --git a/tests/test_card.py b/tests/test_card.py index dc25e18..3e9df7b 100644 --- a/tests/test_card.py +++ b/tests/test_card.py @@ -46,7 +46,7 @@ def card(**kw): c.pill.work = WORK # The panel band's height comes off the pill now that they share a window, # so a Mock pill would otherwise answer `panel_h()` with a Mock. - c.pill.band_h = lambda: ui.PANEL_H + c.pill.band_h = lambda: ui.PANEL_MAX_H c.pill.x, c.pill.y = 900, 560 c.pill.session = mock.Mock(can_take_reply=True, auto_ask_in=None) c.canvas = MeasuringCanvas() @@ -456,16 +456,19 @@ def test_the_count_grows_with_the_answer(self): for n in (6_000, 12_000)] self.assertLess(counts[0], counts[1]) - def test_the_answer_no_longer_sizes_the_card(self): - """It did, and that was the complaint. The card is one shape now. + def test_the_answer_sizes_the_card_again(self): + """It did; then it did not for two commits; now it does, and that is right. - A card that grew with its answer moved its own top edge on every reply, and the - owner's verdict on that family of motion was "sore to eyes". The answer still - decides what is *shown* — `more_line` above proves the head window still reports - what it left out — but not how big the window is. + Fixing the height stopped the card moving and left a hole in it instead. A + FluidVoice demo read frame by frame settled the argument: that overlay's bottom + edge is at y=554 in every frame while the box is snug around two lines, then + three. Snug is what a reader wants; what must not move is the *foot*, and + `Pill._sync_shell` grows the shell upward so the pill row never does. + + Stepping by a whole body line (`_settled_h`) is what keeps it from thrashing. """ - self.assertEqual(self.answered(prose(4_000))._h, - self.answered(prose(200))._h) + self.assertGreater(self.answered(prose(4_000))._h, + self.answered(prose(200))._h) class TestTheExitsCarryTheWholeAnswer(unittest.TestCase): diff --git a/tests/test_editor.py b/tests/test_editor.py index ae11191..cd24906 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -605,7 +605,7 @@ def _bubble(self, note: str, text: str = "a draft"): #: is an unpackable that raises, which is the loud failure this would rather have #: than a fixture silently laying out against a screen of no particular size. b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text, b._sent, b._partial, b._note = text, "", "", note b._editor = None @@ -625,22 +625,20 @@ def test_the_error_clears_the_chip_row(self): f"the note runs to y={note_bottom} and the chips start at y={chips_top}", ) - def test_the_room_comes_out_of_the_body_now_that_the_window_cannot_grow(self): - """It used to grow. The panel is a fixed shape now (`PANEL_H`), so it cannot. + def test_the_bubble_grows_to_make_room_rather_than_clipping(self): + """The other way to stop an overlap is to cut the text off, and for an error + message that is the same defect wearing a different hat. - The property under test never was "the window gets taller" — it was "a long note - is neither clipped nor sitting on the chips", which the sibling test above - asserts directly. What changed is where the room comes from: the body gives it - up, because the body is the one thing on this window that can be windowed and - say so. The note itself is still drawn whole, which is the half that matters — - errors are the longest strings this shows and the ones it is least acceptable to - hide. + Briefly untrue, while the panel was a fixed height — the room came out of the + body instead. The band is snug around its content again (`_settled_h`), so this + is back to what it always asserted, and the foot still does not move: the shell + grows upward. """ short = self._bubble("ok") short._render() long_ = self._bubble(self.ERROR) long_._render() - self.assertEqual(long_._h, short._h) + self.assertGreater(long_._h, short._h) drawn = next(i for i in long_.canvas.items if "WinError 2" in i["text"]) self.assertEqual(drawn["text"], self.ERROR, "the note must not be truncated") @@ -675,7 +673,7 @@ def _bubble(self, **session): b.pill.session = mock.Mock(**fields) b.pill.accent = "#000000" b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -753,7 +751,7 @@ def _bubble(self): ) b.pill.accent = "#000000" b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "Meeting on Tuesday afternoon." b._sent = b._partial = b._note = "" @@ -843,7 +841,7 @@ def _bubble(self, partial: str, note: str = "local: replace('x' -> 'y')"): ) b.pill.accent = "#000000" b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.canvas = MeasuringCanvas() b._text = "seconds, send the question. No auto-ask to press it yourself." b._sent, b._partial, b._note = "", partial, note @@ -873,16 +871,15 @@ def test_the_partial_clears_the_chip_row(self): f"the partial runs to y={bottom} and the chips start at y={chips_top}", ) - def test_the_window_is_the_same_size_whatever_the_partial_is(self): - # It used to grow for a long partial, which meant the window changed shape while - # you were still speaking — the motion the fixed panel exists to end. The - # partial's own cap and the body's are what keep it off the note; the sibling - # test above asserts that directly. + def test_the_bubble_grows_rather_than_clipping(self): + # Grows in whole-line steps rather than continuously (`_settled_h`), and upward, + # so a partial arriving while you speak moves the top edge and leaves every + # control where it was. one = self._bubble("part key towel control") one._render() many = self._bubble(self.PARTIAL) many._render() - self.assertEqual(many._h, one._h) + self.assertGreater(many._h, one._h) def test_a_partial_past_the_cap_keeps_its_tail(self): # Bounded like the draft is, and windowed the same way round: this is the @@ -961,7 +958,7 @@ def bubble(self, first=0.0, last=0.3, lines=60, height=200): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock(mode="dictate", editing=True, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() @@ -1128,7 +1125,7 @@ def bubble(self, text="a draft", sent=""): b.pill = mock.Mock() b.pill.accent = "#000000" b.pill.work = WORK - b.pill.band_h = lambda: ui.PANEL_H + b.pill.band_h = lambda: ui.PANEL_MAX_H b.pill.session = mock.Mock(mode="dictate", editing=False, can_rescue=False, can_take_reply=False, auto_ask_in=None) b.canvas = MeasuringCanvas() diff --git a/tests/test_pill.py b/tests/test_pill.py index 2cdf84d..193a665 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -678,7 +678,7 @@ def test_a_panel_that_is_up_is_moved_with_the_pill(self): # The panels are placed *from* the pill, so moving it is the whole move — but # only for a window somebody can see. p = self.pill(self.ONE) - p.bubble = mock.Mock(_visible=True, width=ui.BUBBLE_W) + p.bubble = mock.Mock(_visible=True, width=ui.BUBBLE_W, _h=ui.PANEL_MAX_H) with mock.patch.object(ui, "_pointer_monitor", return_value=self.TWO): p._sync_monitor() p.bubble.reposition.assert_called_once() @@ -1174,8 +1174,10 @@ def docker(*, showing=True, panel_w=ui.BUBBLE_W, window=None, x=1047, docked_w=u p = ui.Pill.__new__(ui.Pill) p.canvas = mock.Mock() p.session = mock.Mock(mode=DICTATE) - p.bubble = mock.Mock(width=panel_w, _visible=showing) - p.card = mock.Mock(width=panel_w, _visible=False) + # `_h` is a real int: the shell is sized from the band's actual height now, so a + # Mock there would put a Mock into the arithmetic. + p.bubble = mock.Mock(width=panel_w, _visible=showing, _h=ui.PANEL_MAX_H) + p.card = mock.Mock(width=panel_w, _visible=False, _h=ui.PANEL_MAX_H) p.work = (0, 0, 1280, 720) p.full = (0, 0, 1280, 720) p.x, p.y = x, 608 @@ -1211,9 +1213,19 @@ def asked(self, p): return int(w), int(h) def test_a_panel_opening_grows_the_window_upward(self): + # The band reports its own height now that it is snug around its content, so the + # shell is the row plus whatever that is — bounded by the ceiling. p = docker(showing=True) p._sync_shell() - self.assertEqual(self.asked(p), (ui.BUBBLE_W, ui.PANEL_H + ui.PILL_H)) + self.assertEqual(self.asked(p), (ui.BUBBLE_W, ui.PANEL_MAX_H + ui.PILL_H)) + + def test_the_shell_follows_the_band_rather_than_its_ceiling(self): + # A shell sized to the ceiling leaves the row floating below a shorter band, + # which is the detached-boxes look the merge exists to end. Caught in a shot. + p = docker(showing=True) + p.bubble._h = 130 + p._sync_shell() + self.assertEqual(self.asked(p), (ui.BUBBLE_W, 130 + ui.PILL_H)) def test_and_the_foot_does_not_move_when_it_does(self): """The whole of "the controls stay where they are", as arithmetic. From c7805795b898cf3adcf8c9e3ac7e303070ac1d92 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 13:16:00 -0500 Subject: [PATCH 28/41] Hide Flow to the notification area, with an icon that is the way back The need, in the owner's words: there are times where i wanted to dictate but at the same time i wanted to see but i don't want it to keep it on my screen. Parking the window is the easy half and the dangerous one. A Flow with no window and no icon is a process that cannot be reached, configured or quit except through Task Manager, which is invariant 4 wearing a different hat: hidden must not mean gone. So the icon is registered first and its answer is believed - no icon, no hiding, and the bubble says why. That refusal is the test this file exists for. Win32 through ctypes, no new dependency, R16 holds at three. Shell_NotifyIconW needs a window to send clicks to and Tk will not give us one, so flow/tray.py creates its own message-only window and pumps its own loop on its own thread. Nothing in that module touches Tk: the window procedure puts strings on a queue and the UI drains it from _frame, beside the hotkeys and for the same reason. The chord still works while hidden - the hook is global and does not care what is on screen - and _talk_start brings the window back before anything else, because a hold that showed nothing would be an open microphone with no way to tell. ctypes had to be told the signatures. Without them every call is assumed to return c_int and a 64-bit HWND does not fit: CreateWindowExW failed with int too long to convert, inside a thread nobody was watching. Also renames the gestures for the wording the owner asked for by name: push to talk is a thing people already know, and the label used to describe only the mechanics. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 22 +++- flow/tray.py | 308 +++++++++++++++++++++++++++++++++++++++++++++ flow/ui.py | 100 ++++++++++++++- tests/test_tray.py | 198 +++++++++++++++++++++++++++++ 4 files changed, 624 insertions(+), 4 deletions(-) create mode 100644 flow/tray.py create mode 100644 tests/test_tray.py diff --git a/docs/guide.md b/docs/guide.md index 5dde6ac..f3749f3 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -468,8 +468,8 @@ change takes effect on the next press, with no restart. | | | |---|---| -| **Hold to talk, release to send** | The shipped default. Press both keys, speak while they are down, let go. What you said is pasted into whatever window has focus — no third key, no clicking anything, no second shortcut to send. | -| **Press to start, press again to stop** | The original. A clean press-and-release starts listening and the next one stops it, exactly like the `toggle` hotkey. | +| **Push to talk - hold to speak, release to send** | The shipped default. Press both keys, speak while they are down, let go. What you said is pasted into whatever window has focus — no third key, no clicking anything, no second shortcut to send. | +| **Toggle - press to start, press again to stop** | The original. A clean press-and-release starts listening and the next one stops it, exactly like the `toggle` hotkey. | **Pick the hold for a sentence and the toggle for a paragraph.** The hold needs no decision about when you are finished and cannot leave a microphone running by accident. @@ -883,6 +883,24 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. +### Hiding it + +**Settings → Hide to tray** parks the window and leaves an icon in the Windows +notification area. The chord still works while hidden — it is a global hook and does not +care what is on screen — so dictating is unchanged and only the window goes. Pressing it +brings the overlay back for that utterance. + +Left-click the icon to show Flow; right-click it for **Show Flow** and **Quit Flow**. + +**It refuses to hide if the icon does not appear.** `Shell_NotifyIcon` can fail — a shell +still starting, a notification area that will not take another icon — and hiding anyway +would leave you with no window and nothing to click, reachable only through Task Manager. +So the icon is registered first and its answer is believed: no icon, no hiding, and the +bubble says why. + +Windows only. macOS has a menu bar item and Linux has whatever the desktop environment +offers, and neither is `Shell_NotifyIcon`, so the menu entry is simply absent there. + ### On a Mac Send pastes there too, as of this version. It used to copy and stop, which made every diff --git a/flow/tray.py b/flow/tray.py new file mode 100644 index 0000000..3b6caa2 --- /dev/null +++ b/flow/tray.py @@ -0,0 +1,308 @@ +"""A notification-area icon, so Flow can be out of the way without being lost. + +The need, in the owner's words: "there are times where i wanted to dictate but at the +same time i wanted to see but i don't want it to keep it on my screen". The overlay can +already be parked off the desktop — `ui.park` does it — and parking alone is a trap. A +Flow with no window and no icon is a process you cannot reach, cannot configure and +cannot quit except through Task Manager, which is invariant 4's problem wearing a +different hat: hidden must not mean gone. + +So hiding gets an icon, and the icon is the way back. + +**Win32 through ctypes, no new dependency** (R16 holds at three). `Shell_NotifyIconW` +needs a window to send its click messages to, and Tk will not give us one — its window +procedure is Tcl's, and subclassing it to intercept a custom message would put our code +on the path of every event Tk handles. So this creates its own **message-only window** +(`HWND_MESSAGE`), which has no pixels, never draws and exists purely to receive. + +**It runs its own message loop on its own thread**, for the same reason. `GetMessageW` +blocks, and Tk's loop is not ours to block. The two never touch: the window procedure +runs on this thread, and everything it learns is put on a `queue.Queue` that the UI +drains from its own frame pump. Nothing here calls into Tk, which is the rule that makes +threading here safe rather than merely tested. + +**Stock icon, deliberately.** `IDI_APPLICATION` rather than an `.ico` shipped in the +package: an icon file is a binary asset in a repository that has none, and a tray icon +that is obviously a placeholder is more honest than one that took a build step to look +official. It is a line to change when Flow has artwork. +""" + +import ctypes +import queue +import threading +from ctypes import wintypes + +#: What the icon puts on the queue. Strings rather than callbacks, because the callback +#: would then run on this module's thread — and the one rule here is that nothing this +#: file owns ever touches Tk. +SHOW = "show" +QUIT = "quit" + +#: The message the shell sends us for every click on the icon. `WM_APP` and above are +#: reserved for an application's own use, which is exactly what this is. +_WM_APP = 0x8000 +_WM_TRAY = _WM_APP + 1 + +#: Menu command ids. Any positive int the popup can return; they mean nothing outside it. +_ID_SHOW = 1 +_ID_QUIT = 2 + +_WM_DESTROY = 0x0002 +_WM_RBUTTONUP = 0x0205 +_WM_LBUTTONUP = 0x0202 +_WM_LBUTTONDBLCLK = 0x0203 +_WM_COMMAND = 0x0111 + +_NIM_ADD, _NIM_DELETE = 0x0, 0x2 +_NIF_MESSAGE, _NIF_ICON, _NIF_TIP = 0x1, 0x2, 0x4 +_IDI_APPLICATION = 32512 +_IMAGE_ICON = 1 +_LR_SHARED = 0x8000 +_HWND_MESSAGE = -3 +_MF_STRING = 0x0 +_TPM_RETURNCMD = 0x0100 +_TPM_RIGHTBUTTON = 0x0002 + +_LRESULT = ctypes.c_ssize_t +_WNDPROC = ctypes.WINFUNCTYPE( + _LRESULT, wintypes.HWND, wintypes.UINT, wintypes.WPARAM, wintypes.LPARAM +) + + +class _WNDCLASSEXW(ctypes.Structure): + _fields_ = [ + ("cbSize", wintypes.UINT), + ("style", wintypes.UINT), + ("lpfnWndProc", _WNDPROC), + ("cbClsExtra", ctypes.c_int), + ("cbWndExtra", ctypes.c_int), + ("hInstance", wintypes.HINSTANCE), + ("hIcon", wintypes.HICON), + ("hCursor", wintypes.HANDLE), + ("hbrBackground", wintypes.HBRUSH), + ("lpszMenuName", wintypes.LPCWSTR), + ("lpszClassName", wintypes.LPCWSTR), + ("hIconSm", wintypes.HICON), + ] + + +class _NOTIFYICONDATAW(ctypes.Structure): + """The shell's icon record. + + `szTip` is 128 wide characters in every version since Windows 2000 and `cbSize` is + how the shell knows which layout it is being handed — so it is `sizeof` this struct + and never a number typed in. + """ + + _fields_ = [ + ("cbSize", wintypes.DWORD), + ("hWnd", wintypes.HWND), + ("uID", wintypes.UINT), + ("uFlags", wintypes.UINT), + ("uCallbackMessage", wintypes.UINT), + ("hIcon", wintypes.HICON), + ("szTip", wintypes.WCHAR * 128), + ] + + +def _declare() -> None: + """Give ctypes the signatures, rather than letting it guess. + + Without this every call is assumed to return `c_int`, and a 64-bit `HWND` does not + fit in one: `CreateWindowExW` came back as `OverflowError: int too long to convert` + from inside the tray thread, where nothing was watching. The handle-returning calls + are the ones that matter, and the parent handle has to be a real `HWND` too — + `HWND_MESSAGE` is -3, which is only meaningful once ctypes knows it is a pointer. + """ + u = ctypes.windll.user32 + u.CreateWindowExW.restype = wintypes.HWND + u.CreateWindowExW.argtypes = [ + wintypes.DWORD, wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD, + ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, + wintypes.HWND, wintypes.HMENU, wintypes.HINSTANCE, wintypes.LPVOID, + ] + u.DefWindowProcW.restype = _LRESULT + u.DefWindowProcW.argtypes = [wintypes.HWND, wintypes.UINT, + wintypes.WPARAM, wintypes.LPARAM] + u.LoadImageW.restype = wintypes.HANDLE + u.LoadImageW.argtypes = [wintypes.HINSTANCE, wintypes.LPCWSTR, wintypes.UINT, + ctypes.c_int, ctypes.c_int, wintypes.UINT] + u.CreatePopupMenu.restype = wintypes.HMENU + u.TrackPopupMenu.restype = wintypes.BOOL + u.TrackPopupMenu.argtypes = [wintypes.HMENU, wintypes.UINT, ctypes.c_int, + ctypes.c_int, ctypes.c_int, wintypes.HWND, + wintypes.LPVOID] + u.DestroyWindow.argtypes = [wintypes.HWND] + ctypes.windll.kernel32.GetModuleHandleW.restype = wintypes.HINSTANCE + ctypes.windll.kernel32.GetModuleHandleW.argtypes = [wintypes.LPCWSTR] + + +class Tray: + """One notification-area icon, or nothing at all if the shell refuses. + + Every failure is reported rather than raised. A tray icon is a convenience on top of + a working app, and an app that will not start because the notification area was busy + would be a worse trade than an app with no icon — `start()` answers False and the + caller keeps the pill on screen, which is the state everybody had before this file. + """ + + def __init__(self, title: str = "Flow", events: queue.Queue | None = None) -> None: + self.title = title + #: What the icon's clicks arrive on. Owned by the caller when it passes one, so + #: a UI can drain this and its own events from the same place. + self.events: queue.Queue = events if events is not None else queue.Queue() + self.hwnd = 0 + self._thread: threading.Thread | None = None + self._ready = threading.Event() + self._ok = False + #: The window procedure, kept alive here on purpose. ctypes callbacks are garbage + #: like anything else, and one collected while Windows still holds its address is + #: an access violation in a thread nobody is watching. + self._proc = _WNDPROC(self._on_message) + self._class = f"FlowTray{id(self):x}" + + # -- the thread that owns the window ------------------------------------ + + def start(self) -> bool: + """Put the icon in the notification area. Returns whether it is actually there. + + Blocks until the answer is known — at most a moment, and worth waiting for: a + caller that hid its window on the strength of an icon that never appeared would + have hidden it for good. + """ + if self._thread is not None: + return self._ok + self._thread = threading.Thread(target=self._serve, name="flow-tray", + daemon=True) + self._thread.start() + self._ready.wait(timeout=5.0) + return self._ok + + def stop(self) -> None: + """Take the icon away. Safe to call twice, and safe to call if it never worked.""" + if not self.hwnd: + return + try: + _shell().Shell_NotifyIconW(_NIM_DELETE, ctypes.byref(self._icon_data())) + ctypes.windll.user32.DestroyWindow(self.hwnd) + except OSError: + pass + self.hwnd = 0 + + def _icon_data(self, with_icon: bool = False) -> _NOTIFYICONDATAW: + data = _NOTIFYICONDATAW() + data.cbSize = ctypes.sizeof(_NOTIFYICONDATAW) + data.hWnd = self.hwnd + data.uID = 1 + data.uFlags = _NIF_MESSAGE | _NIF_ICON | _NIF_TIP + data.uCallbackMessage = _WM_TRAY + if with_icon: + # `MAKEINTRESOURCE`: a stock icon is identified by an integer squeezed + # into a string pointer, which is what `LPCWSTR(id)` builds here. + data.hIcon = ctypes.windll.user32.LoadImageW( + None, wintypes.LPCWSTR(_IDI_APPLICATION), _IMAGE_ICON, 0, 0, + _LR_SHARED) + data.szTip = self.title + return data + + def _serve(self) -> None: + """Register, create, add the icon, then pump messages until the window dies.""" + try: + self._ok = self._build() + except OSError: + self._ok = False + finally: + self._ready.set() + if not self._ok: + return + msg = wintypes.MSG() + user32 = ctypes.windll.user32 + while user32.GetMessageW(ctypes.byref(msg), None, 0, 0) > 0: + user32.TranslateMessage(ctypes.byref(msg)) + user32.DispatchMessageW(ctypes.byref(msg)) + + def _build(self) -> bool: + _declare() + user32 = ctypes.windll.user32 + cls = _WNDCLASSEXW() + cls.cbSize = ctypes.sizeof(_WNDCLASSEXW) + cls.lpfnWndProc = self._proc + cls.hInstance = ctypes.windll.kernel32.GetModuleHandleW(None) + cls.lpszClassName = self._class + if not user32.RegisterClassExW(ctypes.byref(cls)): + return False + # `HWND_MESSAGE` as the parent: a window with no pixels, no place on the desktop + # and no chance of being shown by accident. It exists to be sent to. + self.hwnd = user32.CreateWindowExW( + 0, self._class, self.title, 0, 0, 0, 0, 0, + wintypes.HWND(_HWND_MESSAGE), None, cls.hInstance, None) + if not self.hwnd: + return False + return bool(_shell().Shell_NotifyIconW( + _NIM_ADD, ctypes.byref(self._icon_data(with_icon=True)))) + + # -- what the shell tells us -------------------------------------------- + + def _on_message(self, hwnd, message, wparam, lparam) -> int: + """The window procedure. Runs on this module's thread and never touches Tk. + + Everything it decides goes on the queue; the UI acts on it from its own loop. + """ + if message == _WM_TRAY: + event = lparam & 0xFFFF + if event in (_WM_LBUTTONUP, _WM_LBUTTONDBLCLK): + self.events.put(SHOW) + elif event == _WM_RBUTTONUP: + self._popup() + return 0 + if message == _WM_DESTROY: + ctypes.windll.user32.PostQuitMessage(0) + return 0 + return ctypes.windll.user32.DefWindowProcW( + wintypes.HWND(hwnd), wintypes.UINT(message), + wintypes.WPARAM(wparam), wintypes.LPARAM(lparam)) + + def _popup(self) -> None: + """The right-click menu: the two things a hidden app has to offer. + + `SetForegroundWindow` first, and the `PostMessage` after, are both from the + documented recipe: a popup owned by a window that is not foreground never + receives the click that dismisses it, and stays on screen until something else + is clicked. + """ + user32 = ctypes.windll.user32 + menu = user32.CreatePopupMenu() + if not menu: + return + try: + user32.AppendMenuW(menu, _MF_STRING, _ID_SHOW, "Show Flow") + user32.AppendMenuW(menu, _MF_STRING, _ID_QUIT, "Quit Flow") + point = wintypes.POINT() + user32.GetCursorPos(ctypes.byref(point)) + user32.SetForegroundWindow(self.hwnd) + chosen = user32.TrackPopupMenu( + menu, _TPM_RETURNCMD | _TPM_RIGHTBUTTON, + point.x, point.y, 0, self.hwnd, None) + user32.PostMessage(self.hwnd, 0, 0, 0) + finally: + user32.DestroyMenu(menu) + if chosen == _ID_SHOW: + self.events.put(SHOW) + elif chosen == _ID_QUIT: + self.events.put(QUIT) + + +def _shell(): + return ctypes.windll.shell32 + + +def available() -> bool: + """Whether this platform has a notification area at all. + + Windows-only by construction. macOS has a menu bar item and Linux has whatever the + desktop environment offers, and neither is `Shell_NotifyIcon` — so this says no + rather than pretending, and the caller keeps its window on screen. + """ + import sys + + return sys.platform == "win32" diff --git a/flow/ui.py b/flow/ui.py index 661acd7..fedc24b 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -15,6 +15,7 @@ import ctypes import math import os +import queue import sys import time import tkinter as tk @@ -40,6 +41,7 @@ ensure as ensure_lexicon, pairs, ) +from . import tray from .notes import Notes from .profile import path_key, resolve_workspace from .refine import EFFORT_DEFAULT, EFFORTS, available @@ -1182,9 +1184,18 @@ def _unload_fonts() -> None: #: What each chord gesture is called in the menu. Phrased as what it *does* rather #: than as its name, because "hold" and "toggle" are the words in the profile and this #: is the row somebody reads once while deciding — the whole sentence is the label. +#: What each gesture is called wherever a user reads it: the Settings menu, the note +#: after a switch, and the startup line. +#: +#: "Push to talk" by name, and the owner asked for it by name — "I think I like the +#: wording push to talk the default so it's more clear". The label used to describe the +#: mechanics only ("Hold to talk, release to send"), which is accurate and makes somebody +#: work out what it is. Push-to-talk is a thing people already know from a decade of +#: voice chat, so naming it does the explaining, and the mechanics still follow it for +#: anyone who has not met the term. GESTURE_LABELS = { - "hold": "Hold to talk, release to send", - "toggle": "Press to start, press again to stop", + "hold": "Push to talk - hold to speak, release to send", + "toggle": "Toggle - press to start, press again to stop", } PLACES = ("bottom", "corner") @@ -1627,6 +1638,10 @@ class Pill(tk.Tk): #: the idle pill this default describes, rather than recursing through `front`. _docked_w = PILL_W _shell_h = PILL_H + #: Whether the window is parked with an icon standing in for it. Class-level for the + #: reason `lite` is: a fixture built with `__new__` must not recurse into `self.tk`. + _hidden = False + _tray = None #: Same reason again, for `_draw`'s motion state (§07): a bare fixture draws the #: resting frame these describe — not hovered, not mid-collapse, opacity untouched. _pointer_in = False @@ -1754,6 +1769,10 @@ def __init__( #: panel is up. Compared in `_sync_shell` so a frame that changes nothing costs #: no `geometry` call. self._shell_h = PILL_H + #: What the notification-area icon puts its clicks on, drained in `_frame`. + #: Built here rather than with the icon, so `_drain_tray` has something to read + #: on every frame whether or not anybody has ever hidden the window. + self._tray_events: queue.Queue = queue.Queue() self.canvas = tk.Canvas( self, width=BUBBLE_W, height=PILL_H, bg=self.bg, highlightthickness=0 @@ -1939,6 +1958,12 @@ def _talk_start(self) -> None: mid-utterance would be indistinguishable from the user having spoken into a microphone that had just been reopened under them. """ + # Hidden Flow still hears the chord — the hook is global and does not care what + # is on screen — and a hold that showed nothing would be an open microphone with + # no way to tell it was open, which is what invariant 4 forbids. So the window + # comes back for the utterance, before anything else here decides not to run. + if self._hidden: + self.show_from_tray() if self._ptt_since is not None: return if getattr(self.session, "editing", False): @@ -2376,6 +2401,11 @@ def choice(label: str, cli) -> None: sub.add_cascade(label="Agent CLI", menu=picker) self._effort_menu(sub) self._model_menu(sub) + if tray.available(): + # Offered where the other once-and-forget settings are, and only where there + # is a notification area to hide into. `hide_to_tray` refuses rather than + # hides if the icon does not take, so this cannot strand anybody. + sub.add_command(label="Hide to tray", command=self.hide_to_tray) self._workspace_menu(sub) if getattr(self.session, "speaker", None) is not None: sub.add_command( @@ -3065,6 +3095,64 @@ def _clear(self) -> None: self.session.draft.clear() self.bubble.hide() + def hide_to_tray(self) -> bool: + """Put Flow out of the way, with an icon to bring it back. + + The need, in the owner's words: "there are times where i wanted to dictate but at + the same time i wanted to see but i don't want it to keep it on my screen". The + chord still works while hidden — it is a global hook and does not care what is on + screen — so dictating is unchanged and only the window goes. + + **The icon comes first, and hiding is conditional on it.** A window parked off the + desktop with nothing in the notification area is a Flow that cannot be reached, + configured or quit except through Task Manager. So `Tray.start()` is asked first + and its answer is believed: no icon, no hiding, and a note saying so. Invariant 4 + in a new place — hidden must not mean gone. + """ + if not tray.available(): + self.front.note("hiding needs the Windows notification area") + return False + if self._tray is None: + self._tray = tray.Tray("Flow - press the chord to talk", self._tray_events) + if not self._tray.start(): + self._flash = FLASH_FRAMES + self.front.note("the notification area would not take an icon") + return False + self._hidden = True + park(self) + return True + + def show_from_tray(self) -> None: + """Bring the window back where the user left it. + + The icon stays. Somebody who hid Flow once will hide it again, and an icon that + vanished on the first click would make the second one a trip through the menus. + """ + if not self._hidden: + return + self._hidden = False + self._sync_shell() + self.deiconify() + self.lift() + + def _drain_tray(self) -> None: + """What the icon decided, acted on from Tk's own thread. + + `tray.Tray` runs its window procedure on a thread of its own and puts strings on + a queue rather than calling back, precisely so this is the only place Tk is + touched — see that module's docstring for why that rule is the whole of the + threading argument here. + """ + while True: + try: + event = self._tray_events.get_nowait() + except queue.Empty: + return + if event == tray.SHOW: + self.show_from_tray() + elif event == tray.QUIT: + self.quit_app() + def quit_app(self) -> None: # Idempotent, because ctrl+C reaches here down either of two paths and nothing # upstream can tell which one ran: caught in `_tick`, or escaping `mainloop` and @@ -3077,6 +3165,11 @@ def quit_app(self) -> None: # re-arm itself against a destroyed interpreter on its way out. self._alive = False try: + # Before the hotkeys and before the window: an icon outliving its process is + # a ghost in the notification area that only a hover clears, and the shell + # gives no second chance to remove one whose window has already gone. + if self._tray is not None: + self._tray.stop() if self.hotkeys is not None: self.hotkeys.stop() self.session.close() @@ -3253,6 +3346,9 @@ def _frame(self) -> None: self._track_target() self._sync_monitor() + # Same rule as the hotkeys below: another thread decided, this one acts. + self._drain_tray() + # Hotkeys arrive on their own thread; Tk is only ever touched from this one. if self.hotkeys is not None: for name in self.hotkeys.drain(): diff --git a/tests/test_tray.py b/tests/test_tray.py new file mode 100644 index 0000000..b27c2dd --- /dev/null +++ b/tests/test_tray.py @@ -0,0 +1,198 @@ +"""Hiding Flow without losing it. + +The need, in the owner's words: "there are times where i wanted to dictate but at the +same time i wanted to see but i don't want it to keep it on my screen". Parking the +window is the easy half and the dangerous one — a Flow with no window and no icon is a +process that cannot be reached, configured or quit except through Task Manager. + +So the property under test is not "it hides". It is **that it refuses to hide unless +there is a way back**, and that the way back works. + +No window is created and no icon is registered here. `flow.tray` is a thin ctypes wrapper +around `Shell_NotifyIconW`, verified against the real shell by hand; what these check is +the decision Flow makes around it. +""" + +import queue +import sys +import unittest +from unittest import mock + +import flow.tray as tray +import flow.ui as ui + + +def pill(**kw): + """A pill with just enough of one to hide and come back.""" + p = ui.Pill.__new__(ui.Pill) + # `front` is a read-only property choosing bubble-or-card by mode, so the stand-in + # goes on the thing it chooses rather than over the property itself. + p.session = mock.Mock(mode=ui.DICTATE) + p.bubble = mock.Mock() + p.card = mock.Mock() + p._tray_events = queue.Queue() + p._tray = None + p._hidden = False + p._flash = 0 + p._ptt_since = None + p.park = mock.Mock() + p.deiconify = mock.Mock() + p.lift = mock.Mock() + p._sync_shell = mock.Mock() + p.quit_app = mock.Mock() + for name, value in kw.items(): + setattr(p, name, value) + return p + + +class TestHidingNeedsAWayBack(unittest.TestCase): + def test_it_hides_once_the_icon_is_actually_there(self): + p = pill() + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon), \ + mock.patch.object(ui, "park") as parked: + self.assertTrue(p.hide_to_tray()) + self.assertTrue(p._hidden) + parked.assert_called_once_with(p) + + def test_an_icon_that_would_not_register_leaves_the_window_alone(self): + """The one that matters. `Shell_NotifyIcon` can fail — a shell that is still + starting, a notification area that is full — and hiding anyway would strand the + user with no window and nothing to click.""" + p = pill() + icon = mock.Mock(**{"start.return_value": False}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon), \ + mock.patch.object(ui, "park") as parked: + self.assertFalse(p.hide_to_tray()) + self.assertFalse(p._hidden) + parked.assert_not_called() + self.assertIn("would not take", p.front.note.call_args.args[0]) + + def test_a_platform_with_no_notification_area_says_so(self): + # macOS has a menu bar item and Linux has whatever the desktop offers; neither + # is `Shell_NotifyIcon`, and pretending otherwise would hide a window for good. + p = pill() + with mock.patch.object(tray, "available", return_value=False), \ + mock.patch.object(ui, "park") as parked: + self.assertFalse(p.hide_to_tray()) + self.assertFalse(p._hidden) + parked.assert_not_called() + + def test_the_icon_is_built_once_and_reused(self): + # Somebody who hides Flow once will hide it again, and a second `Shell_NotifyIcon` + # for the same app is a second icon in the tray. + p = pill() + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), \ + mock.patch.object(tray, "Tray", return_value=icon) as made, \ + mock.patch.object(ui, "park"): + p.hide_to_tray() + p.show_from_tray() + p.hide_to_tray() + made.assert_called_once() + + +class TestComingBack(unittest.TestCase): + def test_showing_puts_the_window_where_it_was(self): + p = pill(_hidden=True) + p.show_from_tray() + self.assertFalse(p._hidden) + p._sync_shell.assert_called_once() + p.deiconify.assert_called_once() + + def test_showing_a_window_that_is_already_up_does_nothing(self): + p = pill(_hidden=False) + p.show_from_tray() + p.deiconify.assert_not_called() + + def test_the_chord_brings_it_back_before_it_opens_the_microphone(self): + """A hold that showed nothing would be an open microphone with no way to tell + it was open — which is invariant 4 read from the other side.""" + p = pill(_hidden=True) + p.session = mock.Mock() + p.session.talk_start.side_effect = lambda *a, **k: None + with mock.patch.object(ui.Pill, "show_from_tray", + autospec=True) as shown, \ + mock.patch.object(ui.Pill, "_pump_talk", autospec=True): + try: + ui.Pill._talk_start(p) + except Exception: + pass + shown.assert_called_once() + + +class TestWhatTheIconSaysArrivesOnTheUIThread(unittest.TestCase): + """`tray.Tray` runs its window procedure on a thread of its own and puts *strings* + on a queue rather than calling back. That is the whole of the threading argument: + Tk is touched from one place, `_frame`, and nothing in `flow/tray.py` touches it.""" + + def test_a_show_click_shows(self): + p = pill(_hidden=True) + p._tray_events.put(tray.SHOW) + p._drain_tray() + self.assertFalse(p._hidden) + + def test_a_quit_click_quits(self): + p = pill() + p._tray_events.put(tray.QUIT) + p._drain_tray() + p.quit_app.assert_called_once() + + def test_an_empty_queue_is_the_ordinary_case_and_costs_nothing(self): + p = pill() + p._drain_tray() + p.quit_app.assert_not_called() + + def test_everything_waiting_is_taken_in_one_pass(self): + # The queue is drained, not sampled: a frame that took one event and left the + # rest would answer a click a frame late for every click before it. + p = pill(_hidden=True) + for _ in range(3): + p._tray_events.put(tray.SHOW) + p._drain_tray() + self.assertTrue(p._tray_events.empty()) + + +class TestTheModule(unittest.TestCase): + def test_it_is_windows_only_and_says_so(self): + for platform, expected in (("win32", True), ("darwin", False), ("linux", False)): + with self.subTest(platform=platform): + with mock.patch.object(sys, "platform", platform): + self.assertIs(tray.available(), expected) + + def test_stopping_an_icon_that_never_started_is_safe(self): + # `quit_app` calls this unconditionally, including after a `hide_to_tray` that + # was refused. + icon = tray.Tray("probe") + icon.stop() + self.assertEqual(icon.hwnd, 0) + + def test_starting_twice_returns_the_first_answer(self): + icon = tray.Tray("probe") + icon._thread = mock.Mock() + icon._ok = True + self.assertTrue(icon.start()) + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: Shell_NotifyIconW") + def test_the_struct_is_the_size_the_shell_expects(self): + # `cbSize` is how the shell knows which layout it has been handed, and it is + # `sizeof` rather than a number typed in — this asserts the field is wired to + # the struct at all. + icon = tray.Tray("probe") + import ctypes + + self.assertEqual(icon._icon_data().cbSize, + ctypes.sizeof(tray._NOTIFYICONDATAW)) + + def test_the_window_procedure_is_held_alive(self): + """A ctypes callback is garbage like anything else, and one collected while + Windows still holds its address is an access violation on a thread nobody is + watching.""" + icon = tray.Tray("probe") + self.assertIsNotNone(icon._proc) + + +if __name__ == "__main__": # pragma: no cover + unittest.main() From becd8dd769462190a78c856d40ce8099d6ffe0c9 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 13:26:56 -0500 Subject: [PATCH 29/41] Fix two regressions the merge introduced: the pill would not move, and would not hide Reported: "I'm not able to move this around on my screen ... it stays on the center always", and "when I try to click on hide to tray it did not hide". Both are mine, from merging the two windows. _sync_shell asked _placed() for x and for the foot on every frame, so a drag was undone before the hand had left the mouse. _sync_dock never had the fault because it only recomputed x when the width changed, which was rare; recomputing unconditionally is what the merge introduced. It now keeps where the window *is* and clamps it, and _placed stays the answer at startup and on a monitor change, which is where _sync_monitor asks it. Hiding failed twice over. park(self) reads win.width, which the panels have and the pill did not, so the call went to tk.Misc.__getattr__ and looked for a Tcl command. And even parked, _sync_shell put the window straight back on the next frame - it runs thirty times a second and won every one of them. Hiding now remembers where the window was, as (x, foot) rather than (x, y): the shell is anchored by its bottom edge, so a panel that was open at hide time and closed by the time it comes back must not move the controls. Verified on the running app: a dragged x of 40 survives every subsequent frame. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 39 +++++++++++++++++++++++++++-- tests/test_tray.py | 61 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index fedc24b..df9af40 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1642,6 +1642,10 @@ class Pill(tk.Tk): #: reason `lite` is: a fixture built with `__new__` must not recurse into `self.tk`. _hidden = False _tray = None + #: Where the window was when it was hidden, as (x, foot). Restored on the way back, + #: because somebody who dragged Flow to the left of their screen did not ask for it + #: to reappear in the middle. + _home = None #: Same reason again, for `_draw`'s motion state (§07): a bare fixture draws the #: resting frame these describe — not hovered, not mid-collapse, opacity untouched. _pointer_in = False @@ -3118,6 +3122,9 @@ def hide_to_tray(self) -> bool: self._flash = FLASH_FRAMES self.front.note("the notification area would not take an icon") return False + # Where it was, as (x, foot) — the foot rather than the top, because that is + # the edge the shell is anchored by and the one a reopened panel measures from. + self._home = (self.x, self.y + self._shell_h) self._hidden = True park(self) return True @@ -3131,6 +3138,9 @@ def show_from_tray(self) -> None: if not self._hidden: return self._hidden = False + if self._home is not None: + x, foot = self._home + self.x, self.y = x, foot - self._shell_h self._sync_shell() self.deiconify() self.lift() @@ -3224,6 +3234,17 @@ def _tick(self) -> None: if self._alive: self.after(30, self._tick) + @property + def width(self) -> int: + """This window's width, under the name `park` and the panels both use. + + The pill had no `width` while the panels did, so `park(self)` — the call that + hides this window — reached `tk.Misc.__getattr__` and went looking for a Tcl + command. "Hide to tray" did nothing at all, twice over: this, and a `_sync_shell` + that put the window straight back. + """ + return self.pill_w + def band_h(self) -> int: """How tall the panel band is: `PANEL_H`, unless the desktop is smaller. @@ -3755,13 +3776,27 @@ def _sync_shell(self) -> None: # here made the shell 224 px tall around a 130 px band and left the row floating # 54 px below it — which the shots caught immediately, because a detached row is # exactly the two-boxes look this window was merged to end. + # Parked, with an icon standing in for it. Re-asserting geometry here is what + # dragged it straight back on screen the moment it was hidden — the frame pump + # runs thirty times a second and this used to win every one of them. + if self._hidden: + return front = self.front band = min(self.band_h(), max(0, int(getattr(front, "_h", 0) or 0))) if getattr(front, "_visible", False) else 0 h = PILL_H + band w = self.pill_w left, top, right, bottom = self.work - foot = self._placed(w)[1] + PILL_H # where the pill row's bottom edge belongs - x = max(left, min(self._placed(w)[0], right - w)) + # **Where it is, not where it belongs.** This asked `_placed` for both, on every + # frame, which meant a drag was undone before the hand had left the mouse: the + # pill snapped back to centre and could not be moved at all. `_sync_dock` never + # had the fault because it only recomputed x when the *width* changed, which was + # rare; recomputing unconditionally is what the merge introduced. + # + # `_placed` is still the answer at startup and whenever the pointer changes + # monitor — `_sync_monitor` asks it there, which is the one place a reposition is + # actually wanted. + foot = self.y + self._shell_h + x = max(left, min(self.x, right - w)) y = max(top + EDGE_AIR, min(foot - h, bottom - h)) if (self.x, self.y, self._shell_h) != (x, y, h): self.x, self.y, self._shell_h = x, y, h diff --git a/tests/test_tray.py b/tests/test_tray.py index b27c2dd..621230d 100644 --- a/tests/test_tray.py +++ b/tests/test_tray.py @@ -30,6 +30,11 @@ def pill(**kw): p.session = mock.Mock(mode=ui.DICTATE) p.bubble = mock.Mock() p.card = mock.Mock() + # Real ints: hiding remembers where the window was, and `tk.Misc.__getattr__` turns + # a missing one into a Tcl lookup rather than an AttributeError. + p.x, p.y = 430, 608 + p._shell_h = ui.PILL_H + p._home = None p._tray_events = queue.Queue() p._tray = None p._hidden = False @@ -196,3 +201,59 @@ def test_the_window_procedure_is_held_alive(self): if __name__ == "__main__": # pragma: no cover unittest.main() + +class TestItComesBackWhereYouLeftIt(unittest.TestCase): + """Somebody who dragged Flow to the left of their screen did not ask for it to + reappear in the middle.""" + + def hidden(self): + p = pill(x=40, y=600) + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), mock.patch.object(tray, "Tray", return_value=icon), mock.patch.object(ui, "park"): + p.hide_to_tray() + return p + + def test_the_position_is_remembered_across_the_hide(self): + p = self.hidden() + self.assertEqual(p._home, (40, 600 + ui.PILL_H)) + + def test_and_restored_on_the_way_back(self): + p = self.hidden() + p.x, p.y = 9999, 9999 # where `park` left it + p.show_from_tray() + self.assertEqual((p.x, p.y), (40, 600)) + + def test_the_foot_is_what_is_remembered_not_the_top(self): + # The shell is anchored by its bottom edge, so a panel that was open when it was + # hidden — and closed by the time it comes back — must not move the controls. + p = pill(x=40, y=500, _shell_h=ui.PILL_H + 140) + icon = mock.Mock(**{"start.return_value": True}) + with mock.patch.object(tray, "available", return_value=True), mock.patch.object(tray, "Tray", return_value=icon), mock.patch.object(ui, "park"): + p.hide_to_tray() + foot = 500 + ui.PILL_H + 140 + p._shell_h = ui.PILL_H + p.show_from_tray() + self.assertEqual(p.y + p._shell_h, foot) + + +class TestTheFramePumpLeavesAHiddenWindowAlone(unittest.TestCase): + """The second half of why "Hide to tray" did nothing. + + `_sync_shell` re-asserts the window's geometry, and it runs thirty times a second. It + used to win every one of them, so a parked window was back on screen before the next + frame had drawn. + """ + + def test_sync_shell_returns_immediately_while_hidden(self): + p = pill(_hidden=True) + p.geometry = mock.Mock() + p.canvas = mock.Mock() + ui.Pill._sync_shell(p) + p.geometry.assert_not_called() + + def test_the_pill_has_a_width_under_the_name_park_uses(self): + # `park(self)` reads `win.width`. The panels had one and the pill did not, so the + # call went to `tk.Misc.__getattr__` and looked for a Tcl command. + p = pill() + self.assertEqual(p.width, ui.BUBBLE_W) + From 05192dd47589cbf1df5cc5ec6a0d89a2d0713fc3 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:00:25 -0500 Subject: [PATCH 30/41] Take the pill row with the panel when the panel size changes Two screenshots of "panel size: larger" showed a wide band above a row that had kept its old width - two boxes of different widths stacked in one window, which is the exact look merging them was meant to end. apply_panel_width rebinds BUBBLE_W while Flow is running, so the width changes with nothing else changing beside it. _sync_shell compared x, y and the height and not the width, so the canvas was never re-placed and _docked_w - which is what _draw measures the row against - kept the value it was built with. The old _sync_dock compared the width and nothing else, so the merge managed to lose the one comparison that mattered here. Verified on the running app across regular, large, larger and back: window, row, band and _docked_w now agree at 420, 520, 640, 420. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 12 ++++++++++-- tests/test_pill.py | 22 ++++++++++++++++++++++ 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index df9af40..d9c61e7 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -3798,8 +3798,16 @@ def _sync_shell(self) -> None: foot = self.y + self._shell_h x = max(left, min(self.x, right - w)) y = max(top + EDGE_AIR, min(foot - h, bottom - h)) - if (self.x, self.y, self._shell_h) != (x, y, h): - self.x, self.y, self._shell_h = x, y, h + # The width is compared too, and leaving it out was a defect. `apply_panel_width` + # rebinds `BUBBLE_W` while Flow is running — the panel-size setting — so `w` + # changes without x, y or the height changing with it. The row then kept the + # width it was built at while the band above it took the new one, which is two + # boxes of different widths stacked in one window, and exactly what a screenshot + # of "panel size: larger" showed. `_docked_w` is what `_draw` measures the row + # against, so it has to move in the same breath as the canvas. + if (self.x, self.y, self._shell_h, self._docked_w) != (x, y, h, w): + self.x, self.y, self._shell_h, self._docked_w = x, y, h, w + self.canvas.configure(width=w) self.canvas.place(x=0, y=h - PILL_H, width=w, height=PILL_H) if self.window_geometry() != (w, self.x, self.y): self.geometry(f"{w}x{h}+{self.x}+{self.y}") diff --git a/tests/test_pill.py b/tests/test_pill.py index 193a665..78c6076 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -1272,6 +1272,28 @@ def test_the_shell_is_clamped_onto_the_screen(self): self.assertGreaterEqual(p.y, 0) self.assertLessEqual(p.y + p._shell_h, 200) + def test_a_panel_size_change_takes_the_row_with_it(self): + """The panel-size setting rebinds `BUBBLE_W` while Flow is running, so the width + changes with nothing else changing beside it. + + Left out of the comparison, the row kept the width it was built at while the band + above it took the new one — two boxes of different widths stacked in one window, + which is what a screenshot of "panel size: larger" showed. `_docked_w` is what + `_draw` measures the row against, so it moves in the same breath as the canvas. + """ + self.addCleanup(ui.apply_panel_width, ui.PANEL_WIDTHS["regular"]) + p = docker(showing=False, x=430) + p._sync_shell() + p.geometry.reset_mock() + ui.apply_panel_width(ui.PANEL_WIDTHS["larger"]) + p.window_geometry = mock.Mock(return_value=(ui.PANEL_WIDTHS["regular"], + p.x, p.y)) + p._sync_shell() + self.assertEqual(p._docked_w, ui.PANEL_WIDTHS["larger"]) + self.assertEqual(p.canvas.place.call_args.kwargs["width"], + ui.PANEL_WIDTHS["larger"]) + self.assertIn(str(ui.PANEL_WIDTHS["larger"]), p.geometry.call_args.args[0]) + def test_the_row_is_placed_at_the_foot_of_whatever_height_it_is(self): # The canvas is the bottom band of the window, not the whole of it — which is # what makes "the foot never moves" true of the pixels and not just of the frame. From 29235004dffe49fcf880eabbcec7a38fcf335bb1 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:05:21 -0500 Subject: [PATCH 31/41] Call PostMessageW, because there is no PostMessage Right-clicking the tray icon showed the menu and then did nothing. user32 exports PostMessageA and PostMessageW and no bare PostMessage - the undecorated name is a C macro - so ctypes raised AttributeError: function 'PostMessage' not found, from inside the tray thread, after the menu had been chosen from and before anything acted on the choice. Nothing in the suite could have caught it, because nothing named the function until somebody clicked. So the suite names them all now: a test walks tray.py's own source for every user32 call it makes and asks Windows whether each one is really there. Sixteen symbols, all present, and it would have failed the moment the typo was written. The click handler also stops anything escaping into Windows. It called us; an exception unwinding into its stack is undefined at best, and a tray that quietly stops answering is the failure this file exists to prevent - so it is caught and printed. Co-Authored-By: Claude Opus 5 --- flow/tray.py | 27 ++++++++++++++++++++------- tests/test_tray.py | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 7 deletions(-) diff --git a/flow/tray.py b/flow/tray.py index 3b6caa2..2906f97 100644 --- a/flow/tray.py +++ b/flow/tray.py @@ -29,6 +29,7 @@ import ctypes import queue +import sys import threading from ctypes import wintypes @@ -133,6 +134,8 @@ def _declare() -> None: ctypes.c_int, ctypes.c_int, wintypes.HWND, wintypes.LPVOID] u.DestroyWindow.argtypes = [wintypes.HWND] + u.PostMessageW.argtypes = [wintypes.HWND, wintypes.UINT, + wintypes.WPARAM, wintypes.LPARAM] ctypes.windll.kernel32.GetModuleHandleW.restype = wintypes.HINSTANCE ctypes.windll.kernel32.GetModuleHandleW.argtypes = [wintypes.LPCWSTR] @@ -250,10 +253,17 @@ def _on_message(self, hwnd, message, wparam, lparam) -> int: """ if message == _WM_TRAY: event = lparam & 0xFFFF - if event in (_WM_LBUTTONUP, _WM_LBUTTONDBLCLK): - self.events.put(SHOW) - elif event == _WM_RBUTTONUP: - self._popup() + try: + if event in (_WM_LBUTTONUP, _WM_LBUTTONDBLCLK): + self.events.put(SHOW) + elif event == _WM_RBUTTONUP: + self._popup() + except Exception as exc: # pragma: no cover - belt for a callback + # Nothing may escape a ctypes callback: Windows called us, and an + # exception unwinding into its stack is undefined at best. Printed + # rather than swallowed, because a tray that silently stops answering + # is the failure this file exists to prevent. + print(f"flow: tray click failed: {exc}", file=sys.stderr, flush=True) return 0 if message == _WM_DESTROY: ctypes.windll.user32.PostQuitMessage(0) @@ -283,7 +293,12 @@ def _popup(self) -> None: chosen = user32.TrackPopupMenu( menu, _TPM_RETURNCMD | _TPM_RIGHTBUTTON, point.x, point.y, 0, self.hwnd, None) - user32.PostMessage(self.hwnd, 0, 0, 0) + # `PostMessageW`, with the W. There is no bare `PostMessage` export in + # user32 — the name is a macro in C that resolves to one of the two — so + # asking for it raised `AttributeError: function 'PostMessage' not found` + # *after* the menu had been chosen from and before anything acted on the + # choice. Right-clicking the icon showed the menu and then did nothing. + user32.PostMessageW(self.hwnd, 0, 0, 0) finally: user32.DestroyMenu(menu) if chosen == _ID_SHOW: @@ -303,6 +318,4 @@ def available() -> bool: desktop environment offers, and neither is `Shell_NotifyIcon` — so this says no rather than pretending, and the caller keeps its window on screen. """ - import sys - return sys.platform == "win32" diff --git a/tests/test_tray.py b/tests/test_tray.py index 621230d..3f2fb2b 100644 --- a/tests/test_tray.py +++ b/tests/test_tray.py @@ -257,3 +257,45 @@ def test_the_pill_has_a_width_under_the_name_park_uses(self): p = pill() self.assertEqual(p.width, ui.BUBBLE_W) +class TestEveryWin32NameResolves(unittest.TestCase): + """The bug this class exists for: `user32.PostMessage` does not exist. + + There is no bare `PostMessage` export — the name is a C macro resolving to the A or + W variant — so `ctypes` raised `AttributeError: function 'PostMessage' not found`. + It raised *at click time*, inside the tray thread, after the menu had been chosen + from and before anything acted on the choice: right-clicking the icon showed a menu + that then did nothing. Nothing in the test suite could have caught it, because + nothing named the function until somebody clicked. + + So the suite names them all. This walks the module's own source for every `user32` + call it makes and asks Windows whether each one is really there — a spelling check + that costs nothing and would have failed the moment the typo was written. + """ + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: user32") + def test_every_user32_function_the_module_names_exists(self): + import ctypes + import pathlib + import re + + source = pathlib.Path(tray.__file__).read_text(encoding="utf-8") + named = {a or b for a, b in + re.findall(r"user32\.(\w+)|u\.(\w+)\.", source)} + self.assertIn("PostMessageW", named, "the source no longer calls what it did") + user32 = ctypes.windll.user32 + missing = sorted(n for n in named if not hasattr(user32, n)) + self.assertEqual(missing, [], f"user32 has no {missing}") + + @unittest.skipUnless(sys.platform == "win32", "Windows-only: shell32") + def test_the_shell_call_exists_too(self): + self.assertTrue(hasattr(tray._shell(), "Shell_NotifyIconW")) + + def test_a_click_that_raises_does_not_escape_into_windows(self): + """Windows called us. An exception unwinding into its stack is undefined at + best, and a tray that silently stops answering is the failure this file exists + to prevent — so it is caught, and said out loud.""" + icon = tray.Tray("probe") + with mock.patch.object(icon, "_popup", side_effect=RuntimeError("boom")): + self.assertEqual( + icon._on_message(0, tray._WM_TRAY, 0, tray._WM_RBUTTONUP), 0) + From 6bca6f350eb8d64b07b9648ba642a6884f5f54b7 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:09:41 -0500 Subject: [PATCH 32/41] Load the model when Flow opens, not when somebody first speaks "loading the model" used to be the first thing a fresh Flow said back - in the bubble, while the user was already talking. The load lands inside that first utterance rather than in front of it: first partial 1 230 ms against ~570 ms for the four behind it. The chord's press-down has warmed the models since push-to-talk shipped, which covers the second use and never the first. warm() is called after the pill is built and before the loop runs, so the window is on screen while the disk does its work rather than after it. It is single-flight and loads on a thread of its own, so nothing at startup waits on it - measured at 0 ms to return, with the model loaded 1.6 s later. --no-warm keeps the old behaviour, for a launcher that starts with the machine, where paying a model load at login is the wrong trade, and for measuring the cold path. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 1 + flow/__main__.py | 18 ++++++++++++++++++ tests/test_main.py | 26 ++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/docs/guide.md b/docs/guide.md index f3749f3..4b155dc 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -265,6 +265,7 @@ click the pill to arm | right-click for the menu | ctrl+alt+Q quits | `--voice X` | voice for spoken replies: a name, part of one, or `male`/`female` | | `--no-auto-ask` | in converse mode, wait for the Ask button instead of a pause | | `--cli NAME` | pin the agent CLI (`codex` or `claude`) instead of trying each in turn | +| `--no-warm` | do not load the model at startup; wait until it is first needed | | `--cli-model NAME` | ask the CLI for this model; remembered, and blank clears it | | `--cli-effort LEVEL` | how hard the CLI may think: `low`, `medium`, `high`, `xhigh`, `max` (default `low`) | | `--cli-timeout SEC` | how long to wait for one CLI call (default 20) | diff --git a/flow/__main__.py b/flow/__main__.py index 45c3fb5..077919f 100644 --- a/flow/__main__.py +++ b/flow/__main__.py @@ -341,6 +341,12 @@ def main(argv: list[str] | None = None) -> int: help="pin the agent CLI instead of trying each in turn " f"({', '.join(c.name for c in CANDIDATES if c.verified)})", ) + ap.add_argument( + "--no-warm", action="store_true", + # For a launcher that starts with the machine, where paying a model load at login + # is the wrong trade — and for measuring the cold path on purpose. + help="do not load the model at startup; wait until it is first needed", + ) ap.add_argument( "--cli-model", default=None, metavar="NAME", # The only way a model name gets into Flow: the settings menu has no text field @@ -787,6 +793,18 @@ def on_send(text: str, target: int | None = None, submit: bool = False) -> str: settings_path=DEFAULT_PATH if args.no_lexicon else lexicon.path, lite=lite, ) + # **Loaded now, not at the first word.** "loading the model" used to be the first + # thing a fresh Flow said back, in the bubble, while somebody was already speaking — + # and the load lands *inside* that first utterance rather than in front of it, so the + # first partial measured 1 230 ms against ~570 ms for the four behind it. The chord's + # press-down has warmed the models since push-to-talk shipped, which covers the + # second use and not the first. + # + # After the pill is built and before the loop runs, so the window is on screen while + # the disk does its work rather than after it. `warm()` returns immediately — it is + # single-flight and does its loading on a thread of its own — so nothing here waits. + if not args.no_warm: + session.warm() try: pill.mainloop() except KeyboardInterrupt: diff --git a/tests/test_main.py b/tests/test_main.py index 89e70c2..f2b4946 100644 --- a/tests/test_main.py +++ b/tests/test_main.py @@ -526,3 +526,29 @@ def test_and_an_unusable_block_is_not_named_under_that_flag_either(self): if __name__ == "__main__": unittest.main() + +class TestTheModelIsLoadedBeforeItIsAskedFor(unittest.TestCase): + """"loading the model" used to be the first thing a fresh Flow said back. + + It said it in the bubble, while somebody was already speaking, because the load lands + *inside* the first utterance rather than in front of it — first partial 1 230 ms + against ~570 ms for the four behind it. The chord's press-down has warmed the models + since push-to-talk shipped, which covers the second use and not the first. + """ + + def test_startup_warms_the_session(self): + _code, _out, _pill, session = launch("win32") + session.return_value.warm.assert_called_once() + + def test_it_happens_off_windows_too(self): + # The load is the same load and the wait is the same wait; nothing about it is + # platform-shaped. + _code, _out, _pill, session = launch("darwin") + session.return_value.warm.assert_called_once() + + def test_no_warm_leaves_it_for_the_first_word(self): + # For a launcher that starts with the machine, where paying a model load at + # login is the wrong trade — and for measuring the cold path on purpose. + _code, _out, _pill, session = launch("win32", ["--no-warm"]) + session.return_value.warm.assert_not_called() + From ca63590aafdc04dcf3083699a5dd3e0df16f3cf2 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Mon, 31 Aug 2026 16:10:02 -0500 Subject: [PATCH 33/41] Put the settings that matter above the draft, not behind a right-click Asked for by name - "Dictate and Converse for sure Then workspace and voices" - with the choice of when to show it left to me. With the panel, never at rest. Those three only mean anything once there is something to send, and an always-on strip would cost 22 px of the idle row, which is the part of this surface everybody has said they like small. FluidVoice does not pay it either: its Dictate / AI Prompt / Actions bar belongs to the app being dictated into, not to the overlay. Mode is a control and the other two are values, which is the whole design. Mode changes what Send does and is switched mid-task, so it is a chip that acts. Workspace and voice are worth *seeing* - which project Ask runs in, without opening anything - and clicking either opens the list the right-click menu already builds, tick and all, rather than growing a second implementation of it. The strip is furniture, so PANEL_MAX_H grew by SETTINGS_H rather than the draft paying for it. Taking it out of the content's share is what the editor tests caught: the live partial's cap is a flat 70 px, and on a panel pegged at the old ceiling it ran straight through the note and the chip row because everything above it had grown by 22 and it had not been told. One test relaxed from assertGreater to assertGreaterEqual, honestly: _settled_h steps by a whole body line, so a note that grew by 14 px can land in the same bucket. That is what the snap is for, and the property that test is really about - the note drawn whole, and clear of the chips - is asserted beside it. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 18 ++++++ flow/ui.py | 132 ++++++++++++++++++++++++++++++++++++++++++- tests/test_bubble.py | 88 +++++++++++++++++++++++++++++ tests/test_editor.py | 8 ++- 4 files changed, 242 insertions(+), 4 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index 4b155dc..91b4f54 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -884,6 +884,24 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. +### The strip above the draft + +Three things sit above the words whenever a panel is up, so the settings that matter +mid-task do not cost a right-click: + +**Dictate ⌄** / **Converse ⌄** — a chip, and clicking it switches. This is the one that +changes what Send does, and it is the one people switch mid-task. + +**workshop:** and **voice:** — values rather than controls. Their worth is being visible +— knowing which project Ask is running in without opening anything — and clicking either +opens the same list the right-click menu has, tick and all. The workspace is shown by the +last part of its path, because that is the part that names the project. + +**It appears with the panel and never at rest.** Those three only mean anything once +there is something to send, and an always-on strip would cost 22 px of the idle row, +which is the part of this surface worth keeping small. FluidVoice does not pay it either +— its bar belongs to the app being dictated into, not to the overlay. + ### Hiding it **Settings → Hide to tray** parks the window and leaves an icon in the Windows diff --git a/flow/ui.py b/flow/ui.py index d9c61e7..6fdffc4 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1097,7 +1097,29 @@ def _unload_fonts() -> None: #: whole body lines (`_settled_h`): a height that can only change when the text gains or #: loses a *line* changes a handful of times an utterance, and a timer that has to be #: cancelled correctly from a render loop is a thing to get wrong. -PANEL_MAX_H = 184 +#: The settings strip's own height, including the air under it. +#: +#: **It appears with the panel and never at rest.** The owner asked for the settings that +#: matter to be reachable without a right-click — "Dictate and Converse for sure Then +#: workspace and voices" — and the choice of *when* was left to me. With a draft up is +#: the answer: those three only mean anything once there is something to send, and an +#: always-on strip would cost 22 px of the idle row, which is the one part of this +#: surface everybody has said they like small. +#: +#: FluidVoice does not pay this either — its `Dictate / AI Prompt / Actions` bar belongs +#: to the app being dictated into, not to the overlay. +SETTINGS_H = 22 + +#: How far apart the read-only values sit from each other. +SETTINGS_GAP = 14 + +#: 184 was the ceiling before the settings strip existed, and the strip is furniture +#: rather than content — so it goes *on top of* the ceiling rather than out of the +#: content's share. Taking it out of the share is what the tests caught: the live +#: partial, whose own `PARTIAL_MAX_H` is a flat 70 px, ran through the note and the chip +#: row on a panel pegged at 184 because everything above it had grown by 22 and it had +#: not been told. +PANEL_MAX_H = 184 + SETTINGS_H #: The shortest a band gets, so a one-word draft still has a panel rather than a sliver. PANEL_MIN_H = 96 @@ -1557,6 +1579,78 @@ def _mix(a: str, b: str, t: float) -> str: ) +def _settings_row(c: tk.Canvas, pill, w: int, y: int, tags="body") -> int: + """The controls worth reaching without a right-click. Returns the height it took. + + Three things, and the split between them is deliberate. **Mode is a control**: it + changes what Send does, it is the thing somebody switches mid-task, and it costs a + right-click and two menu levels today. **Workspace and voice are values**: their + worth is being *visible* — knowing which project Ask is running in without opening + anything — and they open the menu that already exists rather than growing a second + implementation of it. + + Everything here is a chip or a label with a hit region, drawn the way `_lay_out` + draws the row at the foot, so there is one shape language on the surface and one way + a thing on it is clicked. + """ + session = getattr(pill, "session", None) + if session is None: + return 0 + converse = getattr(session, "mode", DICTATE) != DICTATE + label = "Converse" if converse else "Dictate" + mid = y + SETTINGS_H // 2 - 4 + + # The mode, as a chip that acts. `v` rather than a real chevron: the strip is drawn + # in the same ASCII-safe font the rest of this surface uses, and a glyph that falls + # back to a box would be a control that looks broken. + text = f"{label} v" + width = chip_w("mode", text) + _round_rect(c, PAD, y, PAD + width, y + SETTINGS_H - 4, 9, + fill=CHIP, outline="", tags=("settings-mode", tags)) + c.create_text(PAD + width / 2, mid, text=text, fill=CODE, font=FONT_CHIP, + tags=("settings-mode", tags)) + c.tag_bind("settings-mode", "", + lambda _e: getattr(session, "toggle_mode", lambda: None)()) + + # The values. Truncated from the left for the workspace, because the tail of a path + # is the part that names the project and the head is the part everybody shares. + x = PAD + width + SETTINGS_GAP + for name, value, opener in _settings_values(pill, session): + if not value: + continue + shown = f"{name} {value}" + tag = f"settings-{name.strip(':')}" + item = c.create_text(x, mid, anchor="w", text=shown, fill=MUTED, + font=FONT_NOTE, tags=(tag, tags)) + bounds = c.bbox(item) + if bounds is not None: + if bounds[2] > w - PAD: + # Out of room. Dropped rather than clipped: half a path is a worse + # answer than no path, and the menu still has it. + c.delete(item) + break + x = bounds[2] + SETTINGS_GAP + if opener is not None: + c.tag_bind(tag, "", lambda _e, f=opener: f()) + return SETTINGS_H + + +def _settings_values(pill, session): + """`(name, value, opener)` for each read-only value on the strip.""" + workspace = getattr(session, "workspace", "") or "" + if workspace: + workspace = os.path.basename(str(workspace).rstrip("/" + chr(92))) or str(workspace) + speaker = getattr(session, "speaker", None) + voice = "" + if speaker is not None: + voice = "muted" if getattr(session, "muted", False) else ( + getattr(speaker, "name", "") or "on") + return ( + ("workshop:", workspace, getattr(pill, "_menu_workspace", None)), + ("voice:", voice, getattr(pill, "_menu_voice", None)), + ) + + def _panel_chrome(c: tk.Canvas, w: int, h: int, radius, ring_color: str, tags="body", seam: str | None = None) -> None: """The opaque three-hairline elevation every v2 surface shares (decisions.md @@ -2498,6 +2592,32 @@ def _set_trigger(self, word: str) -> None: #: path anybody's `--cwd` resolves to. WORKSPACE_NOT_SET = "(not set)" + def _popup_menu(self, build) -> None: + """Post one of the settings submenus on its own, under the pointer. + + The strip's values open the menu that already exists rather than growing a second + implementation of the same list — the workspace recents and the voice list are + both built with a tick showing the current choice, and two of anything is two + things to keep in step. + + `build` is one of the `_*_menu` methods, which add a cascade to a parent. A + throwaway parent is that cascade's home for the moment it is on screen. + """ + parent = _dark_menu(self) + build(parent) + if parent.index("end") is None: + return # nothing to choose between — see each builder's early return + try: + parent.tk_popup(self.winfo_pointerx(), self.winfo_pointery()) + finally: + parent.grab_release() + + def _menu_workspace(self) -> None: + self._popup_menu(self._workspace_menu) + + def _menu_voice(self) -> None: + self._popup_menu(self._voice_menu) + def _workspace_menu(self, parent: tk.Menu) -> None: """Where questions are asked from, as a list of places already chosen. @@ -5415,7 +5535,10 @@ def _render(self) -> None: # shape now (`PANEL_H`), so the room left in it is *always* a hard number, and a # body still asking for `BODY_MAX_H` would draw 340 px of text through the note # and the chip row of a 184 px panel. - around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + # `SETTINGS_H` is in here for the reason everything else is: the body's budget is + # what is left after the fixed furniture, and a strip the height did not know + # about would be a strip drawn over the first line of the draft. + around = 74 + SETTINGS_H + BODY_ELIDED_H + (note_h + 4 if note_h else 0) if partial_h: around += partial_h + PARTIAL_GAP if self._sent: @@ -5470,7 +5593,7 @@ def _render(self) -> None: if not self._frozen(): # Snug around the draft again, stepping a line at a time rather than # tracking every frame — see `_settled_h`. - self._h = self._settled_h(text_h + extra + 74) + self._h = self._settled_h(text_h + extra + 74 + SETTINGS_H) c.configure(width=BUBBLE_W, height=self._h) self.reposition() @@ -5483,6 +5606,9 @@ def _render(self) -> None: _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, seam="bottom") y = PAD + # Above the words, because it describes what will happen to them. See + # `SETTINGS_H` for why it lives here and not on the idle row. + y += _settings_row(c, self.pill, BUBBLE_W, y) if self._sent: c.create_text( PAD, y, anchor="nw", text="sent", fill=MUTED, diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 3e08082..7f20c4f 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -505,3 +505,91 @@ def test_the_row_is_drawn_above_the_body_it_outlived(self): self.assertIn('tag_raise("chips")', inspect.getsource(ui.Bubble._render)) self.assertIn('tag_raise("chips")', inspect.getsource(ui.ConversationCard._render)) + +class TestTheSettingsStrip(unittest.TestCase): + """The controls worth reaching without a right-click. + + Asked for by name — "Dictate and Converse for sure Then workspace and voices" — and + the split between them is the design: mode is a *control*, because it changes what + Send does and is switched mid-task; workspace and voice are *values*, whose worth is + being visible, and which open the menu that already exists rather than growing a + second implementation of the same list. + """ + + def strip(self, **session): + canvas = MeasuringCanvas() + pill = mock.Mock() + pill.session = mock.Mock(**{"mode": ui.DICTATE, "workspace": "", + "speaker": None, "muted": False, **session}) + took = ui._settings_row(canvas, pill, ui.BUBBLE_W, ui.PAD) + texts = [i["text"] for i in canvas.items if i.get("text")] + return took, texts, canvas, pill + + def test_the_mode_is_named_and_is_a_chip(self): + _took, texts, _c, _p = self.strip() + self.assertTrue(any(t.startswith("Dictate") for t in texts), texts) + + def test_converse_says_converse(self): + _took, texts, _c, _p = self.strip(mode=ui.CONVERSE) + self.assertTrue(any(t.startswith("Converse") for t in texts), texts) + + def test_clicking_the_mode_switches_it(self): + _took, _texts, canvas, pill = self.strip() + fire = next(fn for tag, seq, fn in canvas.bindings + if tag == "settings-mode" and seq == "") + fire(None) + pill.session.toggle_mode.assert_called_once() + + def test_a_value_opens_the_menu_that_already_exists(self): + # Rather than a second implementation of the same list: the workspace recents + # and the voice list are both built with a tick showing the current choice, and + # two of anything is two things to keep in step. + _took, _texts, canvas, pill = self.strip(workspace=r"D:/dev/products/flow") + fire = next(fn for tag, seq, fn in canvas.bindings + if tag == "settings-workshop" and seq == "") + fire(None) + pill._menu_workspace.assert_called_once() + + def test_the_workspace_is_shown_by_its_last_component(self): + # The tail of a path names the project; the head is the part everybody shares. + for path in ("D:/dev/products/flow", "D:" + chr(92) + "dev" + chr(92) + "flow", + "/home/sam/flow"): + with self.subTest(path=path): + _took, texts, _c, _p = self.strip(workspace=path) + self.assertIn("workshop: flow", texts) + + def test_a_workspace_that_is_not_set_takes_no_room(self): + _took, texts, _c, _p = self.strip(workspace="") + self.assertFalse([t for t in texts if t.startswith("workshop")]) + + def test_the_voice_is_absent_when_nothing_speaks(self): + _took, texts, _c, _p = self.strip(speaker=None) + self.assertFalse([t for t in texts if t.startswith("voice")]) + + def test_and_says_muted_when_it_is(self): + _took, texts, _c, _p = self.strip(speaker=mock.Mock(name="x"), muted=True) + self.assertIn("voice: muted", texts) + + def test_it_takes_the_height_the_layout_reserved_for_it(self): + # `around` in `_render` counts `SETTINGS_H`; a strip that took more would be + # drawn over the first line of the draft. + took, _texts, _c, _p = self.strip() + self.assertEqual(took, ui.SETTINGS_H) + + def test_a_pill_with_no_session_draws_nothing(self): + # Fixtures build pills with `__new__`, and a strip that assumed a session would + # take the whole surface down with it. + canvas = MeasuringCanvas() + self.assertEqual(ui._settings_row(canvas, mock.Mock(session=None), + ui.BUBBLE_W, ui.PAD), 0) + + def test_the_ceiling_grew_by_the_strip_rather_than_the_draft_paying_for_it(self): + """The strip is furniture, not content. + + Taking it out of the content's share is what the editor tests caught: the live + partial's own cap is a flat 70 px, and on a panel pegged at the old ceiling it + ran straight through the note and the chip row because everything above it had + grown by 22 px and it had not been told. + """ + self.assertEqual(ui.PANEL_MAX_H, 184 + ui.SETTINGS_H) + diff --git a/tests/test_editor.py b/tests/test_editor.py index cd24906..abac53e 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -638,7 +638,13 @@ def test_the_bubble_grows_to_make_room_rather_than_clipping(self): short._render() long_ = self._bubble(self.ERROR) long_._render() - self.assertGreater(long_._h, short._h) + # `assertGreaterEqual`, not `assertGreater`, and the reason is `_settled_h`: the + # band steps by a whole body line, so a note that grew by 14 px can land in the + # same 17 px bucket as the one before it. That is the point of the snap — the + # window stops changing size for every small thing — and the property this test + # is really about is the next two lines, which is that the note is drawn whole. + # The sibling test above asserts it clears the chips. + self.assertGreaterEqual(long_._h, short._h) drawn = next(i for i in long_.canvas.items if "WinError 2" in i["text"]) self.assertEqual(drawn["text"], self.ERROR, "the note must not be truncated") From e2c54e3b5d4e33938ecd9e545e1597cea83408d6 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 02:38:37 -0500 Subject: [PATCH 34/41] Put the settings on the row as icons, and take the strip off the top The strip of words above the draft lasted one screenshot. "Not looking good instead after progress bar add settings icon and on click on just open settings menu and all good and next to that create icon for voice and dictation and top you can remove it." Right, and for a reason worth writing down: words that name a setting are a sentence *about* the app, while the row is where the app already says what it is doing with a drawn mic and a drawn meter. Three more drawn marks belong there. They also cost nothing at rest, which the strip could not - the row is on screen either way, so PANEL_MAX_H goes back to 184 and the draft gets its 22 px back. A gear that opens the settings menu, a speaker that toggles replies, and lines-of-text or a speech-bubble for the mode. Drawn rather than fonted, like the mic: a missing or substituted font turns a control into a box. The speaker wears a slash when muted rather than disappearing, because "off" has to be legible without remembering what "on" looked like. No speaker at all where nothing can speak - an icon that toggles nothing is worse than an absent one. All three are skipped on a pill too narrow to hold them, since the status word is what says whether Flow is listening. The gear opens the menu the right-click already builds. Everything in it is a dispatcher onto the session or the profile, and two of anything is two things to keep in step. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 26 +++--- flow/ui.py | 208 +++++++++++++++++++++++++------------------ tests/test_bubble.py | 88 ------------------ tests/test_pill.py | 105 +++++++++++++++++++++- 4 files changed, 236 insertions(+), 191 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index 91b4f54..2546366 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -884,23 +884,25 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. -### The strip above the draft +### The icons on the row -Three things sit above the words whenever a panel is up, so the settings that matter +Three marks sit between the meter and the status word, so the settings that matter mid-task do not cost a right-click: -**Dictate ⌄** / **Converse ⌄** — a chip, and clicking it switches. This is the one that -changes what Send does, and it is the one people switch mid-task. +**⚙** opens the settings menu — the same one the right-click builds, not a second copy of +it. + +**🔊** toggles spoken replies, and wears a slash when they are muted. It is absent when +nothing can speak (`--no-speak`, or no voice engine), because an icon that toggles +nothing is worse than an absent one. -**workshop:** and **voice:** — values rather than controls. Their worth is being visible -— knowing which project Ask is running in without opening anything — and clicking either -opens the same list the right-click menu has, tick and all. The workspace is shown by the -last part of its path, because that is the part that names the project. +**≡ / 💬** switches Dictate and Converse: lines of text for dictate, a speech bubble for +converse. The two modes differ in *where the words go*, so the marks are "text" and +"a reply" rather than two abstractions to learn. -**It appears with the panel and never at rest.** Those three only mean anything once -there is something to send, and an always-on strip would cost 22 px of the idle row, -which is the part of this surface worth keeping small. FluidVoice does not pay it either -— its bar belongs to the app being dictated into, not to the overlay. +They are drawn rather than fonted, like the mic beside them — a font that is missing or +substituted turns a control into a box. On a pill too narrow to hold them they are +skipped, because the status word is what says whether Flow is listening. ### Hiding it diff --git a/flow/ui.py b/flow/ui.py index 6fdffc4..2b77945 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1097,29 +1097,23 @@ def _unload_fonts() -> None: #: whole body lines (`_settled_h`): a height that can only change when the text gains or #: loses a *line* changes a handful of times an utterance, and a timer that has to be #: cancelled correctly from a render loop is a thing to get wrong. -#: The settings strip's own height, including the air under it. +#: Room for the three icons that sit between the meter and the status word: settings, +#: voice, mode. #: -#: **It appears with the panel and never at rest.** The owner asked for the settings that -#: matter to be reachable without a right-click — "Dictate and Converse for sure Then -#: workspace and voices" — and the choice of *when* was left to me. With a draft up is -#: the answer: those three only mean anything once there is something to send, and an -#: always-on strip would cost 22 px of the idle row, which is the one part of this -#: surface everybody has said they like small. +#: **They were a strip above the draft first, and the strip was wrong.** Written from +#: "Dictate and Converse for sure Then workspace and voices", it put a chip and two +#: labels across the top of the panel — and seeing it, the owner's answer was "Not +#: looking good instead after progress bar add settings icon ... and top you can remove +#: it". They were right. Words that name a setting are a *sentence about* the app; the +#: row is where the app already says what it is doing, with a drawn mic and a drawn +#: meter, and three more drawn marks belong there rather than in a band of prose. #: -#: FluidVoice does not pay this either — its `Dictate / AI Prompt / Actions` bar belongs -#: to the app being dictated into, not to the overlay. -SETTINGS_H = 22 +#: It also costs nothing when idle, which the strip could not: the row is on screen +#: either way. +ICON_SIZE = 16 +ICON_GAP = 12 -#: How far apart the read-only values sit from each other. -SETTINGS_GAP = 14 - -#: 184 was the ceiling before the settings strip existed, and the strip is furniture -#: rather than content — so it goes *on top of* the ceiling rather than out of the -#: content's share. Taking it out of the share is what the tests caught: the live -#: partial, whose own `PARTIAL_MAX_H` is a flat 70 px, ran through the note and the chip -#: row on a panel pegged at 184 because everything above it had grown by 22 and it had -#: not been told. -PANEL_MAX_H = 184 + SETTINGS_H +PANEL_MAX_H = 184 #: The shortest a band gets, so a one-word draft still has a panel rather than a sliver. PANEL_MIN_H = 96 @@ -1579,76 +1573,98 @@ def _mix(a: str, b: str, t: float) -> str: ) -def _settings_row(c: tk.Canvas, pill, w: int, y: int, tags="body") -> int: - """The controls worth reaching without a right-click. Returns the height it took. +def _gear(c: tk.Canvas, cx: float, cy: float, colour: str, tags) -> None: + """A settings gear, drawn rather than fonted. + + Same reasoning as the mic glyph beside it: a font that is missing, substituted or + scaled differently turns a control into a box, and the one thing every control on + this row has to be is recognisable. Eight teeth as short radial spokes, because at + sixteen pixels a toothed outline reads as a smudge and spokes read as a gear. + """ + r = ICON_SIZE / 2 + for i in range(8): + a = math.pi * i / 4 + c.create_line(cx + math.cos(a) * (r - 4), cy + math.sin(a) * (r - 4), + cx + math.cos(a) * r, cy + math.sin(a) * r, + fill=colour, width=2, tags=tags) + c.create_oval(cx - r + 3, cy - r + 3, cx + r - 3, cy + r - 3, + outline=colour, width=2, tags=tags) + + +def _speaker(c: tk.Canvas, cx: float, cy: float, colour: str, muted: bool, tags) -> None: + """A speaker, with a slash through it when replies are muted. + + The slash rather than a different colour or a missing icon: "off" has to be legible + without remembering what "on" looked like, and an icon that disappears when a setting + is off is a setting nobody can find their way back to. + """ + r = ICON_SIZE / 2 + c.create_rectangle(cx - r, cy - 3, cx - r + 4, cy + 3, + fill=colour, outline=colour, tags=tags) + c.create_polygon(cx - r + 4, cy - 3, cx - 1, cy - r, cx - 1, cy + r, + cx - r + 4, cy + 3, fill=colour, outline=colour, tags=tags) + if muted: + c.create_line(cx + 1, cy - 5, cx + r, cy + 5, fill=colour, width=2, tags=tags) + else: + for i in (0, 1): + c.create_arc(cx - 1 + i * 4, cy - 5 - i * 3, cx + 5 + i * 4, cy + 5 + i * 3, + start=-60, extent=120, style=tk.ARC, + outline=colour, width=2, tags=tags) + + +def _mode_glyph(c: tk.Canvas, cx: float, cy: float, colour: str, converse: bool, + tags) -> None: + """Lines of text for dictate, a speech bubble for converse. + + The two modes differ in *where the words go* — into the window you were in, or to an + agent that answers — so the marks are "text" and "a reply", not two abstractions + somebody has to learn. + """ + r = ICON_SIZE / 2 + if converse: + _round_rect(c, cx - r, cy - r + 1, cx + r, cy + r - 4, 4, + fill="", outline=colour, tags=tags) + c.create_line(cx - 2, cy + r - 4, cx - 4, cy + r, fill=colour, width=2, + tags=tags) + else: + for i, width in enumerate((r * 2, r * 2, r * 1.2)): + y = cy - r + 3 + i * 5 + c.create_line(cx - r, y, cx - r + width, y, fill=colour, width=2, tags=tags) + + +def _row_icons(c: tk.Canvas, pill, x: float, mid: float, tags="row") -> float: + """Settings, voice and mode, between the meter and the status word. - Three things, and the split between them is deliberate. **Mode is a control**: it - changes what Send does, it is the thing somebody switches mid-task, and it costs a - right-click and two menu levels today. **Workspace and voice are values**: their - worth is being *visible* — knowing which project Ask is running in without opening - anything — and they open the menu that already exists rather than growing a second - implementation of it. + Returns the x the caller may draw from next. Each is a hit region as well as a mark: + the gear opens the same menu a right-click opens, the speaker toggles replies, and + the mode glyph switches Dictate and Converse — the two settings the owner named as + worth reaching without a right-click, plus the way to everything else. - Everything here is a chip or a label with a hit region, drawn the way `_lay_out` - draws the row at the foot, so there is one shape language on the surface and one way - a thing on it is clicked. + Voice is drawn only where something can speak. An icon that toggles nothing is worse + than an absent one, and `session.speaker` is None whenever `--no-speak` or a missing + voice engine has made replies impossible. """ session = getattr(pill, "session", None) if session is None: - return 0 + return x converse = getattr(session, "mode", DICTATE) != DICTATE - label = "Converse" if converse else "Dictate" - mid = y + SETTINGS_H // 2 - 4 - - # The mode, as a chip that acts. `v` rather than a real chevron: the strip is drawn - # in the same ASCII-safe font the rest of this surface uses, and a glyph that falls - # back to a box would be a control that looks broken. - text = f"{label} v" - width = chip_w("mode", text) - _round_rect(c, PAD, y, PAD + width, y + SETTINGS_H - 4, 9, - fill=CHIP, outline="", tags=("settings-mode", tags)) - c.create_text(PAD + width / 2, mid, text=text, fill=CODE, font=FONT_CHIP, - tags=("settings-mode", tags)) - c.tag_bind("settings-mode", "", - lambda _e: getattr(session, "toggle_mode", lambda: None)()) - - # The values. Truncated from the left for the workspace, because the tail of a path - # is the part that names the project and the head is the part everybody shares. - x = PAD + width + SETTINGS_GAP - for name, value, opener in _settings_values(pill, session): - if not value: - continue - shown = f"{name} {value}" - tag = f"settings-{name.strip(':')}" - item = c.create_text(x, mid, anchor="w", text=shown, fill=MUTED, - font=FONT_NOTE, tags=(tag, tags)) - bounds = c.bbox(item) - if bounds is not None: - if bounds[2] > w - PAD: - # Out of room. Dropped rather than clipped: half a path is a worse - # answer than no path, and the menu still has it. - c.delete(item) - break - x = bounds[2] + SETTINGS_GAP - if opener is not None: - c.tag_bind(tag, "", lambda _e, f=opener: f()) - return SETTINGS_H - - -def _settings_values(pill, session): - """`(name, value, opener)` for each read-only value on the strip.""" - workspace = getattr(session, "workspace", "") or "" - if workspace: - workspace = os.path.basename(str(workspace).rstrip("/" + chr(92))) or str(workspace) - speaker = getattr(session, "speaker", None) - voice = "" - if speaker is not None: - voice = "muted" if getattr(session, "muted", False) else ( - getattr(speaker, "name", "") or "on") - return ( - ("workshop:", workspace, getattr(pill, "_menu_workspace", None)), - ("voice:", voice, getattr(pill, "_menu_voice", None)), - ) + + def hit(tag: str, command) -> None: + c.tag_bind(tag, "", lambda _e: command()) + + _gear(c, x + ICON_SIZE / 2, mid, MUTED, ("row-gear", tags)) + hit("row-gear", getattr(pill, "open_settings", lambda: None)) + x += ICON_SIZE + ICON_GAP + + if getattr(session, "speaker", None) is not None: + _speaker(c, x + ICON_SIZE / 2, mid, MUTED, + bool(getattr(session, "muted", False)), ("row-voice", tags)) + hit("row-voice", getattr(session, "toggle_speech", lambda: None)) + x += ICON_SIZE + ICON_GAP + + _mode_glyph(c, x + ICON_SIZE / 2, mid, MUTED, converse, ("row-mode", tags)) + hit("row-mode", getattr(session, "toggle_mode", lambda: None)) + return x + ICON_SIZE + ICON_GAP def _panel_chrome(c: tk.Canvas, w: int, h: int, radius, ring_color: str, @@ -2592,6 +2608,16 @@ def _set_trigger(self, word: str) -> None: #: path anybody's `--cwd` resolves to. WORKSPACE_NOT_SET = "(not set)" + def open_settings(self) -> None: + """Post the settings menu on its own, from the gear on the row. + + The same menu the right-click builds, not a second one: everything in it is a + dispatcher onto the session or the profile, and two of anything is two things to + keep in step. The gear is a shortcut to it, which is the whole of what was asked + for — "on click on just open settings menu and all good". + """ + self._popup_menu(self._settings_menu) + def _popup_menu(self, build) -> None: """Post one of the settings submenus on its own, under the pointer. @@ -4062,6 +4088,13 @@ def _draw(self) -> None: else: c.create_rectangle(x, mid - h, x + BAR_W, mid + h, fill=shade, outline="") + # After the meter and before the word, which is where the owner asked for them: + # "after progress bar add settings icon ... next to that create icon for voice + # and dictation". Only when there is room — a narrow pill would otherwise draw + # them over the status word, and the word is the thing that says whether Flow is + # listening. + if w >= PILL_W + 3 * (ICON_SIZE + ICON_GAP): + _row_icons(c, self, METER_X + METER_W + LABEL_GAP, mid) self._draw_label(c, w, mid, accent) def _draw_dots(self, c: tk.Canvas, mid: int, accent: str) -> None: @@ -5538,7 +5571,7 @@ def _render(self) -> None: # `SETTINGS_H` is in here for the reason everything else is: the body's budget is # what is left after the fixed furniture, and a strip the height did not know # about would be a strip drawn over the first line of the draft. - around = 74 + SETTINGS_H + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) if partial_h: around += partial_h + PARTIAL_GAP if self._sent: @@ -5593,7 +5626,7 @@ def _render(self) -> None: if not self._frozen(): # Snug around the draft again, stepping a line at a time rather than # tracking every frame — see `_settled_h`. - self._h = self._settled_h(text_h + extra + 74 + SETTINGS_H) + self._h = self._settled_h(text_h + extra + 74) c.configure(width=BUBBLE_W, height=self._h) self.reposition() @@ -5606,9 +5639,6 @@ def _render(self) -> None: _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, seam="bottom") y = PAD - # Above the words, because it describes what will happen to them. See - # `SETTINGS_H` for why it lives here and not on the idle row. - y += _settings_row(c, self.pill, BUBBLE_W, y) if self._sent: c.create_text( PAD, y, anchor="nw", text="sent", fill=MUTED, diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 7f20c4f..3e08082 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -505,91 +505,3 @@ def test_the_row_is_drawn_above_the_body_it_outlived(self): self.assertIn('tag_raise("chips")', inspect.getsource(ui.Bubble._render)) self.assertIn('tag_raise("chips")', inspect.getsource(ui.ConversationCard._render)) - -class TestTheSettingsStrip(unittest.TestCase): - """The controls worth reaching without a right-click. - - Asked for by name — "Dictate and Converse for sure Then workspace and voices" — and - the split between them is the design: mode is a *control*, because it changes what - Send does and is switched mid-task; workspace and voice are *values*, whose worth is - being visible, and which open the menu that already exists rather than growing a - second implementation of the same list. - """ - - def strip(self, **session): - canvas = MeasuringCanvas() - pill = mock.Mock() - pill.session = mock.Mock(**{"mode": ui.DICTATE, "workspace": "", - "speaker": None, "muted": False, **session}) - took = ui._settings_row(canvas, pill, ui.BUBBLE_W, ui.PAD) - texts = [i["text"] for i in canvas.items if i.get("text")] - return took, texts, canvas, pill - - def test_the_mode_is_named_and_is_a_chip(self): - _took, texts, _c, _p = self.strip() - self.assertTrue(any(t.startswith("Dictate") for t in texts), texts) - - def test_converse_says_converse(self): - _took, texts, _c, _p = self.strip(mode=ui.CONVERSE) - self.assertTrue(any(t.startswith("Converse") for t in texts), texts) - - def test_clicking_the_mode_switches_it(self): - _took, _texts, canvas, pill = self.strip() - fire = next(fn for tag, seq, fn in canvas.bindings - if tag == "settings-mode" and seq == "") - fire(None) - pill.session.toggle_mode.assert_called_once() - - def test_a_value_opens_the_menu_that_already_exists(self): - # Rather than a second implementation of the same list: the workspace recents - # and the voice list are both built with a tick showing the current choice, and - # two of anything is two things to keep in step. - _took, _texts, canvas, pill = self.strip(workspace=r"D:/dev/products/flow") - fire = next(fn for tag, seq, fn in canvas.bindings - if tag == "settings-workshop" and seq == "") - fire(None) - pill._menu_workspace.assert_called_once() - - def test_the_workspace_is_shown_by_its_last_component(self): - # The tail of a path names the project; the head is the part everybody shares. - for path in ("D:/dev/products/flow", "D:" + chr(92) + "dev" + chr(92) + "flow", - "/home/sam/flow"): - with self.subTest(path=path): - _took, texts, _c, _p = self.strip(workspace=path) - self.assertIn("workshop: flow", texts) - - def test_a_workspace_that_is_not_set_takes_no_room(self): - _took, texts, _c, _p = self.strip(workspace="") - self.assertFalse([t for t in texts if t.startswith("workshop")]) - - def test_the_voice_is_absent_when_nothing_speaks(self): - _took, texts, _c, _p = self.strip(speaker=None) - self.assertFalse([t for t in texts if t.startswith("voice")]) - - def test_and_says_muted_when_it_is(self): - _took, texts, _c, _p = self.strip(speaker=mock.Mock(name="x"), muted=True) - self.assertIn("voice: muted", texts) - - def test_it_takes_the_height_the_layout_reserved_for_it(self): - # `around` in `_render` counts `SETTINGS_H`; a strip that took more would be - # drawn over the first line of the draft. - took, _texts, _c, _p = self.strip() - self.assertEqual(took, ui.SETTINGS_H) - - def test_a_pill_with_no_session_draws_nothing(self): - # Fixtures build pills with `__new__`, and a strip that assumed a session would - # take the whole surface down with it. - canvas = MeasuringCanvas() - self.assertEqual(ui._settings_row(canvas, mock.Mock(session=None), - ui.BUBBLE_W, ui.PAD), 0) - - def test_the_ceiling_grew_by_the_strip_rather_than_the_draft_paying_for_it(self): - """The strip is furniture, not content. - - Taking it out of the content's share is what the editor tests caught: the live - partial's own cap is a flat 70 px, and on a panel pegged at the old ceiling it - ran straight through the note and the chip row because everything above it had - grown by 22 px and it had not been told. - """ - self.assertEqual(ui.PANEL_MAX_H, 184 + ui.SETTINGS_H) - diff --git a/tests/test_pill.py b/tests/test_pill.py index 78c6076..ac76068 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -32,6 +32,15 @@ def __init__(self) -> None: self.ovals: list[tuple[float, float, float, float, str]] = [] self.rects: list[tuple[float, float, float, float, str]] = [] self.lines: list[tuple[tuple, str]] = [] + #: (tag, sequence, callback) per `tag_bind`. The row grew hit regions when the + #: settings, voice and mode icons moved onto it, and a click that reaches nothing + #: is the failure worth asserting against. + self.bindings: list[tuple] = [] + self.polys: list[tuple] = [] + self.arcs: list[tuple] = [] + + def tag_bind(self, tag, sequence, callback) -> None: + self.bindings.append((tag, sequence, callback)) def delete(self, *a, **kw) -> None: ... @@ -46,9 +55,11 @@ def place(self, **kw) -> None: def configure(self, **kw) -> None: self.width = kw.get("width", getattr(self, "width", None)) - def create_polygon(self, *a, **kw) -> None: ... + def create_polygon(self, *a, **kw) -> None: + self.polys.append((a, kw.get("fill", ""))) - def create_arc(self, *a, **kw) -> None: ... + def create_arc(self, *a, **kw) -> None: + self.arcs.append((a, kw.get("outline", ""))) def create_oval(self, x1, y1, x2, y2, **kw) -> None: self.ovals.append((x1, y1, x2, y2, kw.get("fill", ""))) @@ -1305,5 +1316,95 @@ def test_the_row_is_placed_at_the_foot_of_whatever_height_it_is(self): self.assertEqual(kw["y"] + kw["height"], p._shell_h) +class TestTheRowIcons(unittest.TestCase): + """Settings, voice and mode, between the meter and the status word. + + **These were a strip of words above the draft first.** Written from "Dictate and + Converse for sure Then workspace and voices", it put a chip and two labels across the + top of the panel — and on seeing it the owner's answer was "Not looking good instead + after progress bar add settings icon ... and top you can remove it". Words that name + a setting are a sentence *about* the app; the row is where the app already says what + it is doing with a drawn mic and a drawn meter, and three more drawn marks belong + there. It costs nothing at rest, too, which the strip could not — the row is on + screen either way. + """ + + def row(self, **session): + c = Canvas() + p = ui.Pill.__new__(ui.Pill) + p.session = mock.Mock(**{"mode": DICTATE, "speaker": None, "muted": False, + **session}) + p.open_settings = mock.Mock() + return c, p, ui._row_icons(c, p, 100, 20) + + def fire(self, c, tag): + next(fn for t, seq, fn in c.bindings + if t == tag and seq == "")(None) + + def tags(self, c): + drawn = set() + for item in c.ovals + c.rects: + drawn |= set(item[-1] if isinstance(item[-1], tuple) else ()) + return drawn + + def test_the_gear_opens_the_settings_menu(self): + # The same menu the right-click builds, not a second one: everything in it is a + # dispatcher onto the session or the profile. + c, p, _x = self.row() + self.fire(c, "row-gear") + p.open_settings.assert_called_once() + + def test_the_mode_glyph_switches_the_mode(self): + c, p, _x = self.row() + self.fire(c, "row-mode") + p.session.toggle_mode.assert_called_once() + + def test_the_speaker_toggles_replies(self): + c, p, _x = self.row(speaker=mock.Mock()) + self.fire(c, "row-voice") + p.session.toggle_speech.assert_called_once() + + def test_there_is_no_speaker_icon_when_nothing_can_speak(self): + # An icon that toggles nothing is worse than an absent one. `speaker` is None + # under `--no-speak` and wherever the voice engine is missing. + c, _p, _x = self.row(speaker=None) + self.assertFalse([b for b in c.bindings if b[0] == "row-voice"]) + + def test_and_it_is_there_when_something_can(self): + c, _p, _x = self.row(speaker=mock.Mock()) + self.assertTrue([b for b in c.bindings if b[0] == "row-voice"]) + + def test_muting_changes_the_mark_rather_than_removing_it(self): + # "Off" has to be legible without remembering what "on" looked like, and an icon + # that vanishes when a setting is off is a setting nobody finds their way back to. + loud, _p, _x = self.row(speaker=mock.Mock(), muted=False) + quiet, _p2, _x2 = self.row(speaker=mock.Mock(), muted=True) + self.assertTrue([b for b in quiet.bindings if b[0] == "row-voice"]) + self.assertNotEqual(len(loud.lines), len(quiet.lines)) + + def test_a_pill_with_no_session_draws_nothing(self): + # Fixtures build pills with `__new__`, and a row that assumed a session would + # take the whole surface down with it. + c = Canvas() + p = ui.Pill.__new__(ui.Pill) + p.session = None + self.assertEqual(ui._row_icons(c, p, 100, 20), 100) + self.assertEqual(c.bindings, []) + + def test_they_are_skipped_on_a_pill_too_narrow_to_hold_them(self): + """The status word is the thing that says whether Flow is listening, and three + icons drawn over it would cost more than they add.""" + p = ui.Pill.__new__(ui.Pill) + p.canvas = Canvas() + p.session = mock.Mock(mode=DICTATE, speaker=None, muted=False) + p._docked_w = ui.PILL_W + p._shell_h = ui.PILL_H + p._flash = 0 + p.armed = False + p._meter_level = 0.0 + ui.Pill._draw(p) + self.assertFalse([b for b in p.canvas.bindings if b[0].startswith("row-")]) + + if __name__ == "__main__": # pragma: no cover unittest.main() From 7f586be10726084d1f084e39f36ad5ec15592842 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 02:49:52 -0500 Subject: [PATCH 35/41] Stop a dictated line running itself in a Mac terminal inject.py has stripped a draft's trailing newline for a terminal since 2026-08-03, because a trailing newline in a shell does not paste, it runs. inject_mac had no such guard: it aimed at whatever was frontmost and pasted verbatim. Somebody dictating into a Mac terminal could have their words execute, and it was live for as long as the paste path was - found by reading the two files side by side rather than by hitting it. frontmost() now answers with the bundle id as well as the name, in the same osascript, because each call is a process launch on the path of every send. Terminals are matched on that id with a fallback on any process name containing "term": the identifiers come from each app's documentation rather than a measurement here, which is a weaker warrant than this repo usually takes, and the asymmetry is why there is a fallback - a list too eager costs a Return somebody has to press, one too shy costs a line that runs. Multiple lines into a terminal warn rather than refuse. Windows refuses, on a measured list of which terminals bracket pastes; nothing here has measured a macOS one, and a refusal built on a guess would block a paste that was always safe. The pipe separator is not decoration: an AppleScript string literal cannot contain a raw line break, so joining the two facts with one is a script that does not compile. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 10 ++++ flow/inject_mac.py | 100 +++++++++++++++++++++++++++++++++++---- tests/test_inject_mac.py | 97 +++++++++++++++++++++++++++++++++++-- 3 files changed, 194 insertions(+), 13 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index 2546366..b88664b 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -948,6 +948,16 @@ did not synthesise keystrokes at all. `enter boom` works the same way — the Return is sent as a key code rather than a typed character, in the same script as the paste so nothing can come forward between them. +**Flow never presses Return for you in a terminal.** A draft ending in a newline does not +paste into a shell, it *runs* — so the trailing newline is stripped whenever the frontmost +app is a terminal, exactly as it is on Windows. The app is identified by its bundle id, +with a fallback on any process name containing "term", because a terminal missed by both +costs the guarantee. + +Multiple lines into a terminal print a warning naming it. Windows *refuses* that paste, +on a measured list of which terminals implement bracketed paste; nothing has measured a +macOS terminal, so this says what might happen rather than acting on a guess. + ### Sending it without touching anything Two words press Send, so the last step of the loop does not need the mouse: diff --git a/flow/inject_mac.py b/flow/inject_mac.py index 87c97d5..853f839 100644 --- a/flow/inject_mac.py +++ b/flow/inject_mac.py @@ -48,6 +48,31 @@ #: prose, which is localised. DENIED_CODES = ("-1719", "1002", "-25211") +#: Bundle identifiers of the terminals a dictated line must never be allowed to *run* +#: in. `inject.py` classifies its target by window class or process name and this is the +#: same rule on the platform that has neither: `prepare` there strips a draft's trailing +#: newline for a terminal, because a trailing newline in a shell does not paste, it runs. +#: macOS had no such guard at all until now — the whole of `inject_mac` aimed at whatever +#: was frontmost and pasted it verbatim. +#: +#: **These are identifiers, not invocation shapes.** They come from each app's own +#: documentation rather than from a measurement here, which is a weaker warrant than this +#: repository usually accepts — so the match falls back to the process *name* containing +#: "term", and a terminal missed by both costs the guarantee rather than breaking the +#: paste. A list that is too eager costs a trailing newline somebody has to press Enter +#: for; too shy costs a line that runs. The asymmetry is the reason for the fallback. +TERMINAL_IDS = frozenset({ + "com.apple.Terminal", + "com.googlecode.iterm2", + "io.alacritty", + "net.kovidgoyal.kitty", + "com.github.wez.wezterm", + "co.zeit.hyper", + "org.tabby", + "com.mitchellh.ghostty", + "dev.warp.Warp-Stable", +}) + #: Names a frontmost process can have when the thing in front is Flow itself. Flow #: pasting into its own draft box is the one outcome that would destroy the text being #: sent, and `ui._bare_window` is what should make it impossible — this is the belt to @@ -110,12 +135,57 @@ def permission_note() -> str: "clipboard, so Cmd-V works in the meantime.") -def frontmost() -> str: - """The name of the app that will receive the paste, or "" if it cannot be asked.""" - ok, out = _run(["osascript", "-e", - 'tell application "System Events" to get name of first process ' - "whose frontmost is true"]) - return out.strip() if ok else "" +def frontmost() -> tuple[str, str]: + """`(name, bundle id)` of the app that will receive the paste, or `("", "")`. + + Both in one `osascript`, because each call is a process launch and this one is on the + path of every send. The bundle id is what identifies a terminal; the name is what a + warning has to be readable with, and what the Flow-has-the-focus check reads. + """ + # `|` rather than a newline between the two: AppleScript string literals cannot + # contain a raw line break, so joining them with one is a script that does not + # compile. A pipe cannot appear in a bundle identifier, and a name carrying one still + # splits correctly because only the *last* field is the identifier. + script = chr(10).join([ + 'tell application "System Events"', + "set p to first process whose frontmost is true", + 'get (name of p) & "|" & (bundle identifier of p)', + "end tell", + ]) + ok, out = _run(["osascript", "-e", script]) + if not ok: + return "", "" + name, _, bundle = out.strip().rpartition("|") + return (name.strip(), bundle.strip()) if name else (bundle.strip(), "") + + +def is_terminal(name: str, bundle: str) -> bool: + """Whether a paste into this app can execute what it lands on.""" + return bundle in TERMINAL_IDS or "term" in (name or "").lower() + + +def prepare(text: str, name: str, bundle: str) -> tuple[str, str]: + """`(payload, warning)` for pasting `text` into that app. `inject.prepare`'s rule. + + **Never submit for the user.** A draft ending in a newline pastes as text plus + Return, which in a shell runs it. The trailing newline is stripped for a terminal; + the user presses Return when they mean to. + + **Say when the lines may run on arrival.** Windows knows which terminals implement + bracketed paste, from a measured list, and refuses outright for the ones that do not. + Nothing here has measured a single macOS terminal, so this warns rather than refuses + and does not claim which behaviour this one has — a refusal built on a guess would + block a paste that was always safe. + """ + if not is_terminal(name, bundle): + return text, "" + payload = text.rstrip(chr(13) + chr(10)) + if chr(10) in payload: + return payload, ( + f"{name or 'this terminal'} may run each line as it arrives, if it does " + "not bracket pastes" + ) + return payload, "" def get_clipboard_text() -> str | None: @@ -175,10 +245,20 @@ def paste( if not text: return False - # Asked before anything is touched, so the refusal costs no clipboard. - front = frontmost() - if front.lower() in FLOW_NAMES: - _warn(f"not pasted: {front} had the focus, not the window you were aiming at") + # Asked before anything is touched, so the refusal costs no clipboard — and asked + # *once*, because the answer decides both whether to paste at all and what to paste. + name, bundle = frontmost() + if name.lower() in FLOW_NAMES: + _warn(f"not pasted: {name} had the focus, not the window you were aiming at") + return False + + # P7's guarantee, on the platform that had none: a draft ending in a newline does not + # paste into a shell, it runs. Classified before the clipboard is touched, the same + # order `inject.paste` uses and for the same reason. + text, hazard = prepare(text, name, bundle) + if hazard: + _warn(hazard) + if not text: return False previous = get_clipboard_text() if restore_clipboard else None diff --git a/tests/test_inject_mac.py b/tests/test_inject_mac.py index d6f6d48..d1a5a7b 100644 --- a/tests/test_inject_mac.py +++ b/tests/test_inject_mac.py @@ -21,9 +21,10 @@ class Ran: """Records the child processes `paste` would start, and answers for them.""" - def __init__(self, front="TextEdit", clipboard="old text", fail=None, reason=""): + def __init__(self, front="TextEdit", clipboard="old text", fail=None, reason="", + bundle="com.apple.TextEdit"): self.calls: list[tuple[list[str], str | None]] = [] - self.front, self.clipboard = front, clipboard + self.front, self.clipboard, self.bundle = front, clipboard, bundle self.fail, self.reason = fail, reason def __call__(self, argv, stdin=None): @@ -34,7 +35,7 @@ def __call__(self, argv, stdin=None): if tool == "pbpaste": return True, self.clipboard if tool == "osascript" and "frontmost" in argv[-1]: - return True, self.front + return True, f"{self.front}|{self.bundle}" return True, "" @property @@ -226,3 +227,93 @@ def test_it_is_ascii_like_every_other_startup_line(self): if __name__ == "__main__": # pragma: no cover unittest.main() + +class TestATerminalNeverRunsWhatWasDictated(unittest.TestCase): + """The guard `inject.py` has had since 2026-08-03, on the platform that had none. + + A draft ending in a newline does not paste into a shell — it *runs*. Windows resolves + its target and strips that newline; `inject_mac` aimed at whatever was frontmost and + pasted it verbatim, so somebody dictating into a Mac terminal could have their words + execute. It was live for as long as the paste path was. + """ + + def setUp(self): + inject_mac.take_warnings() + self.addCleanup(inject_mac.take_warnings) + + def test_a_trailing_newline_is_stripped_for_a_terminal(self): + payload, _why = inject_mac.prepare("ls -la\n", "Terminal", "com.apple.Terminal") + self.assertEqual(payload, "ls -la") + + def test_and_kept_everywhere_else(self): + # Stripping it in an editor would be Flow editing a draft to fit a guess. + payload, _why = inject_mac.prepare("hello\n", "Notes", "com.apple.Notes") + self.assertEqual(payload, "hello\n") + + def test_every_shipped_identifier_is_recognised(self): + for bundle in inject_mac.TERMINAL_IDS: + with self.subTest(bundle=bundle): + self.assertTrue(inject_mac.is_terminal("", bundle)) + + def test_a_terminal_not_on_the_list_is_still_caught_by_name(self): + """The fallback, and the asymmetry that earns it: a list too eager costs a + trailing newline somebody has to press Return for, and one too shy costs a line + that runs.""" + self.assertTrue(inject_mac.is_terminal("Some New Term", "com.unknown.thing")) + self.assertFalse(inject_mac.is_terminal("Notes", "com.apple.Notes")) + + def test_multiple_lines_into_a_terminal_are_reported(self): + # Warned rather than refused: Windows refuses on a *measured* list of which + # terminals bracket pastes, and nothing here has measured a single macOS one. + _payload, why = inject_mac.prepare("a\nb\n", "iTerm2", "com.googlecode.iterm2") + self.assertIn("iTerm2", why) + self.assertIn("each line", why) + + def test_one_line_into_a_terminal_says_nothing(self): + _payload, why = inject_mac.prepare("ls\n", "Terminal", "com.apple.Terminal") + self.assertEqual(why, "") + + def test_the_paste_uses_the_stripped_payload(self): + ran = Ran(front="Terminal", bundle="com.apple.Terminal") + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later"): + self.assertTrue(inject_mac.paste("ls -la\n")) + self.assertEqual(ran.copied(), ["ls -la"]) + + def test_and_the_hazard_reaches_the_user(self): + ran = Ran(front="Terminal", bundle="com.apple.Terminal") + with mock.patch.object(inject_mac, "_run", ran), \ + mock.patch.object(inject_mac, "_restore_later"): + inject_mac.paste("one\ntwo\n") + self.assertTrue([w for w in inject_mac.take_warnings() if "each line" in w]) + + def test_a_draft_that_was_only_a_newline_is_not_pasted(self): + # Stripped to nothing is nothing to send, and an empty clipboard write followed + # by a Cmd-V would paste whatever was there before. + ran = Ran(front="Terminal", bundle="com.apple.Terminal") + with mock.patch.object(inject_mac, "_run", ran): + self.assertFalse(inject_mac.paste("\n")) + self.assertEqual(ran.copied(), []) + + +class TestTheFrontmostQuery(unittest.TestCase): + def test_it_asks_once_for_both_facts(self): + # Each osascript is a process launch and this one is on the path of every send. + ran = Ran() + with mock.patch.object(inject_mac, "_run", ran): + self.assertEqual(inject_mac.frontmost(), + ("TextEdit", "com.apple.TextEdit")) + self.assertEqual(len([c for c, _s in ran.calls if c[0] == "osascript"]), 1) + + def test_the_separator_is_not_a_line_break(self): + """AppleScript string literals cannot contain a raw line break, so joining the + two facts with one is a script that does not compile.""" + ran = Ran() + with mock.patch.object(inject_mac, "_run", ran): + inject_mac.frontmost() + self.assertIn(chr(38) + ' "|" ' + chr(38), ran.calls[0][0][-1]) + + def test_a_refusal_answers_with_two_empty_strings(self): + ran = Ran(fail="frontmost", reason="(-1719)") + with mock.patch.object(inject_mac, "_run", ran): + self.assertEqual(inject_mac.frontmost(), ("", "")) From 1d9454f35ff1850e9c6a8dd7b333d717c59b634f Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 02:56:02 -0500 Subject: [PATCH 36/41] Let the gear post the settings themselves, not a Settings cascade The gear called _settings_menu, which *adds a cascade* to whatever menu it is handed - so clicking it opened a menu whose only row was "Settings >", and every entry was one hover further away than the right-click already had it. A shortcut that costs an extra hover is not a shortcut. Split into _settings_items, which fills a menu somebody else owns, and _settings_menu, which makes one and hangs it off a parent. The gear posts the items; the right-click keeps the cascade, because there it sits among Mode, Draft and Help and has to be one of them. _menu_workspace and _menu_voice went with the strip that used them. Dead code is worse than none. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 5 +++-- flow/ui.py | 29 ++++++++++++++++++----------- tests/test_pill.py | 17 +++++++++++++++-- 3 files changed, 36 insertions(+), 15 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index b88664b..f28c415 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -889,8 +889,9 @@ The clipboard is restored about 0.6 s after the paste, so Flow does not permanen Three marks sit between the meter and the status word, so the settings that matter mid-task do not cost a right-click: -**⚙** opens the settings menu — the same one the right-click builds, not a second copy of -it. +**⚙** opens the settings list itself — the same entries the right-click builds, not a +second copy of them, and without the `Settings` cascade in front of them. A shortcut +that costs an extra hover is not a shortcut. **🔊** toggles spoken replies, and wears a slash when they are muted. It is absent when nothing can speak (`--no-speak`, or no voice engine), because an icon that toggles diff --git a/flow/ui.py b/flow/ui.py index 2b77945..954923b 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -2485,6 +2485,20 @@ def _settings_menu(self, parent: tk.Menu) -> None: invites options to be added to it. """ sub = _dark_menu(parent) + self._settings_items(sub) + parent.add_cascade(label="Settings", menu=sub) + + def _settings_items(self, sub: tk.Menu) -> None: + """Everything under Settings, filled into a menu somebody else owns. + + Split out from `_settings_menu` so the gear on the row can post *this* rather + than a menu whose only entry is a `Settings` cascade — which is what it did, and + what the owner saw: "Remove the layer settings as clicking on icon it can open + directly". A shortcut that costs an extra hover is not a shortcut. + + The right-click still gets the cascade, because there it sits among Mode, Draft + and Help and has to be one of them. + """ self._gesture_menu(sub) self._trigger_menu(sub) self._panel_menu(sub) @@ -2534,7 +2548,6 @@ def choice(label: str, cli) -> None: command=self.session.toggle_auto_ask, ) sub.add_command(label="Open settings folder", command=self._open_settings) - parent.add_cascade(label="Settings", menu=sub) def _trigger_menu(self, parent: tk.Menu) -> None: """The word that presses Send, as a list rather than a text box. @@ -2611,12 +2624,12 @@ def _set_trigger(self, word: str) -> None: def open_settings(self) -> None: """Post the settings menu on its own, from the gear on the row. - The same menu the right-click builds, not a second one: everything in it is a + The same items the right-click builds, not a second set: everything in them is a dispatcher onto the session or the profile, and two of anything is two things to - keep in step. The gear is a shortcut to it, which is the whole of what was asked - for — "on click on just open settings menu and all good". + keep in step. What the gear skips is the *cascade* — posting `_settings_menu` + gave a menu whose only row was `Settings >`, so the shortcut cost an extra hover. """ - self._popup_menu(self._settings_menu) + self._popup_menu(self._settings_items) def _popup_menu(self, build) -> None: """Post one of the settings submenus on its own, under the pointer. @@ -2638,12 +2651,6 @@ def _popup_menu(self, build) -> None: finally: parent.grab_release() - def _menu_workspace(self) -> None: - self._popup_menu(self._workspace_menu) - - def _menu_voice(self) -> None: - self._popup_menu(self._voice_menu) - def _workspace_menu(self, parent: tk.Menu) -> None: """Where questions are asked from, as a list of places already chosen. diff --git a/tests/test_pill.py b/tests/test_pill.py index ac76068..3613825 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -1348,12 +1348,25 @@ def tags(self, c): return drawn def test_the_gear_opens_the_settings_menu(self): - # The same menu the right-click builds, not a second one: everything in it is a - # dispatcher onto the session or the profile. + # The same items the right-click builds, not a second set: everything in them is + # a dispatcher onto the session or the profile. c, p, _x = self.row() self.fire(c, "row-gear") p.open_settings.assert_called_once() + def test_the_gear_posts_the_items_and_not_a_settings_cascade(self): + """A shortcut that costs an extra hover is not a shortcut. + + `open_settings` posted `_settings_menu`, which *adds a cascade* to whatever it is + given — so the gear opened a menu whose only row was `Settings >`, and the whole + list was one hover further away than the right-click had it. + """ + p = ui.Pill.__new__(ui.Pill) + posted = [] + p._popup_menu = posted.append + ui.Pill.open_settings(p) + self.assertEqual(posted, [p._settings_items]) + def test_the_mode_glyph_switches_the_mode(self): c, p, _x = self.row() self.fire(c, "row-mode") From 03f5280009df2c30c5fd8863e70c4b9542411269 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 03:08:02 -0500 Subject: [PATCH 37/41] Name the window Flow is aimed at, at the left of the row The icon was asked for - "if i am on notepad i see notepad icon and when i am on claude ide i see claude icon" - and the name is the half that costs nothing. _track_target already resolves the foreground process for the paste, on the edge rather than per frame, so session.target_app was already sitting there reading claude.exe. The picture needs ExtractIconExW, then GetIconInfo and GetDIBits to get pixels into a PhotoImage, which is a different size of job and can follow. Fixed-width slot, so nothing moves. One that sized itself to the name would shift the mic, the meter and every icon each time you changed window, which is the motion this surface spent a night removing. Long names are cut at the end rather than the start: that is the opposite of what the draft body does and for the opposite reason - a draft is windowed to its tail because the newest words are the ones being spoken, while an application is recognised by its head. _row_shift is a method rather than a local because _draw_dots needs the same number and is called without it. The marching dots take the meter's place, so a shift applied to one and not the other would draw them under the name. Nothing is reserved in Lite, which tracks no foreground window - the row on a Mac is exactly what it always was. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 12 +++++++ flow/ui.py | 84 +++++++++++++++++++++++++++++++++++++++++++--- tests/test_pill.py | 76 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 167 insertions(+), 5 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index f28c415..2e13893 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -884,6 +884,18 @@ that holds what was just sent, so it is somewhere you are already looking. The clipboard is restored about 0.6 s after the paste, so Flow does not permanently own it. +### Where it is aimed + +The left of the row names the window the paste is going into — `Notepad`, `Claude`, +`Code`. It comes free: Flow already resolves the foreground process to decide whether the +target is a terminal, on the edge rather than every frame, so nothing extra is asked for. + +The slot is a fixed width whether or not the name fills it, so changing window moves +nothing else on the row. Long names are cut at the end, because an application is +recognised by the start of its name. + +Absent in Lite, which does not track a foreground window at all. + ### The icons on the row Three marks sit between the meter and the status word, so the settings that matter diff --git a/flow/ui.py b/flow/ui.py index 954923b..631347a 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1097,6 +1097,31 @@ def _unload_fonts() -> None: #: whole body lines (`_settled_h`): a height that can only change when the text gains or #: loses a *line* changes a handful of times an utterance, and a timer that has to be #: cancelled correctly from a render loop is a thing to get wrong. +#: The slot at the left of the row that names the window Flow is aimed at. +#: +#: **The name first, the icon later.** Asked for as an icon — "if i am on notepad i see +#: notepad icon and when i am on claude ide i see claude icon" — and the name is the half +#: that costs nothing: `_track_target` already resolves the foreground process for the +#: paste, on the edge rather than per frame, so `session.target_app` is sitting there +#: reading `claude.exe`. The picture needs `ExtractIconExW`, then `GetIconInfo` and +#: `GetDIBits` to get pixels into a `PhotoImage`, which is a different size of job. +#: +#: **Fixed width, so nothing moves.** A slot that sized itself to the name would shift +#: the mic, the meter and every icon each time you changed window — which is the motion +#: this surface spent a night removing. Names longer than it fits are cut, because a +#: layout that stays still is worth more than the tail of a process name. +#: +#: Reserved only where there is something to put in it: Lite does not track the +#: foreground window at all, so on a Mac the row is exactly what it was. +APP_SLOT_W = 72 +APP_SLOT_GAP = 10 + +#: How many characters fit. Counted rather than measured, because `_draw` runs thirty +#: times a second and `bbox` on every frame to bound a string that changes a few times an +#: hour is a poor trade. Ten is conservative at the 11 px note font — the slot is 72 px +#: and ten characters of it measure under 60. +APP_NAME_CHARS = 10 + #: Room for the three icons that sit between the meter and the status word: settings, #: voice, mode. #: @@ -1573,6 +1598,34 @@ def _mix(a: str, b: str, t: float) -> str: ) +def app_label(process: str) -> str: + """`claude.exe` -> `Claude`. What the row shows for the window Flow is aimed at. + + The extension goes because it is noise in a name; the capital is added only when the + stem is entirely lower case, so `Code` and `WindowsTerminal` keep the shape their + authors gave them and `notepad` gets the one it deserves. + + Cut at the end, not the start. That is the opposite of what the draft body does, and + for the opposite reason: a draft is windowed to its tail because the newest words are + the ones being spoken, while an application is recognised by its *head* — + `WindowsTe…` is obviously Windows Terminal and `…sTerminal` is obviously + nothing. + """ + # A non-string is no name. `session.target_app` is `""` until the first foreground + # window resolves, and a `Mock` in any fixture that builds a pill with `__new__` — + # neither of which should reach the string operations below. + if not isinstance(process, str): + return "" + stem = process.rsplit(".", 1)[0].strip() + if not stem: + return "" + if stem.islower(): + stem = stem[:1].upper() + stem[1:] + if len(stem) > APP_NAME_CHARS: + return stem[:APP_NAME_CHARS - 1] + "…" + return stem + + def _gear(c: tk.Canvas, cx: float, cy: float, colour: str, tags) -> None: """A settings gear, drawn rather than fonted. @@ -4049,9 +4102,19 @@ def _draw(self) -> None: seam = "top" _panel_chrome(c, w, PILL_H, radius, self.ring_color, seam=seam) + # The window Flow is aimed at, named at the left of the row. A fixed-width slot + # whether or not the name fills it, so changing window moves nothing — see + # `APP_SLOT_W`. Zero in Lite, which tracks no foreground window, so the row there + # is exactly what it always was. + shift = self._row_shift() + if shift: + c.create_text(PAD, PILL_H // 2, anchor="w", + text=app_label(self.session.target_app), + fill=MUTED, font=FONT_NOTE) + # Mic glyph: capsule + stand, drawn rather than fonted so there is no # dependency on an emoji font being present and correctly sized. - cx, cy = 22, PILL_H // 2 + cx, cy = 22 + shift, PILL_H // 2 c.create_oval(cx - 4, cy - 9, cx + 4, cy + 1, fill=accent, outline=accent) c.create_arc( cx - 7, cy - 5, cx + 7, cy + 6, start=180, extent=180, @@ -4083,7 +4146,7 @@ def _draw(self) -> None: shade = accent if lvl > 0.04 else MUTED for i in range(BARS): h = self._bar_half_height(i, lvl) - x = METER_X + i * (BAR_W + BAR_GAP) + x = METER_X + shift + i * (BAR_W + BAR_GAP) # Rounded caps, radius half the bar width — FluidVoice draws its bars as # `RoundedRectangle(cornerRadius: barWidth / 2)`, and at four pixels wide # that is the difference between a meter and a bar chart. Squared off @@ -4100,10 +4163,21 @@ def _draw(self) -> None: # and dictation". Only when there is room — a narrow pill would otherwise draw # them over the status word, and the word is the thing that says whether Flow is # listening. - if w >= PILL_W + 3 * (ICON_SIZE + ICON_GAP): - _row_icons(c, self, METER_X + METER_W + LABEL_GAP, mid) + if w >= PILL_W + shift + 3 * (ICON_SIZE + ICON_GAP): + _row_icons(c, self, METER_X + METER_W + LABEL_GAP + shift, mid) self._draw_label(c, w, mid, accent) + def _row_shift(self) -> int: + """How far the mic, the meter and the icons move right for the app-name slot. + + Its own method because `_draw_dots` needs the same number and is called without + it — the marching dots occupy the meter's place, so a shift applied to one and + not the other would draw them under the app name. + """ + if self.lite or not app_label(getattr(self.session, "target_app", "")): + return 0 + return APP_SLOT_W + APP_SLOT_GAP + def _draw_dots(self, c: tk.Canvas, mid: int, accent: str) -> None: """Three marching dots in the slot the meter vacates (§07). @@ -4115,7 +4189,7 @@ def _draw_dots(self, c: tk.Canvas, mid: int, accent: str) -> None: binary-transparent and have nothing to composite against. See `_mix`. """ span = 2 * DOT_R - x = METER_X + (METER_W - (3 * span + 2 * DOT_GAP)) // 2 + x = METER_X + self._row_shift() + (METER_W - (3 * span + 2 * DOT_GAP)) // 2 for i in range(3): dx = x + i * (span + DOT_GAP) + DOT_R c.create_oval(dx - DOT_R, mid - DOT_R, dx + DOT_R, mid + DOT_R, diff --git a/tests/test_pill.py b/tests/test_pill.py index 3613825..7969bbb 100644 --- a/tests/test_pill.py +++ b/tests/test_pill.py @@ -1421,3 +1421,79 @@ def test_they_are_skipped_on_a_pill_too_narrow_to_hold_them(self): if __name__ == "__main__": # pragma: no cover unittest.main() + +class TestTheWindowFlowIsAimedAt(unittest.TestCase): + """The name of the app that will receive the paste, at the left of the row. + + Asked for as an *icon* — "if i am on notepad i see notepad icon and when i am on + claude ide i see claude icon" — and the name is the half that costs nothing. + `_track_target` already resolves the foreground process for the paste, on the edge + rather than per frame, so `session.target_app` is already sitting there reading + `claude.exe`. The picture needs `ExtractIconExW`, `GetIconInfo` and `GetDIBits` to + get pixels into a `PhotoImage`, which is a different size of job. + """ + + def drawn(self, target="notepad.exe", lite=False): + p = pill(armed=True, lite=lite) + p.session.target_app = target + p._draw() + return p + + def test_the_name_is_drawn_at_the_left_of_the_row(self): + p = self.drawn() + self.assertIn("Notepad", [text for _x, _y, text, _f in p.canvas.texts]) + + def test_the_extension_goes_and_a_bare_stem_gets_a_capital(self): + self.assertEqual(ui.app_label("notepad.exe"), "Notepad") + + def test_a_name_its_author_capitalised_is_left_alone(self): + # `Code` and `WindowsTerminal` keep the shape they were given. + self.assertEqual(ui.app_label("Code.exe"), "Code") + + def test_a_long_name_is_cut_at_the_end_not_the_start(self): + """The opposite of what the draft body does, for the opposite reason: a draft is + windowed to its tail because the newest words are the ones being spoken, while an + application is recognised by its head.""" + shown = ui.app_label("WindowsTerminal.exe") + self.assertTrue(shown.startswith("WindowsTe"), shown) + self.assertEqual(len(shown), ui.APP_NAME_CHARS) + + def test_a_non_string_target_is_no_name(self): + # `target_app` is "" until the first foreground window resolves, and a Mock in + # any fixture that builds a pill with `__new__`. + self.assertEqual(ui.app_label(mock.Mock()), "") + self.assertEqual(ui.app_label(""), "") + + def test_the_slot_is_the_same_width_whatever_the_name_is(self): + """A slot that sized itself to the name would shift the mic, the meter and every + icon each time you changed window — the motion this surface spent a night + removing.""" + short = self.drawn("vi.exe") + long_ = self.drawn("WindowsTerminal.exe") + self.assertEqual(short._row_shift(), long_._row_shift()) + + def test_the_meter_moves_over_to_make_room(self): + bare = self.drawn("") + named = self.drawn("notepad.exe") + self.assertEqual(named._row_shift() - bare._row_shift(), + ui.APP_SLOT_W + ui.APP_SLOT_GAP) + self.assertGreater(min(r[0] for r in named.canvas.rects), + min(r[0] for r in bare.canvas.rects)) + + def test_lite_reserves_nothing_because_it_tracks_nothing(self): + # `_track_target` returns early in Lite, so `target_app` never fills — the row on + # a Mac is exactly what it always was. + self.assertEqual(self.drawn("notepad.exe", lite=True)._row_shift(), 0) + + def test_no_target_reserves_nothing_either(self): + self.assertEqual(self.drawn("")._row_shift(), 0) + + def test_the_marching_dots_move_with_the_meter(self): + """`_draw_dots` takes the meter's place and is called without the shift, so one + computed and the other not would draw the dots under the app name.""" + p = self.drawn() + p.canvas.ovals.clear() + p._draw_dots(p.canvas, ui.PILL_H // 2, "#FFFFFF") + self.assertGreater(min(o[0] for o in p.canvas.ovals), + ui.METER_X + ui.APP_SLOT_W) + From 9cc2148157334777d7d61bae34cb831a1cb45a51 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 04:12:47 -0500 Subject: [PATCH 38/41] Draw a gear that looks like a gear, colour the row, and round the shell Three things from looking at the real row beside the mock-up. The settings icon was eight spokes on a ring, which is what an asterisk looks like - the report was "First icon is settings ... so make it settings icon". A gear is a solid body with trapezoidal teeth and a hole through the middle, and at sixteen pixels the hole is what carries it. Nothing here composites, so the hub is punched out by drawing SHELL over the middle, which is exact because SHELL is what is behind it. The three icons take a hue each - gold, cyan, pink - at one chroma and lightness in oklch so none shouts over the others. They are deliberately clear of every hue that already means something: green is capturing, blue is waiting, violet is the conversation card and red is an error, and a settings gear in Flow's green would read as "listening". The mic and meter beside them keep `accent`, because those two *are* the state readout. The shell was 8 px and read, in the owner's words, as "square box". 18 px is the radius the reference overlay actually has. PAD stayed at 14: worth checking rather than assuming, and at this radius the arc never reaches the text. The seeded design canvas moved to design/build/ and out of the sdist - 2.5 MB of editor around a few KB of design put the package 15% over its own size guard, which is the test that caught it. design/*.dc.html are the sources; the seeder rebuilds the rest. Co-Authored-By: Claude Opus 5 --- .gitignore | 3 + design/Lines.dc.html | 127 +++++++++++++++++++++++++++++++++++ design/Main.dc.html | 153 +++++++++++++++++++++++++++++++++++++++++++ design/Today.dc.html | 99 ++++++++++++++++++++++++++++ design/canvas.json | 17 +++++ flow/ui.py | 84 ++++++++++++++++++++---- pyproject.toml | 5 ++ 7 files changed, 474 insertions(+), 14 deletions(-) create mode 100644 design/Lines.dc.html create mode 100644 design/Main.dc.html create mode 100644 design/Today.dc.html create mode 100644 design/canvas.json diff --git a/.gitignore b/.gitignore index 20d9559..92dbe4d 100644 --- a/.gitignore +++ b/.gitignore @@ -37,3 +37,6 @@ __pycache__/ # looking at once and never worth carrying in every clone forever. Point it at a tracked # directory with --out when a particular set is worth keeping. .shots/ + +# Seeded design canvases: build products of design/*.dc.html. +design/build/ diff --git a/design/Lines.dc.html b/design/Lines.dc.html new file mode 100644 index 0000000..cf8a41f --- /dev/null +++ b/design/Lines.dc.html @@ -0,0 +1,127 @@ + + + + + + + + + + + + + +
+ + +
+
{{case.cap}}
+
+
+
+
+
+

{{case.text}}

+
+
+
+
+ + + + +
+
+ + + +
+
+ + + +
+
+ + + +
+
+
+
+
Send
+
+
+
+

{{case.why}}

+
+ + +
+ + + + diff --git a/design/Main.dc.html b/design/Main.dc.html new file mode 100644 index 0000000..9c089c4 --- /dev/null +++ b/design/Main.dc.html @@ -0,0 +1,153 @@ + + + + + + + + + + + + + +
+
+ + +
+ + +
+
+

This is really interesting. The commands moved to the corner, + so the words get the width they were always competing for.

+
+
+
+ + + + +
+
+ + + +
+
+ + + +
+
+ + + +
+
+
+ + +
+
dropped 'So' (filler, ns=0.63 lp=-0.53, final)
+
Send
+
+
+ + +
+
WindowsTe…
+ + + + +
+ +
+ +
+ + + + + + + + + + + +
HELD
+
+ +
+
+ + + + diff --git a/design/Today.dc.html b/design/Today.dc.html new file mode 100644 index 0000000..b6043bb --- /dev/null +++ b/design/Today.dc.html @@ -0,0 +1,99 @@ + + + + + + + + + + + + + +
+
Today
+
+ +
+

This is really interesting.

+
dropped 'So' (filler, ns=0.63 lp=-0.53, final)
+
+
Refine
+
Continue
+
+
Send
+
+
+ +
+
WindowsTe…
+ + + + +
+ +
+ +
+ + + + + + + + + + + +
HELD
+
+ +
+
+ + + + diff --git a/design/canvas.json b/design/canvas.json new file mode 100644 index 0000000..c472c5c --- /dev/null +++ b/design/canvas.json @@ -0,0 +1,17 @@ +{ + "artboards": [ + { "file": "Today.dc.html", "x": 0, "y": 0, "w": 500, "h": 300 }, + { "file": "Main.dc.html", "x": 600, "y": 0, "w": 500, "h": 320 }, + { "file": "Lines.dc.html", "x": 0, "y": 460, "w": 520, "h": 760 } + ], + "annotations": [ + { + "id": "brief", + "x": 600, + "y": 400, + "w": 500, + "text": "Commands become icons in the top-right corner.\nSend keeps its own place at the bottom right.\nThe text area holds three lines whether or not you filled them." + } + ], + "launch": { "view": "canvas" } +} diff --git a/flow/ui.py b/flow/ui.py index 631347a..6731563 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1122,6 +1122,35 @@ def _unload_fonts() -> None: #: and ten characters of it measure under 60. APP_NAME_CHARS = 10 +#: The shell's corner radius. +#: +#: 8 px until somebody looked at it beside the thing it is modelled on and said "the +#: window currently it's square box". They were right: at 420x224 an 8 px corner is a +#: rounded rectangle in name only, and FluidVoice's overlay — the reference this surface +#: has been chasing all along — is visibly a soft-cornered slab. +#: +#: `PAD` stayed at 14, which was worth checking rather than assuming: at this radius the +#: curve only bites the first ~5 px of each edge, and the body's first line and the row's +#: app name both start below and right of that. The mock-up moved padding to 16 because +#: its corner is drawn by CSS on the frame itself; here the chrome is drawn *inside* the +#: canvas and the arc never reaches the text. +PANEL_R = 18 + +#: One hue per row icon, all at the same chroma and lightness in oklch so no control +#: shouts louder than another: `oklch(0.80 0.12 H)` at 85, 200 and 340. +#: +#: **Deliberately clear of every hue that already means something here.** Green is +#: capturing, blue is waiting, violet is the conversation card and red is an error — a +#: settings gear in Flow's green would read as "listening", which is the one thing a +#: control must never do. So the three sit in gaps the state palette does not use. +#: +#: The mic and the meter beside them stay on `accent`, because those two *are* the state +#: readout. Colour on the row means "this is a control"; colour on the meter means "this +#: is what Flow is doing". +ICON_SETTINGS = "#E1B75C" # gold +ICON_VOICE = "#43D5DC" # cyan +ICON_MODE = "#F19FD6" # pink + #: Room for the three icons that sit between the meter and the status word: settings, #: voice, mode. #: @@ -1626,22 +1655,49 @@ def app_label(process: str) -> str: return stem +#: The gear's three radii, as fractions of `ICON_SIZE`: the tooth tip, the body it sits +#: on, and the hole in the middle. +_GEAR_TIP, _GEAR_BODY, _GEAR_HUB = 0.47, 0.34, 0.14 + +#: Half a tooth, in radians, at the body and at the tip. Narrower at the tip is what +#: makes a tooth a tooth rather than a spoke. +_GEAR_WIDE, _GEAR_NARROW = 0.21, 0.13 + + def _gear(c: tk.Canvas, cx: float, cy: float, colour: str, tags) -> None: """A settings gear, drawn rather than fonted. Same reasoning as the mic glyph beside it: a font that is missing, substituted or scaled differently turns a control into a box, and the one thing every control on - this row has to be is recognisable. Eight teeth as short radial spokes, because at - sixteen pixels a toothed outline reads as a smudge and spokes read as a gear. + this row has to be is recognisable. + + **The first version was spokes on a ring and it read as a sun.** Eight lines poking + out of a circle is what an asterisk looks like; a gear is a solid body with + *trapezoidal* teeth and a hole through the middle, and at sixteen pixels the hole is + what carries it. So: eight tapered quads on a filled disc, then the hub punched back + out in `SHELL`. Nothing here composites — the canvas has no alpha — so punching a + hole means drawing the background colour over the middle, which is exact as long as + the row's fill is the one behind it. """ - r = ICON_SIZE / 2 + size = ICON_SIZE for i in range(8): a = math.pi * i / 4 - c.create_line(cx + math.cos(a) * (r - 4), cy + math.sin(a) * (r - 4), - cx + math.cos(a) * r, cy + math.sin(a) * r, - fill=colour, width=2, tags=tags) - c.create_oval(cx - r + 3, cy - r + 3, cx + r - 3, cy + r - 3, - outline=colour, width=2, tags=tags) + corners = [] + for radius, half in ((_GEAR_BODY, _GEAR_WIDE), (_GEAR_TIP, _GEAR_NARROW)): + for side in (-1, 1): + angle = a + side * half + corners.append((cx + math.cos(angle) * radius * size, + cy + math.sin(angle) * radius * size)) + # body-left, body-right, tip-right, tip-left: a quad walked in order, so the + # tooth is a trapezoid rather than a bow tie. + c.create_polygon(*corners[0], *corners[1], *corners[3], *corners[2], + fill=colour, outline=colour, tags=tags) + body = _GEAR_BODY * size + c.create_oval(cx - body, cy - body, cx + body, cy + body, + fill=colour, outline=colour, tags=tags) + hub = _GEAR_HUB * size + c.create_oval(cx - hub, cy - hub, cx + hub, cy + hub, + fill=SHELL, outline=SHELL, tags=tags) def _speaker(c: tk.Canvas, cx: float, cy: float, colour: str, muted: bool, tags) -> None: @@ -1705,17 +1761,17 @@ def _row_icons(c: tk.Canvas, pill, x: float, mid: float, tags="row") -> float: def hit(tag: str, command) -> None: c.tag_bind(tag, "", lambda _e: command()) - _gear(c, x + ICON_SIZE / 2, mid, MUTED, ("row-gear", tags)) + _gear(c, x + ICON_SIZE / 2, mid, ICON_SETTINGS, ("row-gear", tags)) hit("row-gear", getattr(pill, "open_settings", lambda: None)) x += ICON_SIZE + ICON_GAP if getattr(session, "speaker", None) is not None: - _speaker(c, x + ICON_SIZE / 2, mid, MUTED, + _speaker(c, x + ICON_SIZE / 2, mid, ICON_VOICE, bool(getattr(session, "muted", False)), ("row-voice", tags)) hit("row-voice", getattr(session, "toggle_speech", lambda: None)) x += ICON_SIZE + ICON_GAP - _mode_glyph(c, x + ICON_SIZE / 2, mid, MUTED, converse, ("row-mode", tags)) + _mode_glyph(c, x + ICON_SIZE / 2, mid, ICON_MODE, converse, ("row-mode", tags)) hit("row-mode", getattr(session, "toggle_mode", lambda: None)) return x + ICON_SIZE + ICON_GAP @@ -4096,7 +4152,7 @@ def _draw(self) -> None: # could end up on either side of the pill when there was no room above. There # is one window now and the band is always the top of it, so the answer is a # constant and the flag that carried it is gone. - radius = (0, 0, 8, 8) + radius = (0, 0, PANEL_R, PANEL_R) # The row is the lower surface, so it draws nothing on the join — the band's # bottom carries the single line. seam = "top" @@ -4899,7 +4955,7 @@ def _render(self) -> None: # side — `reposition` is what decides above-vs-below, since it already has to. # Always the top band of the one window now: rounded head, squared foot on the # join it shares with the pill row below it. - corners = (8, 8, 0, 0) + corners = (PANEL_R, PANEL_R, 0, 0) _panel_chrome(c, CARD_W, self._h, corners, self.ring_color, seam="bottom") @@ -5716,7 +5772,7 @@ def _render(self) -> None: # side — `reposition` is what decides above-vs-below, since it already has to. # Always the top band of the one window now: rounded head, squared foot on the # join it shares with the pill row below it. - corners = (8, 8, 0, 0) + corners = (PANEL_R, PANEL_R, 0, 0) _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, seam="bottom") y = PAD diff --git a/pyproject.toml b/pyproject.toml index 9c508f7..fdac40d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -112,6 +112,11 @@ exclude = [ "NEEDS_YOU.md", "docs/decisions.md", "docs/history/", + # A seeded design canvas is ~2.5 MB of editor bundled around a few KB of design, and + # it is a *build product* — `design/*.dc.html` plus `canvas.json` are the sources, + # and the seeder rebuilds it from them. One of these alone put the sdist 15% over its + # own size guard, which is the test that caught it. + "design/build/", ] [tool.uv] From 62efc5cb756c2994635a8e4974f66e8c614fa94c Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 04:56:24 -0500 Subject: [PATCH 39/41] Commands become marks in the corner, on both surfaces Asked for whole: "those commands to go as icon on top right and send also settings and three icon aligh to right near help and go through each screen and ensure round and commands to icon in pattern adopted for seemless experience." Refine, Continue, Edit and Was a command - and the card's Use this, Copy and New conversation - are round marks in the panel's top-right corner. Send keeps the foot to itself, and Ask and Done do the same on their surfaces: the one control you cannot take back stays a word, and stays where the hand already goes. Cancel became an x, and Was a command wears the loop, which was the owner's suggestion and the right one. The cluster is laid out right-to-left from the panel's edge, so the rightmost mark is at a fixed address whatever is beside it. The set changes constantly - Edit and Was a command come and go with what was said - and grown from the left it would shift every mark under the hand each time, which is the defect the primary's fixed address already exists to prevent. command_x is a function so that property is testable rather than drawn. A flaw in my own mock-up, found by building it: putting the marks *beside* the text makes the column ~134 px narrower on every line, which is the opposite of what moving them was supposed to buy. They take a band above it instead - 34 px once, and the draft keeps its full width. The band is furniture, so PANEL_MAX_H grew by it rather than the draft paying. Three glyphs took two attempts each and the pictures are why. A wand as a stroke and two dots reads as a slash with specks; a 2 px nib on a 2 px stroke merges into a diagonal; and the command loop at 2 px on a 6 px ring leaves a 2 px hole and renders as a smudge - as arcs and again as closed rings. It is drawn at 1.4 px on a 6.8 px loop now, thinner than every other mark here, which is what makes it read as a knot. The row's three icons moved to the right edge beside the status word. That is the better address as well as the asked-for one: the meter's right edge is read as part of the meter, while the panel's right edge is the one landmark on that row that never moves. Two test helpers now read commands off tag_bind rather than off drawn text. Which primitives a fixture records varies by glyph - Copy draws rectangles and Use this draws lines - so tag-spotting made one visible and the other not. The binding is the hit region, which is what those tests were always about. Co-Authored-By: Claude Opus 5 --- docs/guide.md | 19 +++ flow/ui.py | 291 +++++++++++++++++++++++++++++++++++-------- tests/test_bubble.py | 5 +- tests/test_card.py | 53 +++++--- tests/test_editor.py | 30 +++-- 5 files changed, 319 insertions(+), 79 deletions(-) diff --git a/docs/guide.md b/docs/guide.md index 2e13893..b1386b0 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -896,6 +896,25 @@ recognised by the start of its name. Absent in Lite, which does not track a foreground window at all. +### The commands, as marks in the corner + +Refine, Continue, Edit, Was a command — and on the card, Use this, Copy and New +conversation — are round marks in the panel's top-right corner rather than a row of words +along the foot. `Was a command` wears **⌘**, which is the mark everybody already reads as +*this was an instruction, not text*. + +**Send is the exception, and deliberately.** It keeps the foot to itself, on the right, +labelled. It is the one control you cannot take back, so it stays a word and stays where +your hand already goes. `Ask` and `Done` do the same on the surfaces they belong to. + +The cluster is laid out right-to-left from the panel's edge, so the rightmost mark is at a +fixed address whatever else is beside it. The set changes constantly — Edit and Was a +command come and go with what was said — and grown from the left it would shift every +mark under your hand each time. + +They sit in a band of their own *above* the draft, not beside it: a cluster of four takes +~134 px, and a text column beside it would lose a third of its width on every line. + ### The icons on the row Three marks sit between the meter and the status word, so the settings that matter diff --git a/flow/ui.py b/flow/ui.py index 6731563..cf7f8d0 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1167,7 +1167,28 @@ def _unload_fonts() -> None: ICON_SIZE = 16 ICON_GAP = 12 -PANEL_MAX_H = 184 +#: A glyph for every secondary command, so the row of words above the draft becomes a +#: cluster of marks in the corner. +#: +#: **They sit in a band of their own, above the text and right-aligned, not beside it.** +#: The first sketch put them next to the words, and drawing it made the flaw plain: a +#: cluster of four takes ~134 px, and a body column beside it loses a third of its width +#: on *every* line — the opposite of what moving them was supposed to buy. A band costs +#: 34 px of height once, and the draft keeps all 392 px of its column. +#: +#: Every glyph is drawn, like the mic and the meter: a font that is missing or +#: substituted turns a control into a box. +COMMAND_H = 26 +COMMAND_GAP = 6 + +#: What the cluster costs the panel: the band, plus the air under it. +COMMAND_BAND = COMMAND_H + 8 + +#: The band of command marks is furniture, not content, so it goes *on top of* the +#: ceiling rather than out of the draft's share — the same call `SETTINGS_H` needed, and +#: the same failure if it is not made: the live partial's own cap is a flat 70 px, and on +#: a panel pegged at the old ceiling it runs straight through the note and the foot. +PANEL_MAX_H = 184 + COMMAND_BAND #: The shortest a band gets, so a one-word draft still has a panel rather than a sliver. PANEL_MIN_H = 96 @@ -1741,6 +1762,123 @@ def _mode_glyph(c: tk.Canvas, cx: float, cy: float, colour: str, converse: bool, c.create_line(cx - r, y, cx - r + width, y, fill=colour, width=2, tags=tags) +def command_x(slot: int, right: float = None) -> float: + """The right edge of the command mark in `slot`, counting from the rightmost as 0. + + Laid out right-to-left from the panel's right edge, and that is the whole reason this + is a function worth naming. The set of secondaries changes constantly — Edit and + Was a command come and go with what was said — so a cluster grown from the *left* + would shift every mark under the hand each time the set changed. Anchored on the + right, the rightmost mark is at a fixed address whatever is beside it, which is the + argument the primary chip already won at the foot. + """ + if right is None: + right = BUBBLE_W - PAD + return right - slot * (COMMAND_H + COMMAND_GAP) + + +def _glyph_refine(c, x, y, colour, tags) -> None: + """A wand with a spark at its tip: this rewrites what you said. + + The first attempt was a stroke and two dots, and at sixteen pixels that reads as a + slash with specks on it. A four-point spark — two crossed strokes, the vertical + longer — is what carries "magic" at this size, so the wand got shorter to make room + for it. + """ + c.create_line(x + 2.5, y + 13.5, x + 9, y + 7, fill=colour, width=2, tags=tags) + c.create_line(x + 11.5, y + 1.5, x + 11.5, y + 8.5, fill=colour, width=2, tags=tags) + c.create_line(x + 8, y + 5, x + 15, y + 5, fill=colour, width=2, tags=tags) + + +def _glyph_continue(c, x, y, colour, tags) -> None: + """A plus: keep going, and add to what is there.""" + c.create_line(x + 8, y + 3, x + 8, y + 13, fill=colour, width=2, tags=tags) + c.create_line(x + 3, y + 8, x + 13, y + 8, fill=colour, width=2, tags=tags) + + +def _glyph_edit(c, x, y, colour, tags) -> None: + """A pencil, nib down-left. + + The body is drawn as a thick stroke and the nib as a triangle *past* its end, which + is the whole difference between a pencil and a diagonal line — the first version put + a 2 px nib on a 2 px stroke and the two merged. + """ + c.create_line(x + 5.5, y + 10.5, x + 12, y + 4, fill=colour, width=3, tags=tags) + c.create_line(x + 10.5, y + 2.5, x + 13.5, y + 5.5, fill=colour, width=2, tags=tags) + c.create_polygon(x + 2, y + 14, x + 3.4, y + 9.4, x + 6.6, y + 12.6, + fill=colour, outline=colour, tags=tags) + + +def _glyph_command(c, x, y, colour, tags) -> None: + """The command loop — the owner's suggestion, and the right one. + + "for command universally we can use ⌘". It is the mark everybody already reads as + *this was an instruction, not text*, which is exactly what the chip meant. + + Four open loops on the corners of a square, which is the knot itself: each arc + starts and ends *on* the square's edges, so the two read as one continuous line. + + **The stroke is thinner than every other mark here, and that is the whole trick.** At + 2 px on a 6 px loop the hole is 2 px across and the glyph renders as a smudge — it + did, twice, once as arcs and once as closed rings. 1.4 px on a 6.8 px loop leaves 4 px + of air, which is what makes it read as ⌘ rather than as four blobs. + + Tk's arc angles are degrees counterclockwise from 3 o'clock. Each loop's 90° gap + faces the square, so the top-left starts at 0 and sweeps 270 (east round to south), + leaving the south-east quadrant open for the corner it joins. + """ + r, ring = 3.4, 1.4 + for cx, cy, start_deg in ((5, 5, 0), (11, 5, 270), (5, 11, 90), (11, 11, 180)): + c.create_arc(x + cx - r, y + cy - r, x + cx + r, y + cy + r, + start=start_deg, extent=270, style=tk.ARC, + outline=colour, width=ring, tags=tags) + c.create_rectangle(x + 5, y + 5, x + 11, y + 11, + outline=colour, width=ring, fill="", tags=tags) + + +def _glyph_cancel(c, x, y, colour, tags) -> None: + c.create_line(x + 4, y + 4, x + 12, y + 12, fill=colour, width=2, tags=tags) + c.create_line(x + 12, y + 4, x + 4, y + 12, fill=colour, width=2, tags=tags) + + +def _glyph_take(c, x, y, colour, tags) -> None: + """An arrow down into a line: put this answer into the draft.""" + c.create_line(x + 8, y + 2, x + 8, y + 9, fill=colour, width=2, tags=tags) + c.create_line(x + 4.5, y + 6, x + 8, y + 9.5, fill=colour, width=2, tags=tags) + c.create_line(x + 11.5, y + 6, x + 8, y + 9.5, fill=colour, width=2, tags=tags) + c.create_line(x + 3, y + 13, x + 13, y + 13, fill=colour, width=2, tags=tags) + + +def _glyph_copy(c, x, y, colour, tags) -> None: + """Two sheets, the near one offset — the mark every OS uses for copy.""" + c.create_rectangle(x + 2.5, y + 2.5, x + 9.5, y + 9.5, + outline=colour, width=2, fill="", tags=tags) + c.create_rectangle(x + 6.5, y + 6.5, x + 13.5, y + 13.5, + outline=colour, width=2, fill=SHELL, tags=tags) + + +def _glyph_new(c, x, y, colour, tags) -> None: + """A speech bubble with a plus: start the conversation again.""" + _round_rect(c, x + 2, y + 3, x + 14, y + 11, 3, fill="", outline=colour, tags=tags) + c.create_line(x + 5, y + 11, x + 4, y + 14, fill=colour, width=2, tags=tags) + c.create_line(x + 8, y + 5, x + 8, y + 9, fill=colour, width=2, tags=tags) + c.create_line(x + 6, y + 7, x + 10, y + 7, fill=colour, width=2, tags=tags) + + +#: Key -> glyph. A command with no entry here keeps its word, which is the honest +#: fallback: a mark nobody can read is worse than a label that is merely longer. +COMMAND_GLYPHS = { + "Refine": _glyph_refine, + "Continue": _glyph_continue, + "Edit": _glyph_edit, + "Was a command": _glyph_command, + "Cancel": _glyph_cancel, + "Use this": _glyph_take, + "Copy": _glyph_copy, + "New conversation": _glyph_new, +} + + def _row_icons(c: tk.Canvas, pill, x: float, mid: float, tags="row") -> float: """Settings, voice and mode, between the meter and the status word. @@ -4219,8 +4357,15 @@ def _draw(self) -> None: # and dictation". Only when there is room — a narrow pill would otherwise draw # them over the status word, and the word is the thing that says whether Flow is # listening. - if w >= PILL_W + shift + 3 * (ICON_SIZE + ICON_GAP): - _row_icons(c, self, METER_X + METER_W + LABEL_GAP + shift, mid) + # Against the right edge, beside the status word, rather than trailing the meter. + # Asked for that way — "settings and three icon aligh to right near help" — and + # it is the better address: the meter's right edge is where the *bars* end, which + # moves with nothing but is read as part of the meter, while the right edge is + # the one landmark on this row that never moves at all. + icons_w = 3 * ICON_SIZE + 2 * ICON_GAP + icons_x = w - LABEL_PAD - LABEL_SLOT_W - LABEL_GAP - icons_w + if icons_x >= METER_X + METER_W + shift + LABEL_GAP: + _row_icons(c, self, icons_x, mid) self._draw_label(c, w, mid, accent) def _row_shift(self) -> int: @@ -4726,7 +4871,7 @@ def _show(self) -> None: def _view_h(self) -> int: """What is left for the history once the pinned block and chips have theirs.""" - return max(0, self._h - PAD - self._pinned_h - HELP_FOOT_BAND) + return max(0, self._h - PAD - COMMAND_BAND - self._pinned_h - HELP_FOOT_BAND) def _max_top(self) -> int: height = 0 @@ -4927,7 +5072,7 @@ def _render(self) -> None: # card was free to grow to it; with the card a fixed shape that let the answer be # sized for a 672 px window and drawn into a 184 px one, and the top of the card # — the "agent" label — was cut off by it. - spare = (self.panel_h() - PAD - HELP_FOOT_BAND - q_h - CARD_GAP + spare = (self.panel_h() - PAD - COMMAND_BAND - HELP_FOOT_BAND - q_h - CARD_GAP - BODY_ELIDED_H - (note_h + 4 if self._note else 0)) shown, more, a_h = "", 0, 0 if self._answer: @@ -4946,7 +5091,7 @@ def _render(self) -> None: # Snug around what is on the card, stepping a line at a time — and the pill # row below it does not move when it steps, because the shell grows upward. self._h = self._settled_h( - PAD + history_h + self._pinned_h + HELP_FOOT_BAND) + PAD + COMMAND_BAND + history_h + self._pinned_h + HELP_FOOT_BAND) c.configure(width=CARD_W, height=self._h) self.reposition() @@ -4960,7 +5105,8 @@ def _render(self) -> None: seam="bottom") # -- the history, in what is left above the pinned block - y, floor = PAD, PAD + self._view_h() + # Below the command band, which is furniture the history must not run under. + y, floor = PAD + COMMAND_BAND, PAD + COMMAND_BAND + self._view_h() self._top = min(self._top, self._max_top()) drawn = self._top for i in range(self._top, len(self._history)): @@ -5040,22 +5186,39 @@ def _chips(self) -> None: return self._chips_drawn = key_now c.delete("chips") - x = PAD - y2 = self._h - PAD - y1 = y2 - CHIP_H - for key, label, cmd in specs: - w = chip_w(key, label) - primary = key == "Ask" + + # The same shape the bubble takes: the secondaries as marks in the top-right + # corner, the primary alone at the foot. Two surfaces that laid their controls + # out differently would be two things to learn, and the whole point of moving + # them was one pattern — "commands to icon in pattern adopted for seemless + # experience". + for slot, (key, label, cmd) in enumerate(reversed(specs[1:])): + glyph = COMMAND_GLYPHS.get(key) tag = chip_tag(key) - _round_rect(c, x, y1, x + w, y2, 13, - fill=PRIMARY_FILL if primary else CHIP, - outline="", tags=(tag, "chips")) - c.create_text(x + w / 2, (y1 + y2) / 2, text=label, - fill=PRIMARY_TEXT if primary else CODE, - font=FONT_CHIP_PRIMARY if primary else FONT_CHIP, - tags=(tag, "chips")) + x2 = command_x(slot, CARD_W - PAD) + if glyph is None: + width = chip_w(key, label) + _round_rect(c, x2 - width, PAD, x2, PAD + COMMAND_H, 13, + fill=CHIP, outline="", tags=(tag, "chips")) + c.create_text(x2 - width / 2, PAD + COMMAND_H / 2, text=label, + fill=CODE, font=FONT_CHIP, tags=(tag, "chips")) + else: + _round_rect(c, x2 - COMMAND_H, PAD, x2, PAD + COMMAND_H, + COMMAND_H // 2, fill=CHIP, outline="", tags=(tag, "chips")) + glyph(c, x2 - COMMAND_H + (COMMAND_H - ICON_SIZE) / 2, + PAD + (COMMAND_H - ICON_SIZE) / 2, CODE, (tag, "chips")) c.tag_bind(tag, "", lambda _e, f=cmd: f()) - x += w + 8 + + key, label, cmd = specs[0] # Ask, and it is always first + width = chip_w(key, label) + y2 = self._h - PAD + y1 = y2 - CHIP_H + tag = chip_tag(key) + _round_rect(c, CARD_W - PAD - width, y1, CARD_W - PAD, y2, 13, + fill=PRIMARY_FILL, outline="", tags=(tag, "chips")) + c.create_text(CARD_W - PAD - width / 2, (y1 + y2) / 2, text=label, + fill=PRIMARY_TEXT, font=FONT_CHIP_PRIMARY, tags=(tag, "chips")) + c.tag_bind(tag, "", lambda _e, f=cmd: f()) def tick_countdown(self) -> None: """Repaint when the auto-ask number changes, and only then.""" @@ -5708,7 +5871,9 @@ def _render(self) -> None: # `SETTINGS_H` is in here for the reason everything else is: the body's budget is # what is left after the fixed furniture, and a strip the height did not know # about would be a strip drawn over the first line of the draft. - around = 74 + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + # `COMMAND_BAND` joined the furniture when the secondaries moved to the corner. + # A band the height did not know about is a band drawn over the first line. + around = 74 + COMMAND_BAND + BODY_ELIDED_H + (note_h + 4 if note_h else 0) if partial_h: around += partial_h + PARTIAL_GAP if self._sent: @@ -5763,7 +5928,7 @@ def _render(self) -> None: if not self._frozen(): # Snug around the draft again, stepping a line at a time rather than # tracking every frame — see `_settled_h`. - self._h = self._settled_h(text_h + extra + 74) + self._h = self._settled_h(text_h + extra + 74 + COMMAND_BAND) c.configure(width=BUBBLE_W, height=self._h) self.reposition() @@ -5775,7 +5940,7 @@ def _render(self) -> None: corners = (PANEL_R, PANEL_R, 0, 0) _panel_chrome(c, BUBBLE_W, self._h, corners, self.ring_color, seam="bottom") - y = PAD + y = PAD + COMMAND_BAND if self._sent: c.create_text( PAD, y, anchor="nw", text="sent", fill=MUTED, @@ -5941,47 +6106,63 @@ def _lay_out(self, specs) -> None: return self._chips_drawn = key_now c.delete("chips") - widths = [chip_w(key, label) for key, label, _c in specs] + # The *last* primary, so a row is still laid out sensibly if one ever carries two. pinned = max((i for i, (k, _l, _c) in enumerate(specs) if k in self.PRIMARY_KEYS), default=None) - # Dimming greys the primary's fill but must not move it: a row that reflowed - # every time a CLI call started would undo the whole point of a fixed address. heads = [i for i in range(len(specs)) if i != pinned] - gap = chip_row_gap( - [widths[i] for i in heads], - BUBBLE_W - PAD - (widths[pinned] + CHIP_GAP if pinned is not None else 0), - ) - y2 = self._h - PAD - y1 = y2 - CHIP_H - def draw(i: int, x: float) -> None: + # -- the secondaries, as a cluster of marks in the top-right corner ------------ + # + # They were a row of words along the foot, sharing it with the primary. Asked for + # as icons — "those commands to go as icon on top right and send" — and the move + # earns more than tidiness: the foot now holds one control, so the one thing you + # cannot take back has a row to itself. + # + # Right-aligned and laid out right-to-left, so the *rightmost* mark is at a fixed + # address whatever the set is. The set changes constantly — Edit and Was a command + # come and go with what was said — and a cluster that grew leftward from the left + # edge would move every icon under the hand each time. + right = BUBBLE_W - PAD + for slot, i in enumerate(reversed(heads)): key, label, cmd = specs[i] - primary = key in self.PRIMARY_KEYS and not dim + glyph = COMMAND_GLYPHS.get(key) tag = chip_tag(key) - _round_rect( - c, x, y1, x + widths[i], y2, 13, - fill=PRIMARY_FILL if primary else CHIP, outline="", - tags=(tag, "chips"), - ) - c.create_text( - x + widths[i] / 2, (y1 + y2) / 2, text=label, - fill=DISABLED if dim else (PRIMARY_TEXT if primary else CODE), - font=FONT_CHIP_PRIMARY if primary else FONT_CHIP, - tags=(tag, "chips"), - ) + x2 = command_x(slot, right) + y1 = PAD + if glyph is None: + # No mark for this one, so it keeps its word. A glyph nobody can read is + # worse than a label that is merely longer. + width = chip_w(key, label) + _round_rect(c, x2 - width, y1, x2, y1 + COMMAND_H, 13, + fill=CHIP, outline="", tags=(tag, "chips")) + c.create_text(x2 - width / 2, y1 + COMMAND_H / 2, text=label, + fill=DISABLED if dim else CODE, font=FONT_CHIP, + tags=(tag, "chips")) + right -= width - COMMAND_H # this one is wider than a slot + else: + _round_rect(c, x2 - COMMAND_H, y1, x2, y1 + COMMAND_H, COMMAND_H // 2, + fill=CHIP, outline="", tags=(tag, "chips")) + glyph(c, x2 - COMMAND_H + (COMMAND_H - ICON_SIZE) / 2, + y1 + (COMMAND_H - ICON_SIZE) / 2, + DISABLED if dim else CODE, (tag, "chips")) c.tag_bind(tag, "", lambda _e, f=cmd: f()) - x = PAD - for i in heads: - draw(i, x) - x += widths[i] + gap + # -- the primary, alone at the foot, exactly where it always was --------------- if pinned is not None: - # Never left of where the secondaries actually ended: with a row too wide to - # fit, honouring the pin would draw the primary *over* them. Overlapping - # chips are two hit regions in one place, which is worse than a row that - # ends early — and `chip_row_gap` has already floored the gap at 0 by then. - draw(pinned, max(x, BUBBLE_W - PAD - widths[pinned])) + key, label, cmd = specs[pinned] + width = chip_w(key, label) + y2 = self._h - PAD + y1 = y2 - CHIP_H + tag = chip_tag(key) + lit = not dim + _round_rect(c, BUBBLE_W - PAD - width, y1, BUBBLE_W - PAD, y2, 13, + fill=PRIMARY_FILL if lit else CHIP, outline="", + tags=(tag, "chips")) + c.create_text(BUBBLE_W - PAD - width / 2, (y1 + y2) / 2, text=label, + fill=PRIMARY_TEXT if lit else DISABLED, + font=FONT_CHIP_PRIMARY, tags=(tag, "chips")) + c.tag_bind(tag, "", lambda _e, f=cmd: f()) def tick_countdown(self) -> None: """Repaint when the auto-ask number changes, and only then. diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 3e08082..57edcd2 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -405,7 +405,10 @@ def test_but_a_changed_row_is_rebuilt(self): b.pill.session.can_rescue = True b._render() self.assertNotEqual(self.chips(b), was) - self.assertIn("Was a command", [i["text"] for i in b.canvas.items]) + # By its tag, not its word: the secondaries are marks in the corner now, so + # there is no label to look for — only the hit region a click has to land on. + tags = {t for i in b.canvas.items for t in (i.get("tags") or ())} + self.assertIn(ui.chip_tag("Was a command"), tags) def test_nothing_moves_or_resizes_under_the_pointer(self): # `_visible`, because a window nobody can see is a window nobody is pointing at diff --git a/tests/test_card.py b/tests/test_card.py index 3e9df7b..e7711ea 100644 --- a/tests/test_card.py +++ b/tests/test_card.py @@ -69,6 +69,27 @@ def card(**kw): return c +def commands(c) -> list[str]: + """Which commands are on the card, by their click binding rather than by their word. + + The secondaries are marks in the top-right corner now, so only the primary still + carries a label — a check that looked for text would report `Use this` as vanished + when it had merely stopped being a sentence. Read off `tag_bind` rather than off the + drawn items, because which primitives a fixture records varies by glyph: `Copy` draws + rectangles and `Use this` draws lines, so tag-spotting made one visible and the other + not. The binding is the hit region, which is what these tests are actually about. + """ + keys = ("Ask", "Use this", "Copy", "New conversation") + bound = {tag for tag, _seq, _fn in c.canvas.bindings} + return [k for k in keys if ui.chip_tag(k) in bound] + + +def primary_label(c) -> str: + """The word on the one command that still has one.""" + return next((i["text"] for i in c.canvas.items + if str(i.get("text", "")).startswith("Ask")), "") + + def drawn(c) -> list[str]: """Every string that reached the canvas as visible text, probes excluded. @@ -253,39 +274,39 @@ def test_the_chips_are_inside_the_window_however_tall_the_answer(self): c = card() c.ask("q") c.answer(prose(12_000)) - chips = [i for i in c.canvas.items - if i["text"] in ("Ask", "Use this", "Copy", "New conversation")] - self.assertEqual(len(chips), 4) - for chip in chips: - self.assertLess(chip["y"], c._h, chip["text"]) - self.assertGreater(chip["y"], 0, chip["text"]) + self.assertEqual(len(commands(c)), 4) + # Every command's mark is inside the window: the primary at the foot, the three + # secondaries in the band at the top. + for key in ("Ask", "Use this", "Copy", "New conversation"): + ys = [i["y"] for i in c.canvas.items + if ui.chip_tag(key) in (i.get("tags") or ()) and "y" in i] + for y in ys: + self.assertLess(y, c._h, key) + self.assertGreaterEqual(y, 0, key) class TestTheChips(unittest.TestCase): - def labels(self, c) -> list[str]: - keys = ("Ask", "Use this", "Copy", "New conversation") - return [i["text"] for i in c.canvas.items - if any(i["text"].startswith(k) for k in keys)] + def test_ask_is_there_before_anything_has_been_asked(self): c = card() c.show() - self.assertIn("Ask", self.labels(c)) + self.assertIn("Ask", commands(c)) def test_the_countdown_rides_on_the_ask_chip(self): c = card() c.pill.session.auto_ask_in = 2.4 c.ask("q") - self.assertIn("Ask 3s", self.labels(c)) + self.assertEqual(primary_label(c), "Ask 3s") def test_use_this_and_copy_appear_only_with_an_answer(self): c = card() c.ask("q") - self.assertNotIn("Use this", self.labels(c)) - self.assertNotIn("Copy", self.labels(c)) + self.assertNotIn("Use this", commands(c)) + self.assertNotIn("Copy", commands(c)) c.answer("an answer") - self.assertIn("Use this", self.labels(c)) - self.assertIn("Copy", self.labels(c)) + self.assertIn("Use this", commands(c)) + self.assertIn("Copy", commands(c)) def test_copy_carries_the_whole_answer_and_not_the_head_that_is_drawn(self): # Item 45's promise, restated on this surface: the window is a view, not a diff --git a/tests/test_editor.py b/tests/test_editor.py index abac53e..3c575b6 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -721,15 +721,31 @@ def test_a_waiting_row_does_not_reflow_it_either(self): b._render() self.assertAlmostEqual(self._right_edge(b, "Send"), settled, delta=1) - def test_the_secondaries_still_start_at_the_left_pad(self): - # Pinned right is not centred or spread: the row still reads left to right. + def test_the_rightmost_command_mark_has_a_fixed_address_too(self): + """The secondaries left the foot and became marks in the top-right corner, and + the same argument followed them. + + The set changes constantly — Edit and Was a command come and go with what was + said — so the cluster is laid out right-to-left from the panel's right edge. + Grown the other way it would shift every mark under the hand each time the set + changed, which is the defect the primary's fixed address exists to prevent. + """ import flow.ui as ui - b = self._bubble() - b._render() - it = next(i for i in b.canvas.items if i["text"] == "Refine") - self.assertAlmostEqual(it["x"] - ui.chip_w("Refine", "Refine") / 2, - ui.PAD, delta=1) + self.assertEqual(ui.command_x(0), ui.BUBBLE_W - ui.PAD) + # Slot 0 does not care how many are beside it — that is the property. + self.assertEqual(ui.command_x(0), ui.command_x(0, ui.BUBBLE_W - ui.PAD)) + step = ui.COMMAND_H + ui.COMMAND_GAP + self.assertEqual(ui.command_x(1), ui.command_x(0) - step) + self.assertEqual(ui.command_x(3), ui.command_x(0) - 3 * step) + + def test_the_cluster_never_reaches_the_left_pad(self): + # Four marks is the most the bubble ever shows at once — Refine, Continue, Edit, + # Was a command — and they have to leave the draft's column alone. + import flow.ui as ui + + leftmost = ui.command_x(3) - ui.COMMAND_H + self.assertGreater(leftmost, ui.BUBBLE_W / 2) class TestTheWayBackSitsBesideTheFact(unittest.TestCase): From fb7f7ee98cfd1d2f0d276ac6ff06c89ff847bb38 Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 09:08:15 -0500 Subject: [PATCH 40/41] Ship what the canvas promised: coloured marks, and the note beside Send "This is what you build this is what you promise", against the design canvas. Two things differed, and both were mine. The marks were grey. The canvas colours them - gold, cyan, pink, green - and I shipped neutral on a rule I invented afterwards: colour for settings, grey for actions. It is a defensible rule and it was not the agreed design, which makes it the wrong one. A design agreed and then quietly narrowed is worse than one never shown. The note and the primary were on separate lines. The canvas puts them on one - note left, Send right - and stacking them spent a whole band on a sentence that fits beside the button. The note now ends where the primary begins, so "clears the chips" became a horizontal question rather than a vertical one, and a one-line note costs the panel nothing at all. _lay_out publishes _primary_x before its early return, because the note is drawn earlier in the frame and a render that skipped the chip rebuild would otherwise leave the note measuring against a stale row. Three tests asserted the note sat above the chips, which is exactly what changed. They assert the invariant that survived: the note's column stops before the primary starts, so an error message can never land under Send. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 53 +++++++++++++++++++++++++++++++++++++++----- tests/test_bubble.py | 7 +++++- tests/test_editor.py | 18 ++++++++++----- 3 files changed, 66 insertions(+), 12 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index cf7f8d0..4412c15 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1198,6 +1198,10 @@ def _unload_fonts() -> None: #: band now steps by it. BODY_LINE_H = 17 +#: What one line of `FONT_NOTE` measures. The note shares the chip row now, so this is +#: what decides where its baseline sits in a 26 px chip and how much a second line costs. +NOTE_LINE_H = 14 + #: The live partial's own ceiling, and it needs one for the same reason the draft does: #: it is wrapped to the full body column, so it is a multi-line block whose length nobody #: chose. Five lines at the 14 px `FONT_NOTE` measures. @@ -1865,6 +1869,22 @@ def _glyph_new(c, x, y, colour, tags) -> None: c.create_line(x + 6, y + 7, x + 10, y + 7, fill=colour, width=2, tags=tags) +#: One hue per command, the four from the design canvas: `oklch(0.80 0.12 H)` at 85, +#: 200, 340 and 130. Shipped grey first, on a rule I invented — colour for settings, grey +#: for actions — and the reply was to point at the canvas: "This is what you build this is +#: what you promise". Fair. A design agreed and then quietly narrowed is worse than one +#: never shown. +COMMAND_COLOURS = { + "Refine": "#E1B75C", + "Continue": "#43D5DC", + "Edit": "#F19FD6", + "Was a command": "#A4CD79", + "Cancel": "#F19FD6", + "Use this": "#43D5DC", + "Copy": "#E1B75C", + "New conversation": "#A4CD79", +} + #: Key -> glyph. A command with no entry here keeps its word, which is the honest #: fallback: a mark nobody can read is worse than a label that is merely longer. COMMAND_GLYPHS = { @@ -5206,7 +5226,8 @@ def _chips(self) -> None: _round_rect(c, x2 - COMMAND_H, PAD, x2, PAD + COMMAND_H, COMMAND_H // 2, fill=CHIP, outline="", tags=(tag, "chips")) glyph(c, x2 - COMMAND_H + (COMMAND_H - ICON_SIZE) / 2, - PAD + (COMMAND_H - ICON_SIZE) / 2, CODE, (tag, "chips")) + PAD + (COMMAND_H - ICON_SIZE) / 2, + COMMAND_COLOURS.get(key, CODE), (tag, "chips")) c.tag_bind(tag, "", lambda _e, f=cmd: f()) key, label, cmd = specs[0] # Ask, and it is always first @@ -5261,6 +5282,10 @@ class Bubble(tk.Frame): #: attribute default as `Pill.lite`, and for the same `__getattr__` reason. lite = False + #: Where the primary chip starts, so the note can end before it — they share a row. + #: The default is the right edge, which is what an empty row means. + _primary_x = BUBBLE_W - PAD + #: Declared on the class as well as assigned in `__init__`, for the reason `lite` is: #: `tk.Misc.__getattr__` forwards an unknown attribute to `self.tk`, so on an #: instance built with `__new__` — which is how every UI fixture in this suite builds @@ -5847,7 +5872,9 @@ def _render(self) -> None: # under it: two items sharing a row have to agree who owns which half, and the # one that can wrap is the one that should be told. undo_w = UNDO_W if self._note_undo else 0 - note_w = BUBBLE_W - 2 * PAD - (undo_w + 8 if undo_w else 0) + # Ends where the primary chip begins, because they share a row now. + note_right = getattr(self, "_primary_x", BUBBLE_W - PAD) - CHIP_GAP + note_w = max(60, note_right - PAD - (undo_w + 8 if undo_w else 0)) note_h = self._probe_h(self._note, FONT_NOTE, note_w) if self._note else 0 # Measured for the same reason and never was — see `_partial_slot`. `shown_partial` # is what gets drawn below, so the number the window is sized by and the text it is @@ -5873,7 +5900,10 @@ def _render(self) -> None: # about would be a strip drawn over the first line of the draft. # `COMMAND_BAND` joined the furniture when the secondaries moved to the corner. # A band the height did not know about is a band drawn over the first line. - around = 74 + COMMAND_BAND + BODY_ELIDED_H + (note_h + 4 if note_h else 0) + # A one-line note is free: it sits *on* the chip row rather than above it, so + # only the lines past the first cost the panel anything. + around = (74 + COMMAND_BAND + BODY_ELIDED_H + + (max(0, note_h - NOTE_LINE_H) + 4 if note_h else 0)) if partial_h: around += partial_h + PARTIAL_GAP if self._sent: @@ -5998,7 +6028,12 @@ def _render(self) -> None: # `_lay_out`'s `PAD + CHIP_H` were two numbers that had to agree and nothing # made them. `sw` is what makes a second line grow up into the space the # measurement above just reserved, instead of down onto the chips. - note_baseline = self._h - PAD - CHIP_H - 4 + # On the chip row, not above it. The design put the note and Send on one + # line — "This is what you build this is what you promise" — and stacking + # them cost a whole band for a sentence that fits beside the button. + # Centred on the row: its bottom is half the difference between the chip's + # height and a line of note text. + note_baseline = self._h - PAD - (CHIP_H - NOTE_LINE_H) / 2 c.create_text( PAD, note_baseline, anchor="sw", text=self._note, fill=MUTED, font=FONT_NOTE, width=note_w, tags="body") @@ -6009,7 +6044,7 @@ def _render(self) -> None: # this one has none, but going through it keeps one rule for one job. tag = chip_tag(UNDO_LABEL) c.create_text( - BUBBLE_W - PAD, note_baseline, anchor="se", text=UNDO_LABEL, + note_right, note_baseline, anchor="se", text=UNDO_LABEL, fill=CODE, font=FONT_NOTE, tags=(tag, "body")) c.tag_bind(tag, "", lambda _e: self._undo_edit()) @@ -6101,6 +6136,11 @@ def _lay_out(self, specs) -> None: # Waiting on the CLI dims the row rather than hiding it — same geometry, same # hit regions, just not the thing to reach for while nothing here can act yet. dim = bool(self._act is not None and self._act.waiting) + # Published before the early return, because the note shares this row now and has + # to know where it ends — and a frame that skips the rebuild still draws the note. + primary = next((s for s in specs if s[0] in self.PRIMARY_KEYS), None) + self._primary_x = (BUBBLE_W - PAD - chip_w(primary[0], primary[1]) + if primary else BUBBLE_W - PAD) key_now = (tuple((k, l) for k, l, _c in specs), self._h, self.accent, dim) if key_now == self._chips_drawn or (self._frozen() and self._chips_drawn): return @@ -6145,7 +6185,8 @@ def _lay_out(self, specs) -> None: fill=CHIP, outline="", tags=(tag, "chips")) glyph(c, x2 - COMMAND_H + (COMMAND_H - ICON_SIZE) / 2, y1 + (COMMAND_H - ICON_SIZE) / 2, - DISABLED if dim else CODE, (tag, "chips")) + DISABLED if dim else COMMAND_COLOURS.get(key, CODE), + (tag, "chips")) c.tag_bind(tag, "", lambda _e, f=cmd: f()) # -- the primary, alone at the foot, exactly where it always was --------------- diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 57edcd2..89b3458 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -171,7 +171,12 @@ def test_the_body_slot_is_capped_and_the_note_still_clears_the_chips(self): "find the file specified)") b._render() _top, note_bottom = b.canvas.band("WinError 2") - self.assertLessEqual(note_bottom, b._h - ui.PAD - ui.CHIP_H) + # The note shares the chip row now, so "clears the chips" became a *horizontal* + # question: it ends where the primary begins. Vertically it is on the row on + # purpose — stacking the two cost a whole band for a sentence that fits beside + # the button. + self.assertLessEqual(note_bottom, b._h - ui.PAD) + self.assertLess(b._primary_x, ui.BUBBLE_W - ui.PAD + 1) def test_the_body_cannot_outgrow_a_window_frozen_under_the_hand(self): # Reported from a real session, with a picture: the draft drawn straight diff --git a/tests/test_editor.py b/tests/test_editor.py index 3c575b6..3b0f827 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -620,10 +620,14 @@ def test_the_error_clears_the_chip_row(self): note_top, note_bottom = b.canvas.band("WinError 2") chips_top = b._h - 14 - 26 # `_lay_out`: y2 = _h - PAD, y1 = y2 - 26 self.assertGreater(note_bottom, note_top) - self.assertLessEqual( - note_bottom, chips_top, - f"the note runs to y={note_bottom} and the chips start at y={chips_top}", - ) + # The note is *on* the chip row now, not above it, so the check that matters is + # horizontal: its column has to stop before the primary chip starts. A note that + # ran the full width would put an error message under the Send button. + import flow.ui as ui + + self.assertLessEqual(note_bottom, b._h - ui.PAD) + note = next(i for i in b.canvas.items if "WinError 2" in str(i.get("text", ""))) + self.assertLess(note["x"], b._primary_x - ui.CHIP_GAP) def test_the_bubble_grows_to_make_room_rather_than_clipping(self): """The other way to stop an overlap is to cut the text off, and for an error @@ -652,7 +656,11 @@ def test_a_one_line_note_still_sits_where_it_always_did(self): b = self._bubble("saved") b._render() _top, bottom = b.canvas.band("saved") - self.assertLessEqual(bottom, b._h - 14 - 26) + # The note shares the chip row now, so "clears the chips" became a *horizontal* + # question: it ends where the primary begins. Vertically it is on the row on + # purpose — stacking the two cost a whole band for a sentence that fits beside + # the button. + self.assertLessEqual(bottom, b._h - 14) class TestThePrimaryChipHasAFixedAddress(unittest.TestCase): From da2156b373b4fd308dae0907a617bbf91f9f1ffc Mon Sep 17 00:00:00 2001 From: samartomar <9993940+samartomar@users.noreply.github.com> Date: Tue, 1 Sep 2026 12:35:37 -0500 Subject: [PATCH 41/41] Put the elided count on the marks' row, and give the marks their words back MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things, both from a screenshot of the running app. The count and the icons share the command band now. The band was already paid for and half of it was empty air, while `… 64 earlier lines` took a line of its own underneath — so the draft was a line shorter than it had to be, and the top of the panel read as two rows where the idea was one. The count is wrapped to the room left of the cluster and cannot reach it. And a mark says its name on hover. An icon earns the corner by being small; what it costs is that it says nothing until you already know it. The tip is drawn on the panel's own canvas rather than in a helper window — a second window would fight the one-window shape this overlay was rebuilt around, and on Aqua an override-redirect helper is exactly the fugitive window that took a night to get rid of. `command_slots` is the cluster's walk, pulled out because two callers need it now: the code that draws the marks, and the count that has to stop before them. Co-Authored-By: Claude Opus 5 --- flow/ui.py | 106 ++++++++++++++++++++++++++++++++++++------- tests/test_bubble.py | 70 ++++++++++++++++++++++++++++ tests/test_editor.py | 22 +++++++++ 3 files changed, 181 insertions(+), 17 deletions(-) diff --git a/flow/ui.py b/flow/ui.py index 4412c15..70f1299 100644 --- a/flow/ui.py +++ b/flow/ui.py @@ -1556,6 +1556,37 @@ def chip_tag(key: str) -> str: return "chip-" + key.replace(" ", "-") +#: The canvas tag every hover label is drawn under. One at a time, and deleted by tag +#: rather than tracked: two tips on screen at once is a bug nobody would report. +TIP_TAG = "cmd-tip" + + +def bind_tip(c: tk.Canvas, tag: str, label: str, x: float, y: float) -> None: + """Name the mark tagged `tag` while the pointer is on it, right-aligned at `x, y`. + + A mark earns the corner by being small, and what it costs is that it says nothing + until you already know it. The word comes back on hover, which is the bargain every + toolbar has made — asked for in as many words: "icon can have tool tip". + + Drawn *on the canvas*, not in a `Toplevel`. A helper window would be a second window + in an app whose whole shape is one, and on Aqua an override-redirect helper is + exactly the fugitive window this app spent a night getting rid of. The box is drawn + from the text's own `bbox` and lowered under it, so the label is measured rather + than guessed and no font metric has to be hard-coded here. + """ + def show(_e=None) -> None: + c.delete(TIP_TAG) + text = c.create_text(x, y, anchor="ne", text=label, fill=TEXT, + font=FONT_NOTE, tags=TIP_TAG) + x1, y1, x2, y2 = c.bbox(text) + box = _round_rect(c, x1 - 6, y1 - 3, x2 + 6, y2 + 3, 6, + fill=CHIP, outline=RING, tags=TIP_TAG) + c.tag_lower(box, text) + + c.tag_bind(tag, "", show) + c.tag_bind(tag, "", lambda _e: c.delete(TIP_TAG)) + + def body_window(text: str, budget: int) -> tuple[str, int]: """The last `budget` characters of `text`, and how many lines were left above them. @@ -1781,6 +1812,27 @@ def command_x(slot: int, right: float = None) -> float: return right - slot * (COMMAND_H + COMMAND_GAP) +def command_slots(keys_labels) -> list: + """`(right_edge, width)` for each command mark, given the set *right-to-left*. + + The same walk the cluster is drawn by, pulled out because two callers need it and a + second copy of it would rot: the drawing code, and whatever wants to know where the + cluster *starts* so it can stop before it. A command with no glyph keeps its word + and is wider than a slot, so the walk carries the running right edge rather than + computing each address from its index alone. + """ + out, right = [], BUBBLE_W - PAD + for slot, (key, label) in enumerate(keys_labels): + x2 = command_x(slot, right) + if COMMAND_GLYPHS.get(key) is None: + width = chip_w(key, label) + right -= width - COMMAND_H + else: + width = COMMAND_H + out.append((x2, width)) + return out + + def _glyph_refine(c, x, y, colour, tags) -> None: """A wand with a spark at its tip: this rewrites what you said. @@ -5206,6 +5258,9 @@ def _chips(self) -> None: return self._chips_drawn = key_now c.delete("chips") + # A tip left over from a mark that is about to be redrawn would hang there with + # nothing under it: `` never fires for an item that stopped existing. + c.delete(TIP_TAG) # The same shape the bubble takes: the secondaries as marks in the top-right # corner, the primary alone at the foot. Two surfaces that laid their controls @@ -5286,6 +5341,13 @@ class Bubble(tk.Frame): #: The default is the right edge, which is what an empty row means. _primary_x = BUBBLE_W - PAD + #: Where the command cluster starts, so the elided count can end before it — they + #: share the band. The default assumes a *full* cluster rather than an empty one: + #: this is only read on the frame before the row has been laid out once, and a count + #: clipped a few characters early there is nothing; a count drawn under the marks is + #: the thing this number exists to prevent. + _commands_x = BUBBLE_W - PAD - 4 * (COMMAND_H + COMMAND_GAP) + #: Declared on the class as well as assigned in `__init__`, for the reason `lite` is: #: `tk.Misc.__getattr__` forwards an unknown attribute to `self.tk`, so on an #: instance built with `__new__` — which is how every UI fixture in this suite builds @@ -5902,7 +5964,13 @@ def _render(self) -> None: # A band the height did not know about is a band drawn over the first line. # A one-line note is free: it sits *on* the chip row rather than above it, so # only the lines past the first cost the panel anything. - around = (74 + COMMAND_BAND + BODY_ELIDED_H + # The elided count is free outside the editor: it sits *in* the command band, + # left of the marks, rather than on a line of its own above the draft — asked + # for in as many words, "icons should be in same row that was the idea". Inside + # the editor the line above the box counts what the *box* left out, which is a + # different number about a different thing, so that one still costs a line. + around = (74 + COMMAND_BAND + + (BODY_ELIDED_H if self._editor is not None else 0) + (max(0, note_h - NOTE_LINE_H) + 4 if note_h else 0)) if partial_h: around += partial_h + PARTIAL_GAP @@ -5938,10 +6006,6 @@ def _render(self) -> None: # first, it read `… 2484 more lines` for a 60-line draft and drew a bar on a # draft that fitted — both on the render before the widget existed. extra += edit_h - text_h + 8 + BODY_ELIDED_H - elif earlier: - # Outside the editor this counts what the *window* left out; inside it, the - # line above the box counts what the *box* left out, and both are drawn. - extra += BODY_ELIDED_H if self._sent: extra += 16 # the "sent" label above the words if partial_h: @@ -5992,10 +6056,16 @@ def _render(self) -> None: if earlier: # Said rather than implied: a window with nothing above it reads as the # whole draft, and somebody would go looking for words that are there. + # + # Drawn in the command band, on the marks' own line and clear of where + # they start. The band was already paid for and half of it was empty air + # while the count had a line of its own below it — so the draft gets that + # line back, and the two things at the top of the panel read as one row. c.create_text( - PAD, y, anchor="nw", text=f"… {earlier} earlier lines", fill=MUTED, + PAD, PAD + COMMAND_H / 2, anchor="w", + text=f"… {earlier} earlier lines", fill=MUTED, + width=max(40, self._commands_x - CHIP_GAP - PAD), font=(*FONT_NOTE, "italic"), tags="body") - y += BODY_ELIDED_H # Muted once it has gone: these are no longer the words being worked on. # # `draft` is a second tag on the live text only, and it is what makes @@ -6141,17 +6211,21 @@ def _lay_out(self, specs) -> None: primary = next((s for s in specs if s[0] in self.PRIMARY_KEYS), None) self._primary_x = (BUBBLE_W - PAD - chip_w(primary[0], primary[1]) if primary else BUBBLE_W - PAD) + # The *last* primary, so a row is still laid out sensibly if one ever carries two. + pinned = max((i for i, (k, _l, _c) in enumerate(specs) + if k in self.PRIMARY_KEYS), default=None) + heads = [i for i in range(len(specs)) if i != pinned] + slots = command_slots([(specs[i][0], specs[i][1]) for i in reversed(heads)]) + # Where the cluster starts. Published for the same reason as `_primary_x`: the + # elided count shares the band with the marks now, it is drawn earlier in the + # frame, and a frame that skips the rebuild still draws it. + self._commands_x = min((x2 - w for x2, w in slots), default=BUBBLE_W - PAD) key_now = (tuple((k, l) for k, l, _c in specs), self._h, self.accent, dim) if key_now == self._chips_drawn or (self._frozen() and self._chips_drawn): return self._chips_drawn = key_now c.delete("chips") - # The *last* primary, so a row is still laid out sensibly if one ever carries two. - pinned = max((i for i, (k, _l, _c) in enumerate(specs) - if k in self.PRIMARY_KEYS), default=None) - heads = [i for i in range(len(specs)) if i != pinned] - # -- the secondaries, as a cluster of marks in the top-right corner ------------ # # They were a row of words along the foot, sharing it with the primary. Asked for @@ -6163,23 +6237,19 @@ def _lay_out(self, specs) -> None: # address whatever the set is. The set changes constantly — Edit and Was a command # come and go with what was said — and a cluster that grew leftward from the left # edge would move every icon under the hand each time. - right = BUBBLE_W - PAD - for slot, i in enumerate(reversed(heads)): + for (x2, width), i in zip(slots, reversed(heads)): key, label, cmd = specs[i] glyph = COMMAND_GLYPHS.get(key) tag = chip_tag(key) - x2 = command_x(slot, right) y1 = PAD if glyph is None: # No mark for this one, so it keeps its word. A glyph nobody can read is # worse than a label that is merely longer. - width = chip_w(key, label) _round_rect(c, x2 - width, y1, x2, y1 + COMMAND_H, 13, fill=CHIP, outline="", tags=(tag, "chips")) c.create_text(x2 - width / 2, y1 + COMMAND_H / 2, text=label, fill=DISABLED if dim else CODE, font=FONT_CHIP, tags=(tag, "chips")) - right -= width - COMMAND_H # this one is wider than a slot else: _round_rect(c, x2 - COMMAND_H, y1, x2, y1 + COMMAND_H, COMMAND_H // 2, fill=CHIP, outline="", tags=(tag, "chips")) @@ -6187,6 +6257,8 @@ def _lay_out(self, specs) -> None: y1 + (COMMAND_H - ICON_SIZE) / 2, DISABLED if dim else COMMAND_COLOURS.get(key, CODE), (tag, "chips")) + # The word the mark replaced, on hover, under the mark it belongs to. + bind_tip(c, tag, label, x2, y1 + COMMAND_H + 6) c.tag_bind(tag, "", lambda _e, f=cmd: f()) # -- the primary, alone at the foot, exactly where it always was --------------- diff --git a/tests/test_bubble.py b/tests/test_bubble.py index 89b3458..2f69ab6 100644 --- a/tests/test_bubble.py +++ b/tests/test_bubble.py @@ -513,3 +513,73 @@ def test_the_row_is_drawn_above_the_body_it_outlived(self): self.assertIn('tag_raise("chips")', inspect.getsource(ui.Bubble._render)) self.assertIn('tag_raise("chips")', inspect.getsource(ui.ConversationCard._render)) + + +class TestTheBandHoldsTheCountAndTheMarks(unittest.TestCase): + """`… N earlier lines` shares the command band instead of taking a line under it. + + Asked for from a screenshot of the running app: "66 earlier lines and icons should + be in same row that was the idea". The band was already paid for and half of it was + empty air, so the count cost the draft a whole line for nothing. + """ + + def _elided(self, b): + return next(i for i in b.canvas.items if "earlier lines" in i["text"]) + + def test_the_count_sits_on_the_marks_row(self): + b = bubble(draft(50_000)) + b._render() + it = self._elided(b) + # Centred on the band, which is where the marks are: they run from `PAD` to + # `PAD + COMMAND_H`. + self.assertEqual(it["y"], ui.PAD + ui.COMMAND_H / 2) + self.assertEqual(it["x"], ui.PAD) + + def test_the_count_stops_before_the_cluster_starts(self): + # Sharing a row is only an improvement while the two do not meet: the count is + # wrapped to the room left of the marks, and it is one line, so it cannot. + b = bubble(draft(50_000)) + b._render() + it = self._elided(b) + self.assertLessEqual(it["x"] + it["wrap"], b._commands_x) + self.assertEqual(it["lines"], 1) + + def test_the_draft_gets_the_line_back(self): + # The count no longer costs the body a line, so the same draft in the same + # window shows one more line of what was actually said. + b = bubble(draft(50_000)) + b._render() + lines = max(i["lines"] for i in b.canvas.items if "body" in i["tags"]) + self.assertGreaterEqual(lines, 6) + + +class TestAMarkSaysItsNameOnHover(unittest.TestCase): + """The word the icon replaced, on hover — "icon can have tool tip". + + A mark earns the corner by being small, and the price is that it says nothing until + you already know it. The tip is drawn on the panel's own canvas: a helper window + would be a second window in an app whose whole shape is one. + """ + + def _enter(self, b, key): + tag = ui.chip_tag(key) + return next(f for t, seq, f in b.canvas.bindings + if t == tag and seq == "") + + def test_hovering_a_mark_draws_its_word_and_leaving_takes_it_away(self): + b = bubble(draft(200)) + b.pill.session.can_rescue = True + b._render() + self._enter(b, "Refine")() + self.assertTrue(any(i["text"] == "Refine" and ui.TIP_TAG in i["tags"] + for i in b.canvas.items)) + leave = next(f for t, seq, f in b.canvas.bindings + if t == ui.chip_tag("Refine") and seq == "") + leave(None) + self.assertNotIn(ui.TIP_TAG, [t for i in b.canvas.items for t in i["tags"]]) + + def test_the_primary_keeps_its_word_and_needs_no_tip(self): + b = bubble(draft(200)) + b._render() + self.assertFalse(any(t == ui.chip_tag("Send") and seq == "" + for t, seq, _f in b.canvas.bindings)) diff --git a/tests/test_editor.py b/tests/test_editor.py index 3b0f827..81f31ef 100644 --- a/tests/test_editor.py +++ b/tests/test_editor.py @@ -534,6 +534,8 @@ def tag_bind(self, tag=None, sequence=None, func=None, *a, **kw) -> None: def tag_raise(self, *a, **kw) -> None: ... + def tag_lower(self, *a, **kw) -> None: ... + def itemconfigure(self, *a, **kw) -> None: ... def create_window(self, *a, **kw) -> None: @@ -549,6 +551,9 @@ def create_text(self, x, y, text="", **kw): item = { "x": x, "y": y, "text": text, "anchor": kw.get("anchor", "center"), "h": lines * self.LINE_H.get(size, 17), "lines": lines, + # The wrap budget, recorded because one item is now bounded *horizontally* + # by something beside it — the elided count stops before the command marks. + "wrap": width or 10**6, # Recorded because one surface draws the same slot in two colours to say # whose words they are — the answer in REPLY, the question in MUTED — and a # fake that dropped the colour could not see them swap. @@ -747,6 +752,23 @@ def test_the_rightmost_command_mark_has_a_fixed_address_too(self): self.assertEqual(ui.command_x(1), ui.command_x(0) - step) self.assertEqual(ui.command_x(3), ui.command_x(0) - 3 * step) + def test_command_slots_walks_the_cluster_the_way_it_is_drawn(self): + """One walk, two callers: the marks are drawn from it and the elided count stops + before it. A second copy of this arithmetic is a copy that rots.""" + import flow.ui as ui + + slots = ui.command_slots([("Refine", "Refine"), ("Continue", "Continue")]) + self.assertEqual([w for _x, w in slots], [ui.COMMAND_H, ui.COMMAND_H]) + self.assertEqual(slots[0][0], ui.BUBBLE_W - ui.PAD) + self.assertEqual(slots[1][0], ui.command_x(1)) + + # A command with no glyph keeps its word and is wider than a slot, so everything + # left of it shifts by the difference — which is why the walk carries the edge. + wide = ui.command_slots([("Nameless", "Nameless"), ("Refine", "Refine")]) + width = ui.chip_w("Nameless", "Nameless") + self.assertEqual(wide[0][1], width) + self.assertEqual(wide[1][0], ui.BUBBLE_W - ui.PAD - width - ui.COMMAND_GAP) + def test_the_cluster_never_reaches_the_left_pad(self): # Four marks is the most the bubble ever shows at once — Refine, Continue, Edit, # Was a command — and they have to leave the draft's column alone.