Push-to-talk, a native macOS engine, and the placement work behind them - #1
Draft
samartomar wants to merge 33 commits into
Draft
Push-to-talk, a native macOS engine, and the placement work behind them#1samartomar wants to merge 33 commits into
samartomar wants to merge 33 commits into
Conversation
…ut a hotkey The chord was a toggle that fired one word on release. It is push-to-talk now: press warms the models and opens the microphone, the hold is the utterance, and the release stops capture and pastes what was said. Both gestures ship and Settings > Chord switches between them while Flow runs, because they are good at different things - a hold needs no decision about when you are finished, and a toggle is the only one of the two that survives a paragraph or hands that cannot hold two keys down. The same gesture is on the pill itself, which is the part that matters off Windows: `Chord` is a WH_KEYBOARD_LL hook and there is no such thing on a Mac, so Lite has never had push-to-talk. It does not need a hotkey to. Hold the pill, speak, let go - one button now carries three gestures, judged on what you did: a quick click toggles, a 300 ms hold talks, motion drags. That fixes something older on the way past: `_toggle` was bound to the button *press*, so every drag of the pill used to toggle listening. Five things can send one utterance - the chip, the hotkey, the spoken trigger, converse's countdown, and now a release waiting on a decode. A hold owns one send and whoever gets there first has it, enforced at the single point all five share rather than five guards that would drift. Clear cancels a pending paste, a mode switch drops it, the toggle hotkey mid-hold commits rather than letting `pause()` refuse the decode it just asked for, and a hold is refused while the hand editor is open instead of recording nothing. Two ceilings, neither hypothetical: a hold whose release never arrives stops itself after two minutes, and a paste that never lands gives up after fifteen seconds and says where the words are. Both keep what was said. Placement moved to bottom centre of the monitor under the pointer, taken from FluidVoice's positionWindow - centred on the physical display, stood on the work area. `"place": "corner"` restores the old bottom right. Doing it turned up a real bug: the work area was read once at launch from SystemParametersInfoW, which only ever answers for the primary monitor, so on a two-monitor desk every window Flow drew could land on the screen you were not looking at. Hidden panels park past every display instead of unmapping, so a hold shows the draft in the time a window takes to move. The meter blooms from its centre rather than scrolling a history past, which is FluidVoice's BottomWaveformView arithmetic - envelope, 0.55 response curve and rounded caps. Re-measured the body column at the shipped 420 px rather than the 380 it was taken at: 62 characters a line, not 56, and the tail is 1750 characters for the same 28 laid-out lines. Idle unload went 5 min to 30 - five minutes is inside the gaps of a working session, so the common case was a reload in the middle of somebody's first sentence back rather than memory reclaimed from somebody gone. The comment defending it said "the 141 MB model" while two tiers were resident; it is ~605 MB. Carries the per-app prompt sets already in the tree - refine.py, the apps table and tests/test_apps.py - because profile.py, README.md and guide.md hold both and splitting them needs hunk surgery. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ng it Two things a Mac reported in one breath: the pill had close and maximize buttons on it, and it sat under the Dock. The frame is Aqua's window class, not the redirect flag. `overrideredirect(True)` strips the title bar on Windows and X11, but Tk maps a Toplevel to an NSWindow whose style mask comes from the window class, so on a Mac it did nothing visible. `MacWindowStyle plain noActivates` is the supported way to ask, and it carries the second half too: `noActivates` is Aqua's WS_EX_NOACTIVATE, without which clicking the pill pulls focus off whatever is being dictated into - the defect that made every paste land in the wrong window on Windows, already fixed once and no reason to ship again. The Dock is a work area Flow was not asking for. `_work_area` degrades to the whole screen off Windows, so bottom-centre placement stood the stack on the very bottom edge - behind the Dock on macOS, behind the panel on a bottom-taskbar Linux. `wm_maxsize` looked like the answer and is useless: on Windows it reports the full screen even with a taskbar present, wrong in exactly the way this is meant to fix. What works is maximising a transparent probe window and reading back where the window manager put it, since it has to honour its own panels to do that - exact against SystemParametersInfoW on left, right and bottom. Measured once and cached, because `_sync_monitor` asks every frame. `top` is taken as reported and is a title bar low. `winfo_rooty` is the client area and the frame inset differs between a normal window and a maximised one - measuring the inset and subtracting it made the answer worse, not better (-8 against a true 0). It feeds one thing, the ceiling in `bottom_centre`, where being conservative costs nothing. scripts/mac_probe.py prints every number Tk will give for this, next to what Flow computes from them, so the next platform question is a measurement rather than a third guess. It is what found the `wm_maxsize` dead end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…as nothing to run faster-whisper's weights are on HuggingFace and nowhere else official - SYSTRAN's GitHub ships the library, not the models - so a network that blocks that one host leaves copying 138 MB by hand onto every machine. macOS already has an on-device recogniser whose models belong to the OS. Using it ends the transport problem rather than routing around it. A subprocess, not a binding. The dependency budget is three and PyObjC is not one of them, and Flow already shells out to codex and claude, so a process that reads audio and writes text is a shape this app has. It also puts every Objective-C API behind a pipe, where a crash is an exit code instead of a dead interpreter. The helper is compiled on first use rather than shipped: an unsigned prebuilt binary is a Gatekeeper prompt with a worse story than the source it came from. The seam was already there. `Transcriber` is one method wide, `load`/`unload`/ `take_confidence`/`take_drops` are read through getattr, and clean.py has a documented narrow path for an engine that cannot report `no_speech_prob`. Somebody planned for this. --engine auto will not switch a machine that is working, and that is the whole rule. Apple's recogniser is a *different* engine, not a spare one: no `no_speech_prob`, so the hallucination filter drops to the narrow check; one quality tier where Whisper has two; no biasing, so the re-listen that rescues a mis-heard command comes back unbiased. Changing what Flow hears without being asked would be the wrong default, so auto reaches for it in exactly one case - the Whisper models are not on this machine and cannot be fetched. The startup line always says which engine ran and why. The presence check asks with the hub forced offline and puts the variable back, because it is asked precisely when the network may be unusable and must not become the download it is testing for. Not compiled here - this machine has no Swift. flow_stt.swift is written against SFSpeechRecognizer with requiresOnDeviceRecognition, and `--file` transcribes one recording so Apple's quality can be judged before any of it is wired into a dictation loop. The Python half is tested: the wire format is asserted against the bytes actually written, since the far side reads them with load(as: UInt32.self).littleEndian and a disagreement there is a silent mis-decode rather than an error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… for consent
The macOS decoder is written on a machine with no Swift toolchain, so until now
the compiler was a person on a laptop pasting errors back. The first version
shipped with a hard one in it: Swift allows top-level statements only in a file
called `main.swift`, and this is not one. A macos-latest leg is the fix - the
same runner the suite already uses, doing the one thing that machine can do and
this one cannot.
Three more found by reading it again with that in mind, two of which would have
been runtime rather than a build failure:
* The recogniser delivers its callbacks to the main queue by default, and
`transcribe` blocks the calling thread waiting for one. On the main thread
that is a deadlock, not a slow decode. It gets its own OperationQueue.
* Every read out of `Data` is unaligned now. `Data` carries no alignment
guarantee and `load(as:)` requires one, so the aligned form traps on some
buffers and not others - the worst way to find out.
* `update(from:count:)` is recent; `memcpy` is not, and this has to build on
whatever toolchain a machine happens to have.
Arguments are parsed before `authorize()`. A typo should not raise a permission
prompt, and CI has to reach the usage line on a runner where no permission could
ever be granted - which is what lets the leg prove the binary links and runs at
all rather than merely compiles.
Compile-and-link only, and the comment says why that is the honest limit:
actually running it needs a granted Speech Recognition permission and an
on-device model that only arrives when a human enables Dictation. Faking either
would make the leg green about something it never checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tform CI compiled the helper and said two things. `@main` and a single-file executable are mutually exclusive: Swift parses one file as a script, and the attribute cannot be used in a module with top-level code. The compiler's own note names the fix, so `-parse-as-library` goes in all four places that build this - the file's own header, flow/native.py, the CI leg and the guide. Renaming to main.swift was the alternative; the flag keeps the file named after what it is. `withUnsafeMutableBytes` hands back whatever its closure returns, here `copyBytes`' byte count, and the leg builds with -warnings-as-errors on purpose: this file is edited by people who cannot run it, so a warning is the only signal that reaches them before a user does. Three of mine in the macOS suite, all the same mistake - a test reading the platform it happens to be on rather than pinning one. `ctypes.windll` does not exist off Windows, so patching it needs create=True; and the two that assert the native engine's *refusal* were pinned to win32, because on a macOS runner there is nothing to refuse and they would go build a real binary instead. A test that means one thing on Windows and another on macOS is only half run wherever it passes. The macOS leg was already red on main before this branch - test_lite and test_main reach ctypes.WinDLL, and the font-metric check measures 10 where Windows measures 7. Those are not touched here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… builds it CI found a real one, and it was not about CI: the macOS leg went from 35 seconds on main to 643 on this branch. Every launch on a Mac without Whisper models was compiling Swift and then sitting on `--probe` until it timed out, because the probe waits for an authorization dialog that a headless machine never answers. A user's first launch would have done exactly the same thing. A full minute of nothing before a pill appears, on a machine that asked for none of it, to decide whether to offer an engine it might not use. So `--engine auto` passes compile_if_missing=False and a ten second probe: it uses the helper if it is already built and says how to build it if not. `--engine native` still compiles, because somebody who typed it has asked for the wait. Once it has been run once, auto finds the binary from then on. A timeout is now named as the dialog it is rather than as "probe failed: TimeoutExpired", which is true and useless when the fix is a click. And every one of those reasons reaches `say()`, which writes to a cp437 console - so the em dashes in them were a launch that died on its own explanation. That is how this one was found; there is a test over the module's string literals now so the next one is caught here rather than by a user on Windows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
samartomar
force-pushed
the
push-to-talk
branch
from
August 31, 2026 04:36
3cb4003 to
c8bd61a
Compare
Two guesses at Aqua have been spent already. The pill came back with a title bar and three traffic lights while the panels above it were correctly bare, and the difference between them is that `Pill` *is* the root window - `class Pill(tk.Tk)` - where Bubble and ConversationCard are Toplevels. Aqua builds the root's NSWindow before Tk can restyle it, so what works for a Toplevel need not work for `.`. Five windows, five techniques, labelled on screen. The answer is which ones have no title bar, which is a thing a person can see and this machine cannot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`MacWindowStyle plain` is not frameless, and believing it was is what put a title bar and three traffic lights on the pill. `plain` is a window *class*. A Mac running scripts/mac_frame_probe.py settled it: a Toplevel given only the style comes up decorated, one given only overrideredirect(True) comes up bare, and the call reports `ok` either way - it succeeds at doing the wrong thing. So this was a regression I introduced. Before it, `_no_activate` returned False off Windows and did nothing, and every window was bare because overrideredirect had already done the work. Applying the style afterwards put the frame back on. overrideredirect strips the frame here as it does everywhere else, and the style call stays for `noActivates` alone - the half that keeps a click on the pill from pulling focus off whatever is being dictated into. It is asked for first and the redirect re-asserted after, so the class can never win. The root needs it knocked off twice. Aqua builds the NSWindow when the window is first mapped and a redirect asked for afterwards does not restyle what already exists, which is why Bubble and ConversationCard were bare while the pill - which *is* the root, `class Pill(tk.Tk)` - was not. Withdrawing and remapping forces the rebuild, and only for a window already on screen: the panels are withdrawn when `_no_activate` runs over all three, and deiconifying them would put two empty surfaces in front of the user. Also from that probe: the Mac is on Tk 9.0, not 8.6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Mac put the pill in the top-left corner, and the reason was a sanity check that
only asked "is this a rectangle".
`state("zoomed")` on Aqua does not raise and does not maximise either - it is
accepted and ignored. So the probe stayed the 200x120 it was asked for, at +80+80,
and (80, 80, 280, 200) passed a test that wanted a positive rectangle no bigger
than the screen. The stack was then placed against a work area 200 px wide, which
on a 1512-wide display is the top-left corner. Exactly where it was found.
The test is not "is this a rectangle" but "did the window manager do the thing",
and it is asked two ways because either alone has a hole. A window that never grew
past what it asked for is the direct evidence. A rectangle far smaller than the
display catches a window manager that grew it a little and stopped - a real work
area is the screen minus a Dock or a taskbar, nowhere near half of it.
Still exact against SystemParametersInfoW on Windows, where zoomed does work.
Also: the remap that takes the frame off the root now puts the geometry back. A
window that comes back from a withdraw has been placed by whoever remapped it,
not by whoever positioned it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…end went Clicking Send stopped working on a Mac, and it was the second half of the same mistake as the title bar. `MacWindowStyle plain noActivates` asks Aqua for a window that never activates - and a window that never activates does not take clicks either. The app already said so. `_menu` borrows the foreground on Windows precisely because a WS_EX_NOACTIVATE window would get no input for its popup, and the comment there states that Lite needs none of that: "`_no_activate` cannot take, so the window is in the activation chain like any other and the popup gets its input the ordinary way." That was an invariant, and asking Aqua for the equivalent broke it. So `_no_activate` refuses off Windows again, as it did before any of this, and the frame is handled where it always belonged - beside the `overrideredirect` call in `_shell_window`. `_mac_reframe` withdraws and remaps a window that is already on screen, which is what makes the redirect reach an NSWindow Aqua had already built. That is the whole of the root-window fix; nothing asks for a window class any more. The test for it asserts over the source rather than a mock, because the mistake was *making the call*, and a mock cannot notice a call that is no longer there. The name still appears in prose explaining why it is not used - those comments are the point, so what is checked is `::tk::unsupported::`, the only form that can do damage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…reated A Mac showed the window and then hid it. That was the withdraw-and-remap cycle added to "force Aqua to rebuild the NSWindow" - and it existed only because MacWindowStyle had put a frame back on, which is a thing that call does and overrideredirect had never failed to prevent. Re-reading the probe with that in mind says so plainly. Window 4, the style asked for on a *mapped* window, came up decorated. Window 2, the same style asked for while the window was withdrawn, came up bare. The app was case 4: overrideredirect early, style later, on a window long since mapped. The root was never the problem and `class Pill(tk.Tk)` was a red herring. So both attempts are gone and what is left is the line that had been doing the work all along. Three commits to arrive back at one call, with two ways of making it worse now written down. The tests assert the absence rather than a mechanism, which is the honest shape for this: there is nothing new to exercise, only two things that must not come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…one round trip Diagnosing this platform from a Windows machine has been costing three messages a question - send a photo, now the log, now run the other probe - and a phone photo of a screen cannot answer "is it 24 px above the Dock or 240". `scripts/mac_report.py` drives the real windows, screenshots them, renders the numbers beside them and writes one PNG carrying both. Platform and Tk build, what each work-area method answers, where the stack is therefore placed, how far off centre it is, whether the native engine is ready. A neutral backdrop goes behind it first. `shots.py` does that to keep captures clean; here it is also a privacy line, because the whole point of this file is that the image gets sent to somebody and a full-screen grab of a working machine carries every window that happened to be open on it. The crop is read off the windows rather than reserved with a margin. The first version guessed 320 px above the pill and 60 below, which cropped in the taskbar and cut off the pill's own bottom edge - which is exactly where a title bar or a broken dock seam shows. A black capture is named as the missing Screen Recording permission it is, rather than looking like a bug in Flow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…utput The quickest way to answer a question about this platform has turned out to be somebody's agent running one command and pasting what it printed. Asking that for Pillow and a Screen Recording grant to print twelve numbers is the report getting in the way of being read, so --text skips the capture, the backdrop and the image entirely. Pillow moved to a lazy import for it. The numbers were always most of what the picture was carrying. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…whole screen The maximise probe that finds the taskbar exactly on Windows is accepted and ignored on Aqua, so _tk_work_area fell back to the whole screen and the pill was placed 24 px above 878 - inside a 69 px Dock. Rather than guess a fourth time at this platform, this prints wm maxsize (which Tk's Aqua port answers from NSScreen visibleFrame), where a window asked for +0+0 actually lands, and fullscreen as a control. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ores it
A Mac reported _tk_work_area() answering with the whole 1352x878 screen, identical to
_work_area(). state("zoomed") had neither raised nor maximised, so the fallback did not
fall back: bottom_centre stood the pill 24 px above 878 and it lived inside a 69 px
Dock. The close-up in the report was a picture of Dock icons.
wm maxsize is the call that knows, and only here - Tk's Aqua port answers it from
NSScreen visibleFrame. On Windows it reports the whole screen with a taskbar present,
which is why the maximise probe exists at all. It gives no origin, so the top comes from
where a window asked for +0+0 actually lands, and the bottom - the edge the pill stands
on - is top + height.
Every measurement is shape-checked and the path returns None unless all of them hold, so
if this platform surprises us again the worst case is exactly the old behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ht to check them against scripts/mac_area_probe.py on a 14-inch MacBook Pro, Tk 9.0.3, 1352x878: wm maxsize 1352x736, a window asked for +0+0 landing at y 58, and zoomed answering (80, 108, 280, 228) - the 200x120 probe still at the +80+80 it was asked for, never maximised, no error. So the Dock's top edge is 58 + 736 = 794. Both of those are Tk asking Tk, and three earlier guesses at this platform failed for want of a second source. defaults read com.apple.dock tilesize on the same machine says 69, and 69 plus Apple's padding is the 84 px this leaves below 794. That agreement is now a test. The invented fixtures are replaced by the measured ones throughout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Transcribed from a photo of the terminal as 736, which the same output disproves on the line above - lost: 143 h against an 878 screen is 735, and 58 + 735 is the 793 the probe printed. The Dock is 85 px, not 84, which is if anything the better agreement with the 69 px tilesize the OS reports. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… lite mode mac_report.py died on a Mac with bad attribute "-transparentcolor", raised by Tk before a window had been drawn. -transparentcolor and -toolwindow exist on one platform and are fatal on the others, and _shell_window guarded them on lite alone - safe only because __main__ forces lite off Windows, so the two could never come apart. They came apart the first time something other than __main__ built a Pill. An invariant a caller has to know about is one a caller can miss, and this one is cheap to enforce where it is actually true. mac_report now builds the lite window the app builds, which is the one worth measuring anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ll is bare The Aqua branch fired and answered 821 where the probe says 793 - 28 px too low, one title bar, and the pill went off the Dock and straight back onto it. wm maxsize is a maximum *content* size, so it is short by whatever decoration the window that answered it wears: 735 from a titled probe against 763 from the overrideredirect pill, on the same display. The first version took the origin from a probe and the size from the caller's window, which are not the same window, and counted the title bar twice. Now one probe answers all three - its own title bar from a y clear of any menu bar, the menu bar from +0+0 less that title bar, and the frame height from maxsize plus it - so the decoration appears on both sides and cancels. A bare window and a titled one must now reach the same answer, which is a test. Windows corroborates the arithmetic from the other direction: a 31 px title bar and a 0 px menu bar, there being none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from a Mac: click the app you want to dictate into and Flow's window vanishes, and clicking Send does nothing. Those are one fault - a borderless overrideredirect NSWindow is ordered out when its app stops being frontmost, and a click on a background app's window is spent activating that app rather than pressing what is under it. macOS answers this with a nonactivating NSPanel, which is what FluidVoice's overlay is. Tk 9 exposes it as wm attributes -class and -stylemask - named by Flow's own crash, when Tk refused -transparentcolor and listed what this build does accept. The probe asks for a nonsense value first, because Tk answers that by enumerating the legal ones, so a run is informative even if every variant below is wrong. It records what survived the background on its own and logs which buttons a click reached, so the answer comes back as pasted text rather than a judgement call or a photograph. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first run answered a question it was not asked. Two variants had overrideredirect and two did not, and the two without were both the ones a click reached and the ones still on screen after clicking another app. overrideredirect on Aqua makes a window deaf and fugitive at once, and it is the one line Flow uses on every window it owns. The NSPanel class it was written to test never applied - cannot change the class after the mac window is created, even withdrawn - so it was never the variable. That leaves what overrideredirect was for: taking the title bar off. Tk's complaint about a nonsense style bit listed the real ones, and a mask without the titled bit is a borderless window that was never made deaf. The title bar is now measured rather than looked at - ask for a known y, and how far below it the client area lands is the decoration - so five of the six answers come back in a paste. winfo_viewable is no longer trusted for the sixth: it reported all four windows healthy while two were off the screen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two faults reported from a Mac - click the app you want to dictate into and Flow's window vanishes, and clicking Send does nothing - and one cause. Six variants on screen split on exactly this line: every window without overrideredirect kept its place when another app came forward and had its button reached by a click, and every window with it was deaf and gone. Which is a fair description of what it means - a window the window manager has been told to stop managing. A style mask is the set of bits an NSWindow is built from and titled is the one that puts a title bar on, so a mask with no bits is bare with nothing else given away. Measured at 0 px of decoration against the control's 28, and its button was reached from the background. The NSPanel route is closed and unneeded: Tk answers cannot change the class after the mac window is created even for a window never mapped, and Toplevel(class_=...) names a Tk class rather than an NSWindow one. Nonactivating is about not stealing focus, and these windows never take it. Tk 8.6 on a Mac falls back to the old line rather than getting a frame. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Refinement latency, reported from a Mac where Kiro's default model was slow and failing on retry. Two levers, both read out of each CLI's own --help on a machine that has all three rather than written from memory: --model on all of codex, claude and kiro-cli, and --effort (low, medium, high, xhigh, max) on the last two. codex has no effort flag - its only route is a config key that does not appear in its help - so it is not given one. Effort defaults to low, deliberately. These calls are a rewrite, not a reasoning problem, and the user is watching a spinner between finishing a sentence and having their words. tuned() returns a new Cli rather than threading two arguments through four call layers: Cli is frozen and its argv is a tuple, so everything downstream - resolve, _invoke, _clean, the pill's marker - goes on working unchanged. The flags land after the subcommand and before codex's trailing -, the positional saying the prompt is on stdin, which a flag appended at the end would be read as the value of. Fallbacks are tuned too: reverting to a CLI's own defaults the moment the first candidate failed would be slowest exactly when the user is already waiting longest. Model names cannot be discovered - no CLI will list them - and Flow has no text field to type one into, so --cli-model carries a name in once and the menu is built from the names that have been used. Effort is a menu from the start, being enumerable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner's verdict after every window fault was fixed: it seems on mac there is no actual send it copies in clipboard that is making this no useful on mac. Which was right. The whole idea is to speak into the window you are already working in, and now press Cmd-V yourself is the step Flow exists to remove - so all of the window work led to a clipboard and stopped. inject_mac is inject.py's opposite number, satisfying the same two functions on_send is written against. No SendInput and no window handle exist here: the text goes to the pasteboard with pbcopy and System Events types Cmd-V into whatever is frontmost. It does not need to aim, because Flow's windows are built without the titled bit and never take focus - which is why the window work had to come first. lite stopped being the same question as can-Flow-paste, and the two are now separate: lite is hotkeys and window handles, and on_send is keyed off whether an injector was imported. A Mac is Lite in every other respect and pastes. osascript rather than CGEvent, deliberately: the compiled route is faster and is where this should end up, but it is also an entitlement question and a build step, and both would have to be right before anybody could try it once. This needs no dependency and asks the OS for the same permission. The permission is what will actually bite - Accessibility, granted to the terminal rather than to Flow - so it is detected by error number and answered with the exact path through System Settings, said at startup rather than at the first failed paste. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner's verdict after the paste landed: window being re-draw and adjusting it self ... multiple windows size is problem and it's same bad shape on mac + windows ... sore to eyes. Three separate motions, all of them removed. The pill was PILL_W while nothing was docked and the panel's width while something was, so it jumped 205 -> 420 the instant a draft appeared and back when it went. pill_w now answers the panel width unconditionally, read from the constant rather than through a window that may not exist yet. The bubble and the card sized themselves to their contents on every render. The shots show what that cost: a two-line partial made a 140 px panel and a three-line draft a 215 px one, so the top edge moved while you were still speaking. Both are PANEL_H now and the body elides into it, which is machinery that already existed for the old ceiling. The golden geometry table says it best - every row is the same 420x184 across a draft that differs by fifty times its length. Two arithmetic traps came with it, both found by looking at the shots rather than by reading: the bubble capped its body at BODY_MAX_H unless frozen, and the card sized its answer against the whole desktop, so both drew content for a window far taller than the one they had. The card clipped its own agent label doing it. _float_up keeps the fade and loses the 18 px rise. R14 wanted an appearance the eye could follow when the window also changed size on every render; with the shape fixed the movement was the only thing left moving. 184 px was chosen by looking. At 236 a two-line partial left a hole in the middle of the window, which reads worse than the resizing it replaced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The structure FluidVoice's BottomOverlayView has and Flow did not: text and the control row inside a single rounded shell. Flow drew two operating-system windows kept adjacent by hand, and the seam between them was visible in every screenshot. Bubble and ConversationCard are tk.Frame now, not tk.Toplevel - bands placed at (0,0) inside the pill's window. Their drawing code is canvas-local and did not change; what went is the window around it. Only eleven window-only calls existed between them, which is why this was tractable at all. Deleted: _sync_dock's width negotiation, both repositions' above-or-below decision, _docked_above, park() on the panels, _float_up, and the ordering rule saying whichever ran first left the other with nothing to do. All of it existed to make two windows look like one. _sync_shell is what is left: a height, and a bottom edge that never moves. The foot is the anchor, and that is the whole of the controls staying where they are. A panel opening grows the window upward, so Send, the meter and the chip row stay at the pixel they were at. Growing downward or centring the growth would move every control on the surface every time a draft appeared. Two defects found by looking rather than by reading. The row's seam overpaint cleared rows 0..4 while its inner ring is drawn at y=4, so a second hairline stood 3 px under the divider - invisible before, because a 1 px window gap was already drawing a line there. Measured out of the shot: two RING rows at y=627 and y=642, now one. And band_h had to come off the pill: the panels only ever had to fit the work area while the pill fitted it separately, and sharing a window makes them one sum - a 200 px display put a 224 px shell 32 px past the bottom of it. 39 bubble tests and both dock classes asserted where one window went relative to another, which is not a question any more; they are replaced by TestTheShellIsOneWindow and TestTheBubbleIsABandInThePillsWindow, which assert the properties that survived. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A FluidVoice demo, read frame by frame, settled an argument I had got wrong. Its overlay's bottom edge is at y=554 in every frame of thirteen seconds - idle, two lines, two lines, three lines - and the box is snug around the text in every one of them. It never holds empty space, and at the end it is gone. So the fixed PANEL_H was stricter than the thing it was copying. It stopped the motion and paid for it with a hole in the middle of the window, which is the same complaint the resizing caused wearing different clothes. What must not move is the foot, not the height. PANEL_H becomes PANEL_MAX_H, a ceiling. The band is sized from its content and snapped up to a whole body line by _settled_h. That snap is what replaces FluidVoice's 80 ms debounce: sizing to content is right, and a timer cancelled from a render loop that runs thirty times a second is a thing to get wrong. A height that can only change when the text gains or loses a line changes a handful of times an utterance by construction. The shots caught a regression the tests did not: _sync_shell asked band_h() rather than the band's actual height, so a 130 px band sat in a 224 px shell with the row floating 54 px below it - the detached-boxes look the merge exists to end. Measured on the running app: as a draft grows the shell steps 170 -> 204 -> 221 px while the foot stays at 648 in every frame. Three tests flipped back to asserting growth, which is what they asserted before the fixed height and is right again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The need, in the owner's words: there are times where i wanted to dictate but at the same time i wanted to see but i don't want it to keep it on my screen. Parking the window is the easy half and the dangerous one. A Flow with no window and no icon is a process that cannot be reached, configured or quit except through Task Manager, which is invariant 4 wearing a different hat: hidden must not mean gone. So the icon is registered first and its answer is believed - no icon, no hiding, and the bubble says why. That refusal is the test this file exists for. Win32 through ctypes, no new dependency, R16 holds at three. Shell_NotifyIconW needs a window to send clicks to and Tk will not give us one, so flow/tray.py creates its own message-only window and pumps its own loop on its own thread. Nothing in that module touches Tk: the window procedure puts strings on a queue and the UI drains it from _frame, beside the hotkeys and for the same reason. The chord still works while hidden - the hook is global and does not care what is on screen - and _talk_start brings the window back before anything else, because a hold that showed nothing would be an open microphone with no way to tell. ctypes had to be told the signatures. Without them every call is assumed to return c_int and a 64-bit HWND does not fit: CreateWindowExW failed with int too long to convert, inside a thread nobody was watching. Also renames the gestures for the wording the owner asked for by name: push to talk is a thing people already know, and the label used to describe only the mechanics. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d would not hide Reported: "I'm not able to move this around on my screen ... it stays on the center always", and "when I try to click on hide to tray it did not hide". Both are mine, from merging the two windows. _sync_shell asked _placed() for x and for the foot on every frame, so a drag was undone before the hand had left the mouse. _sync_dock never had the fault because it only recomputed x when the width changed, which was rare; recomputing unconditionally is what the merge introduced. It now keeps where the window *is* and clamps it, and _placed stays the answer at startup and on a monitor change, which is where _sync_monitor asks it. Hiding failed twice over. park(self) reads win.width, which the panels have and the pill did not, so the call went to tk.Misc.__getattr__ and looked for a Tcl command. And even parked, _sync_shell put the window straight back on the next frame - it runs thirty times a second and won every one of them. Hiding now remembers where the window was, as (x, foot) rather than (x, y): the shell is anchored by its bottom edge, so a panel that was open at hide time and closed by the time it comes back must not move the controls. Verified on the running app: a dragged x of 40 survives every subsequent frame. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two screenshots of "panel size: larger" showed a wide band above a row that had kept its old width - two boxes of different widths stacked in one window, which is the exact look merging them was meant to end. apply_panel_width rebinds BUBBLE_W while Flow is running, so the width changes with nothing else changing beside it. _sync_shell compared x, y and the height and not the width, so the canvas was never re-placed and _docked_w - which is what _draw measures the row against - kept the value it was built with. The old _sync_dock compared the width and nothing else, so the merge managed to lose the one comparison that mattered here. Verified on the running app across regular, large, larger and back: window, row, band and _docked_w now agree at 420, 520, 640, 420. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Right-clicking the tray icon showed the menu and then did nothing. user32 exports PostMessageA and PostMessageW and no bare PostMessage - the undecorated name is a C macro - so ctypes raised AttributeError: function 'PostMessage' not found, from inside the tray thread, after the menu had been chosen from and before anything acted on the choice. Nothing in the suite could have caught it, because nothing named the function until somebody clicked. So the suite names them all now: a test walks tray.py's own source for every user32 call it makes and asks Windows whether each one is really there. Sixteen symbols, all present, and it would have failed the moment the typo was written. The click handler also stops anything escaping into Windows. It called us; an exception unwinding into its stack is undefined at best, and a tray that quietly stops answering is the failure this file exists to prevent - so it is caught and printed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"loading the model" used to be the first thing a fresh Flow said back - in the bubble, while the user was already talking. The load lands inside that first utterance rather than in front of it: first partial 1 230 ms against ~570 ms for the four behind it. The chord's press-down has warmed the models since push-to-talk shipped, which covers the second use and never the first. warm() is called after the pill is built and before the loop runs, so the window is on screen while the disk does its work rather than after it. It is single-flight and loads on a thread of its own, so nothing at startup waits on it - measured at 0 ms to return, with the model loaded 1.6 s later. --no-warm keeps the old behaviour, for a launcher that starts with the machine, where paying a model load at login is the wrong trade, and for measuring the cold path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Asked for by name - "Dictate and Converse for sure Then workspace and voices" - with the choice of when to show it left to me. With the panel, never at rest. Those three only mean anything once there is something to send, and an always-on strip would cost 22 px of the idle row, which is the part of this surface everybody has said they like small. FluidVoice does not pay it either: its Dictate / AI Prompt / Actions bar belongs to the app being dictated into, not to the overlay. Mode is a control and the other two are values, which is the whole design. Mode changes what Send does and is switched mid-task, so it is a chip that acts. Workspace and voice are worth *seeing* - which project Ask runs in, without opening anything - and clicking either opens the list the right-click menu already builds, tick and all, rather than growing a second implementation of it. The strip is furniture, so PANEL_MAX_H grew by SETTINGS_H rather than the draft paying for it. Taking it out of the content's share is what the editor tests caught: the live partial's cap is a flat 70 px, and on a panel pegged at the old ceiling it ran straight through the note and the chip row because everything above it had grown by 22 and it had not been told. One test relaxed from assertGreater to assertGreaterEqual, honestly: _settled_h steps by a whole body line, so a note that grew by 14 px can land in the same bucket. That is what the snap is for, and the property that test is really about - the note drawn whole, and clear of the chips - is asserted beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, opened to get CI on it — in particular the new macos-latest leg that compiles
native/flow_stt.swift, which is the one thing that cannot be checked on the machine it was written on.What is here
ctrl+winpress warms the models and opens the mic, the hold is the utterance, the release sends. Both gestures ship and Settings ▸ Chord switches between them live.WH_KEYBOARD_LL, nothing granted but the microphone.--engine autowill not switch a machine that is working.SystemParametersInfoW, which only answers for the primary monitor.What CI is being asked
The suite (2100 tests) already passes on Windows and macOS locally. The new question is the Swift: it is written on Windows and has never been compiled. The first version had a hard error in it — top-level statements outside
main.swift— and three more were found by re-reading rather than by building.Compile and link only. Running it needs a granted Speech Recognition permission and an on-device model that arrives when a human enables Dictation, neither of which a headless runner has.