Skip to content

Surface + document the loopback Local API for on-device agents (Settings toggle + docs) - #715

Open
PaulBlackSwan wants to merge 27 commits into
altic-dev:mainfrom
PaulBlackSwan:feat/local-api-toggle-and-docs
Open

Surface + document the loopback Local API for on-device agents (Settings toggle + docs)#715
PaulBlackSwan wants to merge 27 commits into
altic-dev:mainfrom
PaulBlackSwan:feat/local-api-toggle-and-docs

Conversation

@PaulBlackSwan

@PaulBlackSwan PaulBlackSwan commented Jul 26, 2026

Copy link
Copy Markdown

Description

FluidVoice already ships a loopback-only Local API (LocalAPIServer: POST /v1/transcribe, POST /v1/postprocess, GET /v1/health, plus history/dictionary), but it can only be enabled through the hidden LocalAPIEnabled default and is undocumented. This PR surfaces and documents it so other apps and AI agents running on the same Mac can reuse FluidVoice's transcription instead of bundling and loading their own ASR model.

  • SettingsStore.localAPIEnabled — reuses the existing "LocalAPIEnabled" key (no change to how the server reads config).
  • LocalAPIServer.refresh() — starts/stops the listener live when the setting changes (no app restart).
  • A Local API (on-device agents) toggle in the always-visible App Settings section, off by default.
  • README section + docs/local-api.md (endpoints, request/response, security, limits, examples).

Type of Change

  • 🐞 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 🧹 Chore
  • 📝 Documentation update

Related Issue or Discussion

Related to #529 (Companion CLI for Local API — adoption discussion) and #414 (Extend the LocalAPI to read/set the speech model…). This PR surfaces the existing Local API via a Settings toggle and documents it (endpoints, security, limits), which directly supports the external/agent adoption those threads discuss. No dedicated issue was opened for the toggle+docs itself — happy to file one if you'd prefer.

Testing

  • Tested on Intel Mac
  • Tested on Apple Silicon Mac
  • Tested on macOS version: 26 (Apple Silicon)
  • Ran linter locally: swiftlint --strict --config .swiftlint.yml Sources
  • Ran formatter locally: swiftformat --config .swiftformat Sources
  • Ran tests locally:

xcodebuild -scheme Fluid -configuration Debug buildBUILD SUCCEEDED. Manual checks: toggle on → GET /v1/health 200 on 127.0.0.1:47733; POST /v1/transcribe {"path": …} returns the transcript reusing the already-warm model; toggle off → listener stops; rapid off→on handled cleanly; non-loopback peer refused.

Screenshots / Video

Local API (on-device agents) toggle in App Settings
  • No UI/visual changes; screenshots/video are not applicable.

Notes

Addressed the automated review feedback:

  • Listener-identity race (Codex P2 / Greptile P1): stateUpdateHandler now guards self.listener === listener (weak capture) so a rapid off→on toggle can't orphan or cancel the freshly-installed listener; and connections accepted just before stop() are now dropped instead of handled after disable.
  • Toggle placement: moved out of the accessibility-gated Shortcuts/Options block into the always-visible App Settings section — the Local API is unrelated to global-hotkey/Accessibility permissions, so it shouldn't be hidden behind them.
  • Wording & limits: only the HTTP listener and /v1/transcribe are on-device; /v1/postprocess follows the configured provider (may be remote). Documented the 300 s audio cap (file-path input rejected; inline audioBase64 truncated) and that enabling via defaults needs a relaunch, whereas the Settings toggle applies live.

The loopback-only Local API (POST /v1/transcribe, /v1/postprocess) already
exists but was enable-only via a hidden "LocalAPIEnabled" default and was
undocumented. This surfaces and documents it:

- SettingsStore.localAPIEnabled (reuses the existing "LocalAPIEnabled" key)
- LocalAPIServer.refresh() to start/stop the server live when toggled
- a "Local API (on-device agents)" toggle in Settings (loopback-only, off by default)
- README section + docs/local-api.md documenting endpoints & security

Lets other apps/agents on the same Mac reuse FluidVoice's transcription
instead of loading their own ASR model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added needs PR template Pull request is missing required template content. needs screenshots Pull request needs screenshot or video evidence. labels Jul 26, 2026
@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown

The PR Policy check is blocking this PR because required template information is missing.

Please update the PR description with:

  • Related Issue or Discussion

Visual files detected:

  • Sources/Fluid/Persistence/SettingsStore.swift
  • Sources/Fluid/UI/SettingsView.swift

Screenshots or video are required for UI, UX, settings, onboarding, overlay, menu bar, or visual behavior changes. If this PR has no visual changes, check the no-visual-change box in the template.

If this remains incomplete for 48 hours after opening, the PR may be closed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66dbd621be

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/UI/SettingsView.swift Outdated
Comment thread README.md Outdated
Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift
@greptile-apps

greptile-apps Bot commented Jul 26, 2026

Copy link
Copy Markdown

Greptile Summary

This PR exposes the existing loopback Local API and expands its supported audio contract.

  • Adds an always-visible setting that starts or stops the listener without relaunching the app.
  • Adds bearer-token authentication and loopback-only listener configuration.
  • Adds OGG/Opus decoding, duration validation, request cancellation handling, and focused tests.
  • Documents endpoints, authentication, provider behavior, and input limits.

Confidence Score: 5/5

The PR appears safe to merge based on the currently established behavior.

No blocking failure remains; the only unresolved prior concern depends on whether an already-running transcription provider observes propagated task cancellation.

Reviews (28): Last reviewed commit: "fix: serialize 401 responses as unauthor..." | Re-trigger Greptile

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift
Comment thread docs/local-api.md Outdated
…o-cap docs)

- LocalAPIServer: ignore state callbacks from a superseded listener (guard
  self.listener === listener, weak capture) so a rapid off->on toggle can no
  longer orphan/cancel the newly-installed listener (Codex P2 / Greptile P1).
- Qualify the privacy claim: only the HTTP listener and /v1/transcribe are
  on-device; /v1/postprocess follows the configured provider (may be remote)
  (toggle text, README, docs).
- Describe the accept-time loopback filtering accurately instead of claiming
  it binds 127.0.0.1.
- Document the 300s (5 min) audio-duration cap alongside the 25 MB body cap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PaulBlackSwan
PaulBlackSwan force-pushed the feat/local-api-toggle-and-docs branch from 7d418cb to 0e68a5d Compare July 26, 2026 20:00
@PaulBlackSwan

Copy link
Copy Markdown
Author

Thanks for the thorough review — all four points addressed in 0e68a5d:

  • Listener-identity race (P1): stateUpdateHandler now guards self.listener === listener (weak capture), so a rapid off→on toggle can't orphan or cancel the freshly-installed listener.
  • Privacy wording: clarified that only the HTTP listener and /v1/transcribe are on-device; /v1/postprocess follows the configured provider (may be remote) — toggle text, README and docs.
  • Binding wording: documented the accept-time loopback filtering accurately rather than claiming it binds 127.0.0.1 (left the listener setup unchanged to avoid IPv4/IPv6-loopback regressions — glad to switch to an explicit loopback bind if you prefer).
  • Docs: added the 300 s (5 min) audio-duration cap next to the 25 MB body cap, noting they're independent.

Rebuilt clean (xcodebuild -scheme Fluid → BUILD SUCCEEDED). Also filled in the PR template; a screenshot of the Settings toggle is coming.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e68a5d532

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift
…ed section

The toggle sat next to 'Copy to Clipboard', which lives inside the
`if accessibilityEnabled` block, so it was hidden until the user granted
Accessibility — unrelated to a loopback HTTP API. Moved it to the always-visible
'App Settings' section, and split the long description into description + footnote
so it no longer truncates on one line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PaulBlackSwan

Copy link
Copy Markdown
Author

Follow-up in d8ef83f: moved the toggle out of the accessibility-gated Shortcuts/Options block into the always-visible App Settings section. It was next to "Copy to Clipboard", which lives inside if accessibilityEnabled { … }, so the Local API toggle was hidden until the user granted Accessibility — unrelated to a loopback HTTP API. Also split the long description into description + footnote so it no longer truncates on one line. Rebuilt clean.

settingsToggleRow renders description and footnote on single truncating lines;
the previous footnote overflowed. Trimmed it to the essentials — the full
contract (non-loopback filtering, limits, post-processing locality) lives in
README + docs/local-api.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PaulBlackSwan

Copy link
Copy Markdown
Author

Adding a screenshot of the Toggle

image

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e88f57419b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread docs/local-api.md
Comment thread docs/local-api.md Outdated
- LocalAPIServer: the deferred @mainactor task now also checks the server is
  still running (listener != nil) before creating a handler, so a connection
  accepted just before stop()/toggle-off is dropped instead of handled.
- docs: note that enabling via `defaults` needs an app relaunch (the Settings
  toggle applies live via refresh()); clarify the 300s audio cap rejects
  file-path input but truncates inline audioBase64 input.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PaulBlackSwan
PaulBlackSwan force-pushed the feat/local-api-toggle-and-docs branch from 34c108d to cec22e7 Compare July 27, 2026 20:36
@PaulBlackSwan

Copy link
Copy Markdown
Author

Latest round (cec22e7):

  • Superseded-listener race (re-flagged P1/P2): this was fixed in 0e68a5dstateUpdateHandler guards self.listener === listener (weak capture) before touching state, so a stale .cancelled/.failed callback from an old listener can't clear or cancel the new one. Verified by inspection against the toggle off→on→off sequence.
  • Connections queued before disable (Codex P2): the deferred @MainActor task now also checks listener != nil before creating a handler, so a connection accepted just before stop() is dropped instead of handled after the API is off.
  • Docs: enabling via defaults requires an app relaunch (the Settings toggle applies live via refresh()); clarified that the 300 s audio cap rejects file-path input but truncates inline audioBase64 input.

Also filled the PR template + added the toggle screenshot to the description.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cec22e7eb0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/UI/SettingsView.swift Outdated
Comment thread Sources/Fluid/UI/SettingsView.swift Outdated
@github-actions github-actions Bot removed needs PR template Pull request is missing required template content. needs screenshots Pull request needs screenshot or video evidence. labels Jul 27, 2026
@AmaraFray

Copy link
Copy Markdown

Hi, any ETA on when this might be merged, desperately need access to the local API 🙏

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

This pull request has been marked stale because it has had no activity for 5 days. It will be closed in 2 days if there is no further activity. Add the keep-open or pinned label to prevent automatic closure.

@github-actions github-actions Bot added the stale label Aug 4, 2026
@PaulBlackSwan

PaulBlackSwan commented Aug 4, 2026 via email

Copy link
Copy Markdown
Author

@github-actions github-actions Bot removed the stale label Aug 5, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been marked stale because it has had no activity for 5 days. It will be closed in 2 days if there is no further activity. Add the keep-open or pinned label to prevent automatic closure.

@github-actions github-actions Bot added the stale label Aug 12, 2026
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d1ce5934a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift
Comment thread docs/local-api.md Outdated
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

1 similar comment
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ebca65d88

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIAudioDecoder.swift Outdated
Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 166ca005bb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/UI/SettingsView.swift Outdated
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6285201dbe

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread docs/local-api.md
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@altic-dev

altic-dev commented Aug 13, 2026 via email

Copy link
Copy Markdown
Owner

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f6788f75f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 687e040390

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIServer.swift
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06ad4b9279

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/Fluid/Services/LocalAPI/LocalAPIRouter.swift
@PaulBlackSwan

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 730e427884

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@github-actions github-actions Bot removed the stale label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants