Skip to content

Meeting block: FluidAudio transcription, live transcript UI, AI summary#9

Merged
max4c merged 74 commits intomainfrom
dev
Apr 2, 2026
Merged

Meeting block: FluidAudio transcription, live transcript UI, AI summary#9
max4c merged 74 commits intomainfrom
dev

Conversation

@max4c
Copy link
Copy Markdown
Owner

@max4c max4c commented Apr 2, 2026

Summary

  • FluidAudio (Whisper) transcription — 5s chunked recording via AVAudioEngine + AsrManager; SFSpeechRecognizer kept as #else fallback. Model pre-warmed at startup so first recording starts immediately.
  • Floating recording pill — always-on-top NSPanel with animated blue bars, live duration counter, stop button, and tap-to-navigate.
  • Live transcript drawer — right-aligned chat bubbles (#B1D4F9), sentence-level splitting, search with auto-focus, copy button with checkmark feedback, scroll-to-bottom on open.
  • AI summary generationclaude --model haiku via shell process; cleanTranscript + extractStructuredSections; results injected as editable Block children via MarkdownBlockParser.parse. Ladybug Generate button with spinner + disable guard.
  • Meeting block UX — editable wrapping title, summary in lighter gray vs user notes, ladybug SF Symbol icon, chevron/button hit-area fixes, no spurious permissions from shell login flag.
  • Simplify passrmsLevel helper (dedup), O(1) transcript append (was O(N²) rejoin), chunk timer cleanup on error, parseSections computed once per render, transcriptBottomAnchorID constant, markdownToBlocks replaced by MarkdownBlockParser.parse.

Test plan

  • Record a meeting and verify FluidAudio transcribes correctly (no Speech Recognition dialog)
  • Floating pill appears during recording, hides on stop and on block deletion
  • Transcript drawer opens/closes with chevron; search focuses automatically; copy shows checkmark
  • Generate produces summary blocks in lighter gray; button hides after generation
  • Title field wraps and is editable; Resume/Stop/Record buttons have full hit area

🤖 Generated with Claude Code

max4c added 30 commits March 30, 2026 22:50
max4c and others added 28 commits March 31, 2026 09:10
…nflicts)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Took main's versions of AppSettings, MailModels, AppSettingsStore,
  CalendarService, GoogleAuthService, MailService, MeetingNoteService,
  MailFeatureTests (mail intelligence feature set wins)
- ContentView: merged handleRecordingChange() — kept dev's blockId
  callback logic inside main's extracted helper method
- project.pbxproj: kept both sides' file refs (GatewayView, MentionPickerView
  from dev; KeychainSecretStore, MailIntelligence* from main), kept
  WebKit.framework (used by MailPaneView), dropped ThreadPickerRow ref
  (file does not exist), added MARKETING_VERSION + .dev bundle ID from main

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…summary

- Switch transcription to FluidAudio (Whisper) with 5s chunked recording via
  AVAudioEngine + AsrManager; keep SFSpeechRecognizer as #else fallback
- Add floating recording pill (always-on-top NSPanel) with animated bars,
  duration counter, stop button, and tap-to-navigate
- Transcript drawer with right-aligned chat bubbles, search, copy-with-checkmark
  feedback, and scroll-to-bottom on open
- AI summary via claude CLI: cleanTranscript + extractStructuredSections, result
  injected as editable Block children via MarkdownBlockParser.parse
- afterStateView: editable title (axis: .vertical), ladybug Generate button,
  generate guard/disable, summary content in lighter gray vs user notes
- Sidebar chat icon updated to bubble.left.and.bubble.right
- Simplify pass: extract rmsLevel helper, incremental transcript join (O(1) append
  vs O(N2) rejoin), timer cleanup on chunk error, parseSections computed once per
  render, transcriptBottomAnchorID constant, markdownToBlocks uses MarkdownBlockParser

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1adaa37ff5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 27 to 29
<key>NSMicrophoneUsageDescription</key>
<string>Bugbook needs microphone access to record meeting audio for live transcription.</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>Bugbook uses speech recognition to transcribe meeting recordings in real-time.</string>
<key>SUFeedURL</key>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore speech usage description for fallback recognizer

This commit removes NSSpeechRecognitionUsageDescription, but the new fallback path still calls SFSpeechRecognizer.requestAuthorization when FluidAudio is unavailable (TranscriptionService.startRecording under #else). On macOS, requesting Speech permission without that Info.plist key causes the app to fail at runtime, so builds/environments that rely on the fallback recognizer will not be able to start recording.

Useful? React with 👍 / 👎.

Comment on lines +1203 to +1205
// Find the @<filter> token nearest to the anchor and replace it.
if let range = block.text.range(of: searchToken) {
block.text.replaceSubrange(range, with: mention)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replace mention at anchor instead of first token match

insertMention currently replaces block.text.range(of: searchToken), which always targets the first matching @... token in the block. If the same mention prefix appears earlier in the text, selecting a mention from the picker edits the wrong occurrence and leaves the one at the cursor unchanged. The picker already tracks mentionPickerAnchorPos, so this replacement should be anchored to that position.

Useful? React with 👍 / 👎.

@max4c max4c merged commit 7329e61 into main Apr 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant