Skip to content

PR_26171_BETA_081-message-playback-through-tts-engine#40

Merged
ToolboxAid merged 1 commit into
mainfrom
pr/26171-BETA-081-message-playback-through-tts-engine
Jun 21, 2026
Merged

PR_26171_BETA_081-message-playback-through-tts-engine#40
ToolboxAid merged 1 commit into
mainfrom
pr/26171-BETA-081-message-playback-through-tts-engine

Conversation

@ToolboxAid

Copy link
Copy Markdown
Owner

Summary

  • Add server-owned Emotion Settings to Messages TTS profile API responses without schema changes.
  • Require Message Studio playback to resolve a selected TTS Profile Emotion Setting before speaking.
  • Pass selected profile language/voice and emotion setting pitch/rate/volume/preset through the TextToSpeechEngine-backed registry for Play Part and Play Message.
  • Keep Stop routed through the TextToSpeechEngine-backed registry stop path.

Validation

  • node --check toolbox/messages/messages.js
  • node --check src/dev-runtime/messages/messages-sqlite-service.mjs
  • npx playwright test tests/playwright/tools/MessagesTool.spec.mjs
  • npx playwright test tests/playwright/tools/TextToSpeechFunctional.spec.mjs --reporter=list
  • node --test tests/tools/Text2SpeechShell.test.mjs
  • npm run test:workspace-v2
  • git diff --check

@ToolboxAid ToolboxAid merged commit 1451a11 into main Jun 21, 2026
1 check passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2122063eff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +15 to +19
const DEFAULT_TTS_EMOTION_SETTINGS = Object.freeze([
Object.freeze({ active: true, emotion: "calm", emotionLabel: "Calm", pitch: 1, rate: 1, ssmlLikePreset: "normal", volume: 1 }),
Object.freeze({ active: true, emotion: "urgent", emotionLabel: "Urgent", pitch: 1.08, rate: 1.15, ssmlLikePreset: "normal", volume: 1 }),
Object.freeze({ active: true, emotion: "whisper", emotionLabel: "Whisper", pitch: 0.95, rate: 0.9, ssmlLikePreset: "normal", volume: 0.55 }),
Object.freeze({ active: true, emotion: "angry", emotionLabel: "Angry", pitch: 0.98, rate: 1.1, ssmlLikePreset: "normal", volume: 1 }),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cover all seeded emotions in fallback TTS profile

When there are no active API TTS profiles, activeTtsProfileOptions() falls back to DEFAULT_TTS_PROFILE; however the fallback Emotion Settings added here only cover Calm/Urgent/Whisper/Angry while the seeded active emotions also include Excited, Sad, and Mysterious. In that no-active-profile fallback path, any message or part using one of the omitted seeded emotions now fails in speakTarget() with a missing Emotion Setting error instead of playing with the emotion values as it did before. Include all seeded emotions or derive these fallback settings from state.emotionProfiles.

Useful? React with 👍 / 👎.

Comment on lines +287 to +290
const setting = settings.find((candidate) => (
emotionSettingKey(candidate.emotion) === selectedEmotionKey
|| emotionSettingKey(candidate.emotionLabel) === selectedEmotionKey
));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match emotion settings without slug collisions

If two active Emotion Profiles have distinct names that normalize to the same slug, such as High Energy and High-Energy, the API permits both names but this lookup can return the first setting for either profile because it checks the slugified candidate.emotion before the exact label. Playback then applies the wrong pitch/rate/volume for whichever profile appears later in the settings list. Prefer matching by a stable emotion profile key, or at least check the exact emotionLabel before the slug.

Useful? React with 👍 / 👎.

Comment on lines +464 to +466
const emotionSetting = selectedEmotionSettingForProfile(profile, target.emotionProfile);
if (!emotionSetting.ok) {
return { message: emotionSetting.message, ok: false };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate speech readiness against the target profile

When a message or part has its own TTS profile selected in the row, selectedSpeechTarget() stores that profile and testSelectedSpeech() passes target.profile to speakTarget(), but this new readiness check validates the separate Inspector dropdown profile instead. With per-profile Emotion Settings, Test Speech can be disabled by an unrelated profile or shown as ready and then fail on click. Check target.profile || selectedTtsProfile() here so readiness validates the same profile that will actually speak.

Useful? React with 👍 / 👎.

Comment on lines +963 to +966
const emotionSetting = selectedEmotionSettingForProfile(profile, target.emotionProfile);
if (!emotionSetting.ok) {
return visiblePlaybackError(emotionSetting.message);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preflight all parts before queuing playback

When Play Message is used on a multi-part message and a later part lacks a matching Emotion Setting, earlier loop iterations have already called speakTarget() and queued audio before this new guard returns the error. That means the user hears a partial message even though playback is reported as blocked for a missing setting. Resolve settings for every active part before calling the TTS registry, or otherwise stop/avoid queuing earlier parts when any part cannot be played.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant