Fix for issue 559

[briankwest]

Testing the Audio Mute State Preservation Fix

This document describes how to test the fix for the issue where audio mute state was not preserved when switching input devices.

Issue Description

Problem: When a call was muted using call.muteAudio(), the audio track's enabled flag was set to false. However, when call.setAudioInDevice() was invoked to switch the input device, a new audio track was created without preserving the enabled = false state, causing the previously muted call to become unmuted.

Fix: Modified setAudioInDevice() and setVideoDevice() methods in packages/common/src/webrtc/BaseCall.ts to preserve the enabled state from the old track when creating a new track.

Final Implementation

The fix was implemented in commit 1c8cea0 and includes:

Core Changes in BaseCall.ts

1. Enhanced setAudioInDevice() method:

async setAudioInDevice(deviceId: string): Promise<void> {
  const { instance } = this.peer
  const senders = await instance.getSenders()
  const sender = senders.find(
    ({ track: { kind } }: RTCRtpSender) => kind === 'audio',
  )
  if (sender) {
    const newStream = await getUserMedia({
      audio: { deviceId: { exact: deviceId } },
    })
    const audioTrack = newStream.getAudioTracks()[0]

    // 🎯 THE FIX: Preserve the enabled state from the old audio track
    const { localStream } = this.options
    const oldAudioTracks = localStream.getAudioTracks()
    if (oldAudioTracks.length > 0) {
      audioTrack.enabled = oldAudioTracks[0].enabled
    }

    sender.replaceTrack(audioTrack)
    this.options.micId = deviceId

    localStream.getAudioTracks().forEach((t) => t.stop())
    localStream.getVideoTracks().forEach((t) => newStream.addTrack(t))
    this.options.localStream = newStream
  }
}

2. Enhanced setVideoDevice() method:

async setVideoDevice(deviceId: string): Promise<void> {
  const { instance } = this.peer
  const senders = await instance.getSenders()
  const sender = senders.find(
    ({ track: { kind } }: RTCRtpSender) => kind === 'video',
  )
  if (sender) {
    const newStream = await getUserMedia({
      video: { deviceId: { exact: deviceId } },
    })
    const videoTrack = newStream.getVideoTracks()[0]

    // 🎯 THE FIX: Preserve the enabled state from the old video track
    const { localElement, localStream } = this.options
    const oldVideoTracks = localStream.getVideoTracks()
    if (oldVideoTracks.length > 0) {
      videoTrack.enabled = oldVideoTracks[0].enabled
    }

    sender.replaceTrack(videoTrack)
    attachMediaStream(localElement, newStream)
    this.options.camId = deviceId

    localStream.getAudioTracks().forEach((t) => newStream.addTrack(t))
    localStream.getVideoTracks().forEach((t) => t.stop())
    this.options.localStream = newStream
  }
}

Testing Options

There are two ways to test this fix:

Option 1: Flask Automated Test App (Recommended)

A comprehensive Flask application was created specifically for testing this fix with automated test sequences.

Location: packages/js/examples/flask-mute-test/

Features:

• Automatic JWT token generation
• Automated test sequence
• Real-time test result reporting
• Comprehensive device switching tests
• Console logging and debugging

Quick Setup:

cd packages/js/examples/flask-mute-test
pip install -r requirements.txt
export SIGNALWIRE_SPACE="your-space.signalwire.com"
export SIGNALWIRE_PROJECT_ID="your-project-id"
export SIGNALWIRE_TOKEN="your-api-token"
./build-and-copy-sdk.sh
python app.py

Then navigate to http://localhost:3000 and follow the automated test instructions.

Option 2: Manual Testing with Vanilla Example

Location: packages/js/examples/vanilla-calling/

Setup:

# Build the SDK with the fix
npm run setup js

# Start development server
cd packages/js
npm run watch-es5

Navigate to https://localhost:9000 for manual testing.

Detailed Test Procedures

Prerequisites

1. SignalWire Account: Active project with API credentials
2. Multiple Audio Devices: At least 2 audio input devices (built-in mic + USB headset)
3. Multiple Video Devices: At least 2 video input devices (optional, for video testing)
4. Modern Browser: Chrome, Firefox, Safari, or Edge with WebRTC support

Automated Test Sequence (Flask App)

The Flask app runs the following automated tests:

1. Initial State Check: Verifies audio starts unmuted
2. Mute Audio: Calls muteAudio() and verifies mute state
3. 🎯 Device Switch While Muted: Switches audio device and verifies mute state is preserved (THE FIX)
4. Unmute Audio: Calls unmuteAudio() and verifies unmute state
5. Device Switch While Unmuted: Switches audio device and verifies unmute state is preserved

Manual Test Steps

1. Connect to SignalWire

   • Enter Project ID and JWT token
   • Enable audio (and optionally video)
   • Click "Connect"

2. Make a Call

   • Enter destination and source numbers
   • Click "Call" and wait for call to become active

3. Test Audio Mute State Preservation

   Step 3a: Basic Mute Test

   • Click "Mute Audio"
   • Verify button changes to "Unmute Audio"
   • Check console for mute state confirmation

   Step 3b: Device Switch Test (The Fix)

   • While call is muted, select different audio device
   • Expected: Audio remains muted after device switch
   • Verification: Button stays in "Unmute Audio" state

   Step 3c: Unmute Test

   • Click "Unmute Audio"
   • Switch to another audio device
   • Expected: Audio remains unmuted after device switch

4. Test Video Mute State Preservation (if video enabled)

   • Repeat same steps with video mute/unmute and video device switching

Expected Results

✅ Success Criteria (Fix Working)

• Mute Preservation: Audio/video remains muted after device switching
• Unmute Preservation: Audio/video remains unmuted after device switching
• UI Consistency: Buttons correctly reflect actual track state
• Console Logging: Correct mute state reported in console
• Track State: track.enabled matches expected state

❌ Failure Indicators (Fix Broken)

• Mute State Lost: Audio/video becomes unmuted after device switching when it should remain muted
• UI Inconsistency: Button state doesn't match actual track state
• Console Errors: JavaScript errors during device switching
• Track State Mismatch: track.enabled doesn't match expected state

Technical Details

How the Fix Works

1. Before Device Switch: Store the current track's enabled state
2. Create New Track: Generate new audio/video track with new device
3. Preserve State: Set newTrack.enabled = oldTrack.enabled
4. Replace Track: Update the peer connection with the new track
5. Update Stream: Replace the old track in the local stream

Key Code Changes

The fix adds these critical lines to both setAudioInDevice() and setVideoDevice():

// Preserve the enabled state from the old track
const oldTracks = localStream.getAudioTracks() // or getVideoTracks()
if (oldTracks.length > 0) {
  newTrack.enabled = oldTracks[0].enabled
}

Files Modified

• packages/common/src/webrtc/BaseCall.ts - Core fix implementation
• packages/js/examples/flask-mute-test/ - Complete automated test suite
  • app.py - Flask server with JWT generation
  • templates/index.html - Test interface with automation
  • build-and-copy-sdk.sh - SDK build script
  • requirements.txt - Python dependencies
  • README.md - Detailed setup instructions

Browser Compatibility

Tested and working in:

• ✅ Chrome 120+
• ✅ Firefox 119+
• ✅ Safari 17+
• ✅ Edge 120+

Troubleshooting

Common Issues

1. HTTPS Required: WebRTC requires HTTPS for device access
2. Device Permissions: Browser needs explicit permission for audio/video
3. Device Labels: Device names may not appear until permissions granted
4. CORS Issues: Ensure proper CORS configuration if using external servers

Debug Commands

# Verify SDK build
ls -la packages/js/dist/

# Rebuild SDK if needed
npm run setup js

# Check TypeScript compilation
cd packages/common && npx tsc --noEmit

# Test Flask app
cd packages/js/examples/flask-mute-test
python app.py

Console Debugging

// Manual testing in browser console
const call = window.currentCall;

// Check current track state
console.log('Audio enabled:', call.localStream.getAudioTracks()[0].enabled);

// Test mute preservation
call.muteAudio();
console.log('After mute:', call.localStream.getAudioTracks()[0].enabled); // Should be false

// Test device switch with mute preservation
call.setAudioInDevice('deviceId').then(() => {
  console.log('After device switch:', call.localStream.getAudioTracks()[0].enabled); // Should still be false
});

Development Workflow

To test changes to the SDK:

1. Make Changes: Edit source code in packages/common/src/
2. Build SDK: Run npm run setup js or use Flask app's build-and-copy-sdk.sh
3. Test: Use Flask app for automated testing or vanilla example for manual testing
4. Verify: Ensure all test cases pass before committing

Commit Information

• Fix Commit: 1c8cea0 - "Fix for issue 559"
• Files Changed: 10 files, +1717 lines
• Branch: mute_fix
• Issue: When switching the input device during an ongoing muted call, the call unintentionally becomes unmuted #559 - Audio mute state not preserved when switching input devices

This fix ensures that the mute state is properly preserved when switching input devices, resolving the issue where previously muted calls would become unmuted after device changes.

[jpsantosbh]

consider adding this to .gitignore since this will be built