Multitrack Files

[kaixxx]

Discussed in #239

^{Originally posted by sebastian-stix October 29, 2025}
I don't need speaker identification, since I export multitrack audio for Auphonic, which already gives me perfectly separated speakers. Once this feature is available, noScribe will definitely be my first choice.

[kaixxx]

Thank you for the suggestion. The use case with Auphonic might be a bit special, in particular since Auphonic already comes with its own transcription solution, as far as I can tell. However, we have a similar situation with Zoom recordings. So, I might consider this idea for a future version of noScribe. Unfortunately, it might require some non-trivial changes in the architecture of the whole app. So, I don't want to promise anything.

[sebastian-stix]

Great!
I'm using reaper (ultraschall.fm) and export Multitrack by default. If noScribe can create transcripts from that, this will replace auphonic for me completely.

[spackmat]

Hey, I have used OBS Studio for some of my interview recordings and also put me and the interviewee on different (stereo) audio tracks. I then use ffmpeg do merge those into one stereo track:

ffmpeg.exe -i "myinputwith2stereoaudiotracks.mp4" -filter_complex "[0:a:0]pan=mono|c0=FL[a1];[0:a:1]pan=mono|c0=FL[a2];[a1][a2]amerge=inputs=2[aout]" -map 0:v -map "[aout]" -c:v copy -c:a aac -ac 2 -b:a 192k "myoutputwithonepannedstereotrack.mp4"

Since noScribe already utilizes ffmpeg for an audio extracting step, that seems to be just one simple switch to enable mixing down a two track file to a single stereo track one. Maybe the detection for the occurrence of a two track input can be automatted with a hint how to downmix more than two input tracks manually.

[mutlusun]

You are right that would be feasible. However, the long-term goal of noscribe would be to be able to transcribe the different files / track independently and get perfect speaker diarization on this way.

However, adding a hint for manual mixing would be great.

[spackmat]

You are right that would be feasible. However, the long-term goal of noscribe would be to be able to transcribe the different files / track independently and get perfect speaker diarization on this way.

That would be much better, indeed. But until that, an in-built downmix or a hint how to do that manually would save people from having to know or figure out how to achieve that. (I have that knowledge, but nearly none of my students do.) UI-wise that could be done as a warning or an error at the audio-extraction step, when more than one track is detected by ffmpeg: "That file has more than one audio track. Only the first one will be transcribed with the current version of noScribe. If you want to downmix the tracks into one, you can do that with our shipped ffmpeg an the following command: THE COMMAND"

[kaixxx]

I'm a little confused by "... mixing down a two track file to a single stereo track one." Do you mean mixing it down to a mono track? This would indeed be very good. I think this also doesn't need a switch or a separate message, but should be the default behavior.

That is, of course, until we manage to transcribe both tracks separately and get perfect speaker separation. I agree that this would be the ideal solution, but it may require some fundamental changes to the internal architecture of noScribe.

[spackmat]

Do you mean mixing it down to a mono track?

I mixed my two audio tracks to one stereo track, where the first track is the left channel and the second is the right channel. This is what my ffmpeg command does. That way I still had a clear separation that I hoped would help to separate the speakers. If that is not the case, then a mono mix sounds like a better solution.

[kaixxx]

Ah, I see. So if you have a stereo track with one speaker in the left channel and one in the right, noScribe is transcribing it correctly? This is what I was hoping, but never really tested.

I understand now that you have two separate files that you want to combine into one transcript. This clashes a little bit with the default behavior implemented in the upcoming version 0.7: If multiple files are selected, multiple transcription jobs will be created and placed in the queue (which is also new in 0.7). I don't want to change this now, since we are very close to the release. But I might consider adding a dialog in the future, asking whether multiple files should be combined into one transcript or not.

[spackmat]

I understand now that you have two separate files that you want to combine into one transcript.

No, not two files, just one mp4 file with multiple audio tracks. One for me (local, straight from the mic) and one from the interviewee (coming from PC audio out of the video conference). In OBS Studio you can record your audio sources to seperate tracks in the output file.

From those multiple tracks, noScribe only uses the first at the moment. So a downmix to one (stereo) track has to be done prior.

My resulting audio with one speaker left and the other right worked quite well then.

[gernophil]

Wouldn't it be easier to make two (or more) separate transcripts by for every track and then combine the outputs outside of noScribe?

[spackmat]

Wouldn't it be easier to make two (or more) separate transcripts by for every track and then combine the outputs outside of noScribe?

What do you mean by easier? It would roughly take double the processing time, also a preparation of the input file and adds a (in case of overlapping speakers messy) manual merging step for the resulting transcript. Versus a quite well working speaker detection on the downmix.

All my other files (coming directly from Teams with its useless transcription) had a stereo downmix. The transcription with noScribe was much better, despite it does not have access to the isolated speaker streams. The separation is cleaner in Teams, but the transcription is so bad that it twists the meaning of what is said in many places. Astonishing.

[gernophil]

To me it doesn't make sense, if you already have you're speakers in two different files (or tracks, which is basically the same) and then merge them just to have them separated by AI again or do I get anything wrong here? You can disable the speaker recognition, which should speed things up a lot, if you already have them separated.

[gernophil]

And of course the merging shouldn't happen manually, but programmatic using the time codes.

[spackmat]

I accept the loss of accuracy here and it is working good enough. I'm always looking for a simple to explain workflow so that I can tell my students how to use it on their own. An extra step to merge two transcripts together adds complexity to an already overwhelming task. Maybe I'm just missing a simple and automatically working tool for that.

(Too) many of my students even don't know how to work with a real computer, as they are used only to iPads that were their workhorses at school. So even the splitting up or downmix of the multitrack audio with ffmpeg looks like wizardry for those students. (I'm aware that we ourselves in the late 1990s were able to figure that stuff out even without anyone teaching us and without well written tutorials on the internet. And that that need made us the mature users we are now.)