Optimize weighted embedding extraction with pyannote 3.1

[juanmc2005]

With pyannote 3.1, we could do only 1 forward pass of the audio instead of num_speakers when extracting embeddings with weights. This is probably at least one of the causes behind the pytorch version of the wespeaker embedding model being that much slower.

This optimization would also reduce the latency of pyannote/embedding so both would need to be re-computed in the README table.

Important: we should verify that this method is also compatible with masking (e.g. in speechbrain embeddings)

[juanmc2005]

After a quick run of the diarization pipeline with 5s latency, it seems that pyannote 3.1 is hurting performance: DER=27.3 -> DER=29.1 with the exact same models and hyper-parameters.

This is most certainly due to the new API for embedding models compatible with weighted stats pooling, so contrary to my initial idea of leaving this for v0.10, I think it's more a compatibility issue and less a matter of latency optimization.

[juanmc2005]

cc @hbredin, in case you can think of any other changes that could have caused this

[hbredin]

cc @hbredin, in case you can think of any other changes that could have caused this

I can't think of any. I did not witness any change of performance in offline speaker diarization when switching from hbredin/wespeaker-voxceleb-resnet34-LM (ONNX) to pyannote/wespeaker-voxceleb-resnet34-LM(PyTorch).

[juanmc2005]

@hbredin I narrowed it down to a change in the interpolation method for the weights:

- weights = F.interpolate(
-     weights, size=num_frames, mode="linear", align_corners=False
- )
+ weights = F.interpolate(weights, size=num_frames, mode="nearest")

Any particular reason for this change? I guess interpolating before calling the model should do the trick but I'm curious.

[hbredin]

MPS support. "linear" interpolation is not yet supported with MPS backend.