Running tts on constrained hardware no gpu

[JRMeyer]

>>> nmstoker
[May 20, 2019, 11:53am]

Has anyone looked at the practicalities of running this TTS inference on
constrained hardware, such as a mobile phone / Raspberry Pi?

I haven't got to the point of trying this myself yet but it would be
useful to hear if anyone tried it and/or if it's on the road map for the
project.

I'm assuming the inference time would be measurably longer, if it's
possible at all - of course, maybe not having a GPU would be a deal
breaker (??)

If it weren't exceptionally slow it might still be reasonably usable for
a number of scenarios as it's fairly easy make the Demo server cache
results (helpful where the bulk of your responses are typically from a
common set of spoken output, which wouldn't need inference after the
initial time)

[This is an archived TTS discussion thread from discourse.mozilla.org/t/running-tts-on-constrained-hardware-no-gpu]

[JRMeyer]

>>> erogol
[May 21, 2019, 10:05am]

I don't think you can go down as much as RasPi, but if you don't use a
neural vocoder with Tacotron architecture, TTS is able to reach
real-time on CPU.

Nevertheless, our ultimate goal is to optimize all the code to be able
to run on low resource systems. Any contribution ahead is also always
welcome.

[Archived Post]

[JRMeyer]

>>> geneing
[May 21, 2019, 4:32pm]

Inference from text to mel spectrograms runs fine on an older 4 core
Intel cpu. It's about 10x faster than real time. There are many tricks
that can be used to speedup and reduce memory use further - pruning
weights, quantization to int16 or even int8. This should make it fast
enough to run on a high end phone.

WaveRNN type vocoders are fast enough for real time synthesis on a
laptop CPU and may be fast enough on a mobile with some additional
optimization.

However, while it may run fast enough it may drain the battery too fast
for some applications (e.g. audiobooks).

[Archived Post]

[JRMeyer]

>>> ljackov
[April 9, 2020, 9:12am]

Hi!

We've managed to create a C++ implementation of the Tacotron
multi-speaker embedding model based on OpenCV which runs near real-time
or faster on contemporary mobile devices. The training is done in Python
using the original MozilaTTS implementation and then the data is
converted into easier-to-read-from-CPP layout.

[Archived Post]

[JRMeyer]

>>> kms
[April 24, 2020, 11:43pm]

Can you point to any examples of executable code that does this? I'm
very interested in running TTS on constrained systems.

[Archived Post]

[JRMeyer]

>>> nmstoker
[April 26, 2020, 6:56pm]

Quick update: I've managed to get TTS up and running on a Raspberry Pi 4
as I was curious to see if it was possible. I'm planning to write up the
details soon.

In essence it's slow (as expected) but could just about be feasible for
batched speech generation (ie not live response scenarios but where
you're generating output from a chunk of text overnight, such as reading
out a report that you'd want to listen to in the morning)

It uses the released Tacotron2 model that's part of the TTS model wheel
but without the PWGAN part (haven't looked at that yet). Therefore it
uses Griffin Lim. The quality seems good although a couple of sounds
occasionally seem a little off (as if there's something up with the
phonemes being passed or something about the model being used with the
current branch, as I ended up installing from master which may have some
differences to the branch at the point the model was packaged)

Am producing some timings now but so far it varies between 0.6s and 1.5s
per character (which was more easily measured that the time of the
output audio; back of an envelop calculations suggest one hour would
take around five to six hours to produce).

[Archived Post]

[JRMeyer]

>>>carduque
[May 2, 2020, 6:28am]

Hello Neil, I'm very interested to run TTS on my Rpi4. Did you write
some instructions or highlights I could go trough? slash
Many thanks.

[Archived Post]

[JRMeyer]

>>> nmstoker
[May 4, 2020, 4:46pm]

Hi [[Archived Post]