Adding the Chinese language 汉语 for TTS

[dd-rongfa]

🚀 Feature

Motivation

Pitch

Alternatives

Additional context

[snakers4]

We are still on HSK 1, so probably a bit early to start working on Chinese, we need more understanding of the language, unlike Romanian languages, which all are kind of similar.

But the feedback on how a Chinese model would ideally accept the inputs would be highly appreciated, i.e. how will you input words? Can you ask your fellow Chinese native speakers how they would do it?

1. Via pinyin without tones, e.g. wo de mao tai da le;
2. Via pinyin with some tones, e.g. wo3 de0 mao1 tai4 da4 le0;
3. Via simplified 汉字;

I would bet on option 3. But as far as I have learned, the Chinese language has the following issues:

• The same 汉字 has multiple readings depending on context;
• Many 汉字 have similar readings;

This is somehow alleviated by the fact that probably 90% of cases can be just handled using a dictionary of bi- and tri-grams of 汉字, but for some cases some understanding of broader context would be required. Which would require googling in Chinese using Chinese VPN and Chinese websites, which would require at least HSK 3 or 4 knowledge. Chicken or egg problem, I guess.

We are solving / have partially solved the same problem in Russian. But if the community lends a hand - this of course can be done.

[snakers4]

A quick search gave me this library

• https://github.com/mozillazg/python-pinyin/

But I would need some input from the community to choose a proper tool. And ideally of course, we would need to build our own if this works.

[snakers4]

Also, we found some high quality TTS data in Chinese, but maybe some Chinese only resources can be pointed out by native speakers.

[wang-haoxian]

We are still on HSK 1, so probably a bit early to start working on Chinese, we need more understanding of the language, unlike Romanian languages, which all are kind of similar.

But the feedback on how a Chinese model would ideally accept the inputs would be highly appreciated, i.e. how will you input words? Can you ask your fellow Chinese native speakers how they would do it?

1. Via pinyin without tones, e.g. wo de mao tai da le;
2. Via pinyin with some tones, e.g. wo3 de0 mao1 tai4 da4 le0;
3. Via simplified 汉字;

I would bet on option 3. But as far as I have learned, the Chinese language has the following issues:

• The same 汉字 has multiple readings depending on context;
• Many 汉字 have similar readings;

This is somehow alleviated by the fact that probably 90% of cases can be just handled using a dictionary of bi- and tri-grams of 汉字, but for some cases some understanding of broader context would be required. Which would require googling in Chinese using Chinese VPN and Chinese websites, which would require at least HSK 3 or 4 knowledge. Chicken or egg problem, I guess.

We are solving / have partially solved the same problem in Russian. But if the community lends a hand - this of course can be done.

Personally I'd go with option 2, this is much more promising than the others. The repo https://github.com/mozillazg/python-pinyin/ you have found seems very useful to me. When we learn to read and speak at the school, that's basically what we do. Thank you again for working on this language. It's very nice of you. And I like Silero ❤️

[wang-haoxian]

As for the datasets, I think the aishell-3 is one of the most used one with Apache 2.0 license. PaddlePaddle is made by Baidu. As a big tech their choices maybe inspiring. You may want to check the dataset references in https://github.com/PaddlePaddle/PaddleSpeech#model-list

[dd-rongfa]

谢谢你。汉语确实很困难，当时只是想表达一下希望有这个功能。我们一般是输入简体汉字的。拼音只是辅助读音的。一般TTS需求都是文字（简体汉字）转语音的。最后，我一直再用你的VAD，感谢！
Thank you. Chinese is really difficult, and at that time I just wanted to express my desire to have this function. We generally enter simplified Chinese characters. Pinyin is only an auxiliary pronunciation. In general, TTS requires text (simplified Chinese characters) as input to obtain speech. Finally, I have your VAD, it's very useful, thanks! I'm not very familiar with github, should I close this issue?

[snakers4]

Let's keep the issue open as a reminder, When we have time, we will do something

[snakers4]

We implemented Chinese, but only for private commercial projects. No available resources to re-implement for public domain. But it is possible to make a compact proper TTS model that would work with Chinese tones and the Russian language at the same time.