add Kandinsky 2.0 - the first multilingual text2image model

[0-NiK-0]

Ability to write Prompt in more than 100 languages.

Kandinsky 2.0
https://github.com/ai-forever/Kandinsky-2.0
https://huggingface.co/sberbank-ai/Kandinsky_2.0
https://fusionbrain.ai/diffusion

Model architecture:
It is a latent diffusion model with two multilingual text encoders:

mCLIP-XLMR 560M parameters
mT5-encoder-small 146M parameters
These encoders and multilingual training datasets unveil the real multilingual text-to-image generation experience!

Kandinsky 2.0 was trained on a large 1B multilingual set, including samples that we used to train Kandinsky.

In terms of diffusion architecture Kandinsky 2.0 implements UNet with 1.2B parameters.

Kandinsky 2.0 architecture overview:

[comfyanonymous]

How well does this perform compared to stable diffusion?

[0-NiK-0]

Demo
https://t.me/kandinsky21_bot
https://rudalle.ru/kandinsky2
https://huggingface.co/spaces/ai-forever/Kandinsky2.1
Info
https://habr.com/ru/companies/sberbank/articles/725282/
https://www.reddit.com/r/StableDiffusion/comments/12bf5k2/kandinsky_21_beats_stable_diffusion_and_allows/

[AkoZ]

as huggingface is down for K2.1, mlay be hard to make it emerge ?

[ke1ne]

@comfyanonymous It seems to be a product of russian bank. The Russia is doing a genocidal war against Ukraine, killing, torturing and raping people with a one reason: just being not Russian. It's a country blackmailing a world with nuclear weapons.
It will be totally anti-ethical to support these guys in any way.

[playlogitech]

@comfyanonymous It seems to be a product of russian bank. The Russia is doing a genocidal war against Ukraine, killing, torturing and raping people with a one reason: just being not Russian. It's a country blackmailing a world with nuclear weapons. It will be totally anti-ethical to support these guys in any way.

Lets cancel everything russian just because of actions of 1 guy in kremlin, yea. Brainless propaganda works well

[ke1ne]

@comfyanonymous It seems to be a product of russian bank. The Russia is doing a genocidal war against Ukraine, killing, torturing and raping people with a one reason: just being not Russian. It's a country blackmailing a world with nuclear weapons. It will be totally anti-ethical to support these guys in any way.

Lets cancel everything russian just because of actions of 1 guy in kremlin, yea. Brainless propaganda works well

It's not Putin shoots people, launches missiles and rapes children. It's a "regular good russian guy", with family and friends, who decided to kill innocent people because of "I'm poor kitty, I have no choise, it's all orders of bad Putin, I'm a real victim!"
It's a propaganda BS! Russian people in Russian army kill Ukrainians using money from Russian local businesses (taxes) and foreign businesses who didn't stop to work with them. They can stop just now, but they deciding to continue.

[playlogitech]

@comfyanonymous It seems to be a product of russian bank. The Russia is doing a genocidal war against Ukraine, killing, torturing and raping people with a one reason: just being not Russian. It's a country blackmailing a world with nuclear weapons. It will be totally anti-ethical to support these guys in any way.

Lets cancel everything russian just because of actions of 1 guy in kremlin, yea. Brainless propaganda works well

It's not Putin shoots people, launches missiles and rapes children. It's a "regular good russian guy", with family and friends, who decided to kill innocent people because of "I'm poor kitty, I have no choise, it's all orders of bad Putin, I'm a real victim!" It's a propaganda BS! Russian people in Russian army kill Ukrainians using money from Russian local businesses (taxes) and foreign businesses who didn't stop to work with them. They can stop just now, but they deciding to continue.

They cant stop and will not. This is not how dictatorship countries works.

This russian bank is private business same as other private companies, like mail.ru group, yandex. They just works and doing their own things, many of their employees do not even live in russia long ago.

[osi1880vr]

I do not comment on the russia bashing, or any bashing here I just try to not take any side here and go away from any political point of view...
But if we would ask ourselfs at any time for the political correctnes of the country where code is coming from, how could we ever use any code from any coder in the world. Any big country in the civil world has done bad things at some point in history( if you dont think so start reading history books). so how can we accept any use of any code from any country at any time? in my world which is maybe very small one ;) there is no space for politics in developing code. if we let out politics from code we might help to create a better world in a way where people learn there is ways to deal with each other outside of politics and propaganda. thats my 2 cent to that. I did see how well that kandinsky thing performs and it is a shame not to use it.
And if you still think youre from the good guys and others are the bad guys, just think again and think about what happens right now to the guy that runs wikileaks.... (edit) and why it happens to that guy.

[0-NiK-0]

Kandinsky 2.1 inherits best practicies from Dall-E 2 and Latent diffucion, while introducing some new ideas.

As text and image encoder it uses CLIP model and diffusion image prior (mapping) between latent spaces of CLIP modalities. This approach increases the visual performance of the model and unveils new horizons in blending images and text-guided image manipulation.

For diffusion mapping of latent spaces we use transformer with num_layers=20, num_heads=32 and hidden_size=2048.

Other architecture parts:

Text encoder (XLM-Roberta-Large-Vit-L-14) - 560M
Diffusion Image Prior — 1B
CLIP image encoder (ViT-L/14) - 427M
Latent Diffusion U-Net - 1.22B
MoVQ encoder/decoder - 67M
Kandinsky 2.1 was trained on a large-scale image-text dataset LAION HighRes and fine-tuned on our internal datasets.

[AkoZ]

Ok.. very interested in art generations for may be its freedom inside (yep may be ?) .. then to use it with comfyUI, maybe there is a start to integrate it now ?

NB: ... a better world even/against dictator fools. Then kandinsky2.1 is here, let's use it !

[v4r4rth]

@comfyanonymous any update on whether this model will see support on ComfyUI?
Looking forward to seeing new possibilities outside Stable Diffusion based models!

[okolenmi]

This russian bank is private business same as other private companies, like mail.ru group, yandex. They just works and doing their own things, many of their employees do not even live in russia long ago.

Yandex and mail.ru supporting crimes against Ukraine in all ways they can do inside of Russia. They will remove any hatefull content towards putin / war / etc. They will help to spread misinformation immediately if a random guy from government tell them so. I have used it, so I know how this "propaganda ecosystem" works. You have very bad examples of companies, I would say.

[y27390641-sudo]

Интересно, что Kandinsky хорошо работает не только с английскими, но и с русскими описаниями. На практике результат сильно зависит от формулировки запроса и самой модели. Я для сравнения разных вариантов иногда использую https://ranvik.ru/image — удобно быстро проверить несколько идей и выбрать наиболее удачный результат.