ai-forever
/

RUDOLPH-350M

PyTorch

RUDOLPH

text-image

image-text

decoder

Model card Files Files and versions Community

shonenkov commited on Jan 31, 2022

Commit

ad95e5b

•

1 Parent(s): 8616eeb

update weights

Browse files

Files changed (2) hide show

README.md +4 -4
pytorch_model.bin +1 -1

README.md CHANGED Viewed

@@ -2,7 +2,7 @@
 RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP
-<img src="https://raw.githubusercontent.com/sberbank-ai/ru-dolph/master/pics/rudolph-generated.png?token=GHSAT0AAAAAABQH6MSSZP3PVXVAVWPGGYHKYOYRU4A" height="60" border="2"/>
 Model was trained by [Sber AI](https://github.com/sberbank-ai) and [SberDevices](https://sberdevices.ru/) teams.
@@ -10,7 +10,7 @@ Model was trained by [Sber AI](https://github.com/sberbank-ai) and [SberDevices]
 * Language: `Russian`
 * Type: `encoder-decoder`
 * Num Parameters: `350M`
-* Training Data Volume: `35 million text-image pairs`
 # Model Description
@@ -23,9 +23,9 @@ Model was trained by [Sber AI](https://github.com/sberbank-ai) and [SberDevices]
 The primary proposed method is to modify the sparse transformer's attention mask to better control multi-modalities and up to the next level with "hyper-modality". It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with image condition without attention to left text.
-<img src="https://raw.githubusercontent.com/sberbank-ai/ru-dolph/master/pics/attention_masks.png?token=GHSAT0AAAAAABQH6MSTFSJ6ICVIJOU7S7OAYOYRVGQ" height="40" border="2"/>
 # Authors
 + Alex Shonenkov: [Github](https://github.com/shonenkov), [Kaggle GM](https://www.kaggle.com/shonenkov)
-+ Michael Konstantinov: [Mishin Learning](https://t.me/mishin_learning), [Transformer Community](https://transformer.community/)

 RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP
+<img src="https://raw.githubusercontent.com/sberbank-ai/ru-dolph/master/pics/rudolph-generated.png" height="60" border="2"/>
 Model was trained by [Sber AI](https://github.com/sberbank-ai) and [SberDevices](https://sberdevices.ru/) teams.
 * Language: `Russian`
 * Type: `encoder-decoder`
 * Num Parameters: `350M`
+* Training Data Volume: `156 million text-image pairs`
 # Model Description
 The primary proposed method is to modify the sparse transformer's attention mask to better control multi-modalities and up to the next level with "hyper-modality". It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with image condition without attention to left text.
+<img src="https://raw.githubusercontent.com/sberbank-ai/ru-dolph/master/pics/attention_masks.png" height="40" border="2"/>
 # Authors
 + Alex Shonenkov: [Github](https://github.com/shonenkov), [Kaggle GM](https://www.kaggle.com/shonenkov)
++ Michael Konstantinov: [Mishin Learning](https://t.me/mishin_learning), [Transformer Community](https://transformer.community/)

pytorch_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:18cc11ff7e7911ad3e18948f51d39ba6440050815e284af9f9b14064f77b2440
 size 707460385

 version https://git-lfs.github.com/spec/v1
+oid sha256:60803e0119ce050d4e9a235ab574a213e490d27cef93d90f3e6dd7495adcf7e1
 size 707460385