weifeng-chen
commited on
Commit
•
64b920f
1
Parent(s):
294f468
update name
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ tags:
|
|
13 |
- feature-extraction
|
14 |
---
|
15 |
|
16 |
-
# Taiyi-CLIP-
|
17 |
|
18 |
- Github: [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
|
19 |
- Docs: [Fengshenbang-Docs](https://fengshenbang-doc.readthedocs.io/)
|
@@ -42,15 +42,15 @@ We follow the experimental setup of CLIP to obtain powerful visual-language inte
|
|
42 |
|
43 |
| model | dataset | Top1 | Top5 |
|
44 |
| ---- | ---- | ---- | ---- |
|
45 |
-
| Taiyi-CLIP-
|
46 |
|
47 |
**Zero-Shot Text-to-Image Retrieval**
|
48 |
|
49 |
| model | dataset | Top1 | Top5 | Top10 |
|
50 |
| ---- | ---- | ---- | ---- | ---- |
|
51 |
-
| Taiyi-CLIP-
|
52 |
-
| Taiyi-CLIP-
|
53 |
-
| Taiyi-CLIP-
|
54 |
|
55 |
## 使用 Usage
|
56 |
|
@@ -65,8 +65,8 @@ import numpy as np
|
|
65 |
|
66 |
query_texts = ["一只猫", "一只狗",'两只猫', '两只老虎','一只老虎'] # 这里是输入文本的,可以随意替换。
|
67 |
# 加载Taiyi 中文 text encoder
|
68 |
-
text_tokenizer = BertTokenizer.from_pretrained("IDEA-CCNL/Taiyi-CLIP-
|
69 |
-
text_encoder = BertModel.from_pretrained("IDEA-CCNL/Taiyi-CLIP-
|
70 |
|
71 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg" # 这里可以换成任意图片的url
|
72 |
# 加载openclip的image encoder
|
|
|
13 |
- feature-extraction
|
14 |
---
|
15 |
|
16 |
+
# Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese
|
17 |
|
18 |
- Github: [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
|
19 |
- Docs: [Fengshenbang-Docs](https://fengshenbang-doc.readthedocs.io/)
|
|
|
42 |
|
43 |
| model | dataset | Top1 | Top5 |
|
44 |
| ---- | ---- | ---- | ---- |
|
45 |
+
| Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese | ImageNet1k-CN | 54.35% | 80.64% |
|
46 |
|
47 |
**Zero-Shot Text-to-Image Retrieval**
|
48 |
|
49 |
| model | dataset | Top1 | Top5 | Top10 |
|
50 |
| ---- | ---- | ---- | ---- | ---- |
|
51 |
+
| Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese | Flickr30k-CNA-test | 60.82% | 85.00% | 91.04% |
|
52 |
+
| Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese | COCO-CN-test | 60.02% | 83.95% | 93.26% |
|
53 |
+
| Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese | wukong50k | 66.85% | 92.81% | 96.69% |
|
54 |
|
55 |
## 使用 Usage
|
56 |
|
|
|
65 |
|
66 |
query_texts = ["一只猫", "一只狗",'两只猫', '两只老虎','一只老虎'] # 这里是输入文本的,可以随意替换。
|
67 |
# 加载Taiyi 中文 text encoder
|
68 |
+
text_tokenizer = BertTokenizer.from_pretrained("IDEA-CCNL/Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese")
|
69 |
+
text_encoder = BertModel.from_pretrained("IDEA-CCNL/Taiyi-CLIP-RoBERTa-326M-ViT-H-Chinese").eval()
|
70 |
|
71 |
url = "http://images.cocodataset.org/val2017/000000039769.jpg" # 这里可以换成任意图片的url
|
72 |
# 加载openclip的image encoder
|