Github url added along with urls of base model used
Browse files
README.md
CHANGED
@@ -1,3 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
# How to use the model?
|
2 |
|
3 |
In order to use the model use can use the class in model.py like the example below:
|
@@ -14,7 +24,7 @@ from transformers import AutoTokenizer, AutoModel
|
|
14 |
|
15 |
model = Net()
|
16 |
# If you use model on cpu you need the map_location part
|
17 |
-
model.load_state_dict(torch.load("clip_model.pt", map_location=torch.device('cpu')))
|
18 |
model.eval()
|
19 |
|
20 |
tokenizer = AutoTokenizer.from_pretrained("dbmdz/distilbert-base-turkish-cased")
|
@@ -35,7 +45,7 @@ def predict(img,text_vec):
|
|
35 |
mask = torch.Tensor(token_list["attention_mask"]).long()
|
36 |
|
37 |
|
38 |
-
image_vec, text_vec = model(input, text , mask)
|
39 |
print(F.softmax(torch.matmul(image_vec,text_vec.T),dim=1))
|
40 |
|
41 |
img = Image.open("dog.png") # A dog image
|
@@ -44,5 +54,3 @@ text_vec = ["Çimenler içinde bir köpek.","Bir köpek.","Çimenler içinde bir
|
|
44 |
predict(img,text_vec) # Probabilities for each description
|
45 |
|
46 |
```
|
47 |
-
|
48 |
-
|
|
|
1 |
+
|
2 |
+
To build the model I used Resnet18 for image part and Turkish-DistillBert for text part.
|
3 |
+
Turkish-DistillBert: [dbmdz/distilbert-base-turkish-cased]
|
4 |
+
|
5 |
+
You can get more information (and code :tada:) on how to train or use the model on my [github].
|
6 |
+
|
7 |
+
[dbmdz/distilbert-base-turkish-cased]: https://huggingface.co/dbmdz/distilbert-base-turkish-cased
|
8 |
+
|
9 |
+
[github]: https://github.com/kesimeg/turkish-clip
|
10 |
+
|
11 |
# How to use the model?
|
12 |
|
13 |
In order to use the model use can use the class in model.py like the example below:
|
|
|
24 |
|
25 |
model = Net()
|
26 |
# If you use model on cpu you need the map_location part
|
27 |
+
model.load_state_dict(torch.load("clip_model.pt", map_location=torch.device('cpu')))
|
28 |
model.eval()
|
29 |
|
30 |
tokenizer = AutoTokenizer.from_pretrained("dbmdz/distilbert-base-turkish-cased")
|
|
|
45 |
mask = torch.Tensor(token_list["attention_mask"]).long()
|
46 |
|
47 |
|
48 |
+
image_vec, text_vec = model(input, text , mask)
|
49 |
print(F.softmax(torch.matmul(image_vec,text_vec.T),dim=1))
|
50 |
|
51 |
img = Image.open("dog.png") # A dog image
|
|
|
54 |
predict(img,text_vec) # Probabilities for each description
|
55 |
|
56 |
```
|
|
|
|