Files changed (1) hide show
  1. README.md +59 -0
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - tr
5
+ pipeline_tag: image-text-to-text
6
+ tags:
7
+ - Turkish
8
+ - turkish
9
+ - LLaVA
10
+ datasets:
11
+ - liuhaotian/LLaVA-CC3M-Pretrain-595K
12
+ ---
13
+
14
+ <img src="./CosmosLLaVA.png"/>
15
+
16
+ # Llava-CosmosLlama
17
+
18
+ This is a Turkish visual language model designed for multi-modal visual instruction-following tasks. It utilizes the LLaVA (Large Language and Vision Assistant) architecture, integrating the `ytucosmos/Turkish-Llama-8b-Instruct-v0.1` language model. The model is capable of processing both visual (image) and textual inputs, allowing it to understand and execute instructions provided in Turkish.
19
+
20
+ # Model Details
21
+ The model was pretrained with a translated version of the **[LLaVA-CC3M-Pretrain-595K](https://huggingface.co/datasets/liuhaotian/LLaVA-CC3M-Pretrain-595K)** dataset.<br>
22
+ It was further fine-tuned using subsets the following datasets to enhance its visual reasoning and understanding capabilities:
23
+ - **[Stanford GQA](https://cs.stanford.edu/people/dorarad/gqa/about.html)**
24
+ - **[VisualGenome](https://homes.cs.washington.edu/~ranjay/visualgenome/index.html)**
25
+ - **[COCO](https://cocodataset.org/#home)**
26
+
27
+ ## Example Usage
28
+ ```python
29
+ from lmdeploy import pipeline, ChatTemplateConfig
30
+ from lmdeploy.vl import load_image
31
+
32
+ pipe = pipeline("ytu-ce-cosmos/model_name",
33
+ chat_template_config=ChatTemplateConfig(model_name='llama3'))
34
+
35
+ url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-im-captioning.jpg"
36
+ image = load_image(url)
37
+
38
+ response = pipe(('Bu resimde öne çıkan ögeler nelerdir?', image))
39
+
40
+ print(response)
41
+ """
42
+ Resimde, çiçeklerle dolu bir bahçede yavru bir köpek ve arka planda bir ağaç yer alıyor. Köpek, çiçeklerin arasında otururken ve etrafını saran çiçeklerin arasından bakarken görülebiliyor. Bu sahne, köpeğin bahçede geçirdiği zamanın tadını çıkardığı ve çevresini keşfettiği sakin ve huzurlu bir atmosferi yansıtıyor.
43
+ """
44
+ ```
45
+
46
+ # Acknowledgments
47
+ - Computing resources used in this work were provided by the National Center for High Performance Computing of Turkey (UHeM).
48
+ - Thanks to the generous support from the Hugging Face team, it is possible to download models from their S3 storage 🤗
49
+
50
+
51
+ # Citation
52
+ ```bibtex
53
+ ...
54
+ ```
55
+
56
+ ### Contact
57
+ COSMOS AI Research Group, Yildiz Technical University Computer Engineering Department <br>
58
+ https://cosmos.yildiz.edu.tr/ <br>
59