README.md

---
library_name: transformers
tags:
- Medical Visual Question Answering
- VQA
- IDEFIC
- 9B
- 4 Bit
- LORA
- Combining base with Adapter models
license: apache-2.0
---

# Contributed by:
- Shashwath P
- Shashank Ashok
- Akilan Yohendiran


# Total downloads all time - 2106

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
The following model is an experimental fine tuned model of the 
IDEFIC 9B version, for medical Visual Question Answering.
It uses a dataset combined from SLAKE and VQARAD.
Check the following repository for the notebooks of training,merging and inference.
https://github.com/Shashwathp/Idefic_medical_vqa


### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [@Shashwath01,@Akill19,@Shashank91097 ]
- **Model type:** [Multimodal, Visual Question Answering]
- **Language(s) (NLP):** [English]
- **License:** [Apache - 2.0]
- **Finetuned from model [optional]:** [IDEFIC 9B]

### Dataset

https://huggingface.co/datasets/Shashwath01/VQARAD_SLAKE

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/Shashwathp/Idefic_medical_vqa
<!--- **Paper :** https://ieeexplore.ieee.org/document/10616779-->


## How to Get Started with the Model

Check the below link to get started with inferencing.
https://github.com/Shashwathp/Idefic_medical_vqa/blob/main/inference.ipynb


<!--## Citation

 If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. 

[1] S. Punneshetty, S. Ashok, M. Niranjanamurthy, and S. V. N. Murthy, "Fine Tuning Idefic 9b With LORA for Multimodal Medical VQA," in *Proceedings of the 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS)*, India, Apr. 2024, pp. 1-8. DOI: 10.1109/ICKECS61492.2024.10616779.-->