|
--- |
|
library_name: transformers |
|
tags: |
|
- Medical Visual Question Answering |
|
- VQA |
|
- IDEFIC |
|
- 9B |
|
- 4 Bit |
|
- LORA |
|
- Combining base with Adapter models |
|
license: apache-2.0 |
|
--- |
|
|
|
# Contributed by: |
|
- Shashwath P |
|
- Shashank Ashok |
|
- Akilan Yohendiran |
|
|
|
|
|
# Total downloads all time - 2106 |
|
|
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
The following model is an experimental fine tuned model of the |
|
IDEFIC 9B version, for medical Visual Question Answering. |
|
It uses a dataset combined from SLAKE and VQARAD. |
|
Check the following repository for the notebooks of training,merging and inference. |
|
https://github.com/Shashwathp/Idefic_medical_vqa |
|
|
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
|
- **Developed by:** [@Shashwath01,@Akill19,@Shashank91097 ] |
|
- **Model type:** [Multimodal, Visual Question Answering] |
|
- **Language(s) (NLP):** [English] |
|
- **License:** [Apache - 2.0] |
|
- **Finetuned from model [optional]:** [IDEFIC 9B] |
|
|
|
### Dataset |
|
|
|
https://huggingface.co/datasets/Shashwath01/VQARAD_SLAKE |
|
|
|
### Model Sources |
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
- **Repository:** https://github.com/Shashwathp/Idefic_medical_vqa |
|
<!--- **Paper :** https://ieeexplore.ieee.org/document/10616779--> |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
Check the below link to get started with inferencing. |
|
https://github.com/Shashwathp/Idefic_medical_vqa/blob/main/inference.ipynb |
|
|
|
|
|
<!--## Citation |
|
|
|
If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. |
|
|
|
[1] S. Punneshetty, S. Ashok, M. Niranjanamurthy, and S. V. N. Murthy, "Fine Tuning Idefic 9b With LORA for Multimodal Medical VQA," in *Proceedings of the 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS)*, India, Apr. 2024, pp. 1-8. DOI: 10.1109/ICKECS61492.2024.10616779.--> |
|
|
