Document created on September 7 2023
Endo_Llama: Thyroid Edition (v1)
Model Description
What is it?
Endo_Llama is a large language model (LLama 2) finetuned to answer questions related to endocrinology, with a focus on thyroid-related issues in its first version. This model aims to assist healthcare professionals and researchers in obtaining better information about thyroid conditions, treatments, and best practices.
Input Models input text only.
Output Models generate text only.
Model Architecture
Model Type: Llama 2 Parameters: ~7B Training Data: Curated dataset of endocrinology questions and answers, with a focus on thyroid issues for Version 1.
Intended Use
Clinical decision support Medical research
Limitations
Not a substitute for professional medical advice May not cover all nuances of a complex medical condition Version 1 is primarily focused on thyroid-related issues
Training Data
The model is trained on a curated dataset of endocrinology questions and answers, with an emphasis on thyroid conditions.
Usage
coming soon!
library_name: peft
Training procedure
The following bitsandbytes quantization config was used during training:
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: float16
Framework versions
PEFT 0.5.0
PEFT 0.5.0
- Downloads last month
- 0