rombodawg commited on
Commit
a027214
1 Parent(s): 54a6747

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -13,6 +13,8 @@ base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
13
  ---
14
  Codellama-3-8B-Finetuned-Instruct
15
 
16
- This model is llama-3-8b-instruct from metal (uploaded by unsloth) trained on the full 65k Codefeedback dataset + the additional 150k Code Feedback Filtered Instruction dataset combined. You can find that dataset linked bellow. This Ai model was trained with the new Qalore method developed by my good friend on discord and fellow Replete-Ai worker walmartbag. The Qalore method uses Qlora training along with the methods from Galore for aditional reductions in Vram allowing for llama-3-8b to be loaded on 14.5 gb of vram. This allowed this training to be completed on an rtx a4000 16gb in 130 hours for less than $20.
 
 
17
 
18
  - https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData
 
13
  ---
14
  Codellama-3-8B-Finetuned-Instruct
15
 
16
+ This model is llama-3-8b-instruct from metal (uploaded by unsloth) trained on the full 65k Codefeedback dataset + the additional 150k Code Feedback Filtered Instruction dataset combined. You can find that dataset linked bellow. This Ai model was trained with the new Qalore method developed by my good friend on discord and fellow Replete-Ai worker walmartbag.
17
+
18
+ The Qalore method uses Qlora training along with the methods from Galore for aditional reductions in Vram allowing for llama-3-8b to be loaded on 14.5 gb of vram. This allowed this training to be completed on an rtx a4000 16gb in 130 hours for less than $20.
19
 
20
  - https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData