Update README.md
Browse files
README.md
CHANGED
|
@@ -19,4 +19,10 @@ This model is llama-3-8b-instruct from Meta (uploaded by unsloth) trained on the
|
|
| 19 |
|
| 20 |
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A4000 16GB in 130 hours for less than $20.
|
| 21 |
|
|
|
|
|
|
|
| 22 |
- https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A4000 16GB in 130 hours for less than $20.
|
| 21 |
|
| 22 |
+
Dataset used for training this model:
|
| 23 |
+
|
| 24 |
- https://huggingface.co/datasets/Replete-AI/OpenCodeInterpreterData
|
| 25 |
+
|
| 26 |
+
Qalore notebook for training:
|
| 27 |
+
|
| 28 |
+
- https://colab.research.google.com/drive/1bX4BsjLcdNJnoAf7lGXmWOgaY8yekg8p?usp=sharing
|