Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ tags:
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
-
We have developed and released the family
|
14 |
|
15 |
We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
|
16 |
|
@@ -37,7 +37,7 @@ We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-lla
|
|
37 |
## Training process
|
38 |
**Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
|
39 |
|
40 |
-
![
|
41 |
|
42 |
### Hardware
|
43 |
|
@@ -61,7 +61,8 @@ We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-lla
|
|
61 |
| **epsilon** | 1e-6 |
|
62 |
| **Gradient Cliping** | 1.0 |
|
63 |
|
64 |
-
###
|
|
|
65 |
|
66 |
```
|
67 |
compute_environment: LOCAL_MACHINE
|
@@ -165,10 +166,10 @@ Despite being undertrained, the model demonstrates an emerging grasp of sound-te
|
|
165 |
```
|
166 |
@article{Llama-3-Sound: Sound Instruction LLM 2024,
|
167 |
title={Llama-3-Sound},
|
168 |
-
author={
|
169 |
year=2024,
|
170 |
month=July},
|
171 |
-
url={https://huggingface.co/jan-hq/
|
172 |
```
|
173 |
|
174 |
## Acknowledgement
|
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
+
We have developed and released the family [Jan-Llama3](https://huggingface.co/collections/jan-hq/jan-llama3-668e4dad446c8736208dca4f). This family is natively understanding audio and text input.
|
14 |
|
15 |
We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
|
16 |
|
|
|
37 |
## Training process
|
38 |
**Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
|
39 |
|
40 |
+
![train_loss_curve/png](https://cdn-uploads.huggingface.co/production/uploads/65713d70f56f9538679e5a56/9bv-kpnqrTxaBhiYrVHN7.png)
|
41 |
|
42 |
### Hardware
|
43 |
|
|
|
61 |
| **epsilon** | 1e-6 |
|
62 |
| **Gradient Cliping** | 1.0 |
|
63 |
|
64 |
+
###
|
65 |
+
Accelerate FSDP Config
|
66 |
|
67 |
```
|
68 |
compute_environment: LOCAL_MACHINE
|
|
|
166 |
```
|
167 |
@article{Llama-3-Sound: Sound Instruction LLM 2024,
|
168 |
title={Llama-3-Sound},
|
169 |
+
author={Homebrew Research},
|
170 |
year=2024,
|
171 |
month=July},
|
172 |
+
url={https://huggingface.co/jan-hq/Jan-Llama3-0708}
|
173 |
```
|
174 |
|
175 |
## Acknowledgement
|