jan-hq commited on
Commit
55e94a0
1 Parent(s): ad191ce

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -10,7 +10,7 @@ tags:
10
 
11
  ## Model Details
12
 
13
- We have developed and released the family Llama-3-8B-Sound. This family is natively understanding audio and text input.
14
 
15
  We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
16
 
@@ -37,7 +37,7 @@ We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-lla
37
  ## Training process
38
  **Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
39
 
40
- ![training_loss_curve/png](https://cdn-uploads.huggingface.co/production/uploads/65713d70f56f9538679e5a56/12vqghBGus1Bb2OTjNezl.png)
41
 
42
  ### Hardware
43
 
@@ -61,7 +61,8 @@ We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-lla
61
  | **epsilon** | 1e-6 |
62
  | **Gradient Cliping** | 1.0 |
63
 
64
- ### Accelerate FSDP Config
 
65
 
66
  ```
67
  compute_environment: LOCAL_MACHINE
@@ -165,10 +166,10 @@ Despite being undertrained, the model demonstrates an emerging grasp of sound-te
165
  ```
166
  @article{Llama-3-Sound: Sound Instruction LLM 2024,
167
  title={Llama-3-Sound},
168
- author={JanAI},
169
  year=2024,
170
  month=July},
171
- url={https://huggingface.co/jan-hq/llama-3-sound-init-checkpoint-4340}
172
  ```
173
 
174
  ## Acknowledgement
 
10
 
11
  ## Model Details
12
 
13
+ We have developed and released the family [Jan-Llama3](https://huggingface.co/collections/jan-hq/jan-llama3-668e4dad446c8736208dca4f). This family is natively understanding audio and text input.
14
 
15
  We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
16
 
 
37
  ## Training process
38
  **Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
39
 
40
+ ![train_loss_curve/png](https://cdn-uploads.huggingface.co/production/uploads/65713d70f56f9538679e5a56/9bv-kpnqrTxaBhiYrVHN7.png)
41
 
42
  ### Hardware
43
 
 
61
  | **epsilon** | 1e-6 |
62
  | **Gradient Cliping** | 1.0 |
63
 
64
+ ###
65
+ Accelerate FSDP Config
66
 
67
  ```
68
  compute_environment: LOCAL_MACHINE
 
166
  ```
167
  @article{Llama-3-Sound: Sound Instruction LLM 2024,
168
  title={Llama-3-Sound},
169
+ author={Homebrew Research},
170
  year=2024,
171
  month=July},
172
+ url={https://huggingface.co/jan-hq/Jan-Llama3-0708}
173
  ```
174
 
175
  ## Acknowledgement