princeton-nlp commited on
Commit
ed635e9
1 Parent(s): a437579

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -15,4 +15,8 @@ license: llama2
15
 
16
  **License**: Must comply with license of Llama2 since it's a model derived from Llama2.
17
 
18
- Sheared-LLaMA-2.7B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**. We used roughly 0.4B tokens to perform the pruning experiment.
 
 
 
 
 
15
 
16
  **License**: Must comply with license of Llama2 since it's a model derived from Llama2.
17
 
18
+ Sheared-LLaMA-2.7B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**.
19
+ We used roughly 0.4B tokens to perform the pruning experiment. This model could be a good use to study
20
+ - effective data mixtures for continued pre-training
21
+ - comparisons to other pruning techniques
22
+ - extensive evaluations to understand how pruning affects knowledge and reasoning capabilities of LLMs