kuotient commited on
Commit
6b9c2aa
1 Parent(s): 472acbb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -9,10 +9,14 @@ tags:
9
  ---
10
  # Llama-3-6B-Instruct-pruned
11
  *Experimental*
 
12
  Using [PruneMe](https://github.com/arcee-ai/PruneMe) to find minimal average distance. Thank you for awesome toolkit @arcee-ai !
13
  <img src="./distance.png" alt="distance" width="390"/>
14
  *It shows pruning the 22-30 layer is the best option, but I'm worried about drasitical change between 22 to 23.*
15
 
 
 
 
16
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
 
18
  ## Merge Details
 
9
  ---
10
  # Llama-3-6B-Instruct-pruned
11
  *Experimental*
12
+
13
  Using [PruneMe](https://github.com/arcee-ai/PruneMe) to find minimal average distance. Thank you for awesome toolkit @arcee-ai !
14
  <img src="./distance.png" alt="distance" width="390"/>
15
  *It shows pruning the 22-30 layer is the best option, but I'm worried about drasitical change between 22 to 23.*
16
 
17
+ ### Disclaimer
18
+ I haven't done any post-training (called 'healing' process as the [paper](https://arxiv.org/abs/2403.17887) suggests), will do it later but no guarantee at all.
19
+
20
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
21
 
22
  ## Merge Details