ManniX-ITA commited on
Commit
fab270f
1 Parent(s): 0eae476

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - merge
5
+ - mergekit
6
+ - lazymergekit
7
+ - NousResearch/Hermes-3-Llama-3.1-8B
8
+ - akjindal53244/Llama-3.1-Storm-8B
9
+ ---
10
+
11
+ # llama3.1-8b-h3storm
12
+
13
+ llama3.1-8b-h3storm is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
14
+ * [NousResearch/Hermes-3-Llama-3.1-8B](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B)
15
+ * [akjindal53244/Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B)
16
+
17
+ ## 🧩 Configuration
18
+
19
+ ```yaml
20
+ slices:
21
+ - sources:
22
+ - model: NousResearch/Hermes-3-Llama-3.1-8B
23
+ layer_range:
24
+ - 0
25
+ - 32
26
+ - model: akjindal53244/Llama-3.1-Storm-8B
27
+ layer_range:
28
+ - 0
29
+ - 32
30
+ merge_method: slerp
31
+ base_model: NousResearch/Hermes-3-Llama-3.1-8B
32
+ parameters:
33
+ t:
34
+ - filter: self_attn
35
+ value:
36
+ - 0
37
+ - 0.5
38
+ - 0.3
39
+ - 0.7
40
+ - 1
41
+ - filter: mlp
42
+ value:
43
+ - 1
44
+ - 0.5
45
+ - 0.7
46
+ - 0.3
47
+ - 0
48
+ - value: 0.5
49
+ dtype: float16
50
+
51
+ ```