kromeurus commited on
Commit
181eb19
1 Parent(s): ab2558c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -43
README.md CHANGED
@@ -1,43 +1,66 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # merge
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the breadcrumbs merge method using parts/summanus.ds.9b as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * parts/summanusara.atp1
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: parts/summanus.ds.9b
30
- # No parameters necessary for base model
31
- - model: parts/summanusara.atp1
32
- parameters:
33
- density: [0.33, 0.01, 0.33]
34
- weight: 0.8
35
- gamma: 0.001
36
- merge_method: breadcrumbs
37
- base_model: parts/summanus.ds.9b
38
- parameters:
39
- normalize: true
40
- int8_mask: true
41
- dtype: bfloat16
42
-
43
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - kromeurus/L3-Blackened-Sunfall-15B
4
+ - Hastagaras/Jamet-8B-L3-MK.V-Blackroot
5
+ - TheDrummer/Llama-3SOME-8B-v2
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+
11
+ ---
12
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/667eea5cdebd46a5ec4dcc3d/HzAhXawzvRnvlmatPrwld.jpeg)
13
+
14
+ Well, this merge didn't go as expected, at all. Went in trying to make an 8B downscale of [Blackfall Summanus](https://huggingface.co/kromeurus/L3-Blackfall-Summanus-v0.1-15B) and a comical amount of dumb mistakes later, managed to make this surprisingly solid merge.
15
+ I don't know either, I'm still processing how this model exists because I fat-fingered my keyboard. Anyways, here is Sammanus Ara. For more information, please look at the original model card above.
16
+
17
+ ### Quants
18
+
19
+ [Q8_0 only GGUF](https://huggingface.co/kromeurus/L3-8.9B-Blackfall-SummanusAra-v0.1-Q8-GGUF) by me.
20
+
21
+ Additional quants are not available yet.
22
+
23
+ ### Details & Recommended Settings
24
+
25
+ Compared to the OG 15B version, BF Summanus Ara is surprisingly capable for it's size while keeping most of the original attributes. Obviously, won't be as verbose or colorful due to natural limitations,
26
+ though no less eloquent. A little more precise and coherent, somehow sticks to the example text to a T exactly like Aethora v2 despite not adding it into the merge. Not as chatty as expected
27
+ with the additional models, paces itself quite well.
28
+
29
+
30
+
31
+ Rec. Settings:
32
+ ```
33
+ Template: Model Default
34
+ Temperature: 1.3
35
+ Min P: 0.08
36
+ Repeat Penelty: 1.05
37
+ Repeat Penelty Tokens: 256
38
+ ```
39
+
40
+ ### Models Merged
41
+
42
+ * [kromeurus/L3-Blackened-Sunfall-15B](https://huggingface.co/kromeurus/L3-Blackened-Sunfall-15B)
43
+ * [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot)
44
+ * [TheDrummer/Llama-3SOME-8B-v2](https://huggingface.co/TheDrummer/Llama-3SOME-8B-v2)
45
+
46
+ I first made passthrough merges of the models listed above into separate parts that has aspects of what I wanted in the final model, then did a ties merge with said parts as seen below.
47
+
48
+ ### Configs
49
+
50
+ ```yaml
51
+ models:
52
+ - model: parts/summanus.ds.9b
53
+ # No parameters necessary for base model
54
+ - model: parts/summanusara.atp1
55
+ parameters:
56
+ density: [0.33, 0.01, 0.33]
57
+ weight: 0.8
58
+ gamma: 0.001
59
+ merge_method: breadcrumbs
60
+ base_model: parts/summanus.ds.9b
61
+ parameters:
62
+ normalize: true
63
+ int8_mask: true
64
+ dtype: bfloat16
65
+
66
+ ```