kromeurus commited on
Commit
325e6e0
1 Parent(s): f545d6b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -42
README.md CHANGED
@@ -1,49 +1,44 @@
1
  ---
2
- base_model: []
 
 
 
 
 
 
 
 
3
  library_name: transformers
4
  tags:
5
  - mergekit
6
  - merge
7
-
8
  ---
9
- # siithamov3
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the breadcrumbs_ties merge method using merge/siithamol3.1 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * merge/formaxext.3.1
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- base_model: merge/siithamol3.1
29
- dtype: float32
30
- merge_method: breadcrumbs_ties
31
- out_dtype: bfloat16
32
- parameters:
33
- int8_mask: 1.0
34
- normalize: 0.0
35
- slices:
36
- - sources:
37
- - layer_range: [0, 32]
38
- model: merge/siithamol3.1
39
- parameters:
40
- density: 0.9
41
- gamma: 0.01
42
- weight: [0.5, 0.0, 8.0, 0.8, 0.9, 1.0]
43
- - layer_range: [0, 32]
44
- model: merge/formaxext.3.1
45
- parameters:
46
- density: 0.9
47
- gamma: 0.01
48
- weight: [0.5, 0.2, 0.2, 0.1, 0.0]
49
  ```
 
 
 
 
 
 
 
1
  ---
2
+ base_model:
3
+ - ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
4
+ - gradientai/Llama-3-8B-Instruct-Gradient-1048k
5
+ - ArliAI/Llama-3.1-8B-ArliAI-Formax-v1.0
6
+ - Sao10K/L3-8B-Niitama-v1
7
+ - Sao10K/L3-8B-Stheno-v3.3-32K
8
+ - tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b
9
+ - Sao10K/L3-8B-Tamamo-v1
10
+ - vicgalle/Roleplay-Hermes-3-Llama-3.1-8B
11
  library_name: transformers
12
  tags:
13
  - mergekit
14
  - merge
 
15
  ---
16
+ HUZZAH, a model that's actually good! Just took seven tries. Fixed spacial understanding and literacy, toned down a little of the clingy instruct following, and more turn based RP forward.
17
+
18
+ ### Quants
19
+
20
+ [A few GGUFs](https://huggingface.co/kromquant/L3.1-Siithamo-v0.4-8B-GGUFs) by me.
21
+
22
+ ### Details & Recommended Settings
23
+
24
+ (Still testing; details subject to change)
25
+
26
+ Sticks to instructs well, dynamic writing, roleplay focused generations, and more solid intelligence. Less rambley though still outputs a bit of text. Has near perfect recall up to
27
+ 32K. Be clear and explicit with model instructs, including the intended format (Asterix, quotes, etc).
28
+
29
+ Rec. Settings:
30
+ ```
31
+ Template: L3
32
+ Temperature: 1.3
33
+ Min P: 0.1
34
+ Repeat Penalty: 1.05
35
+ Repeat Penalty Tokens: 256
36
+ Dyn Temp: 0.9-1.05 at 0.1
37
+ Smooth Sampl: 0.18
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  ```
39
+
40
+ ### Merge Theory
41
+
42
+ This sucked
43
+
44
+ ### Config