kromeurus commited on
Commit
ce9cc0f
1 Parent(s): 325e6e0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +83 -4
README.md CHANGED
@@ -3,11 +3,11 @@ base_model:
3
  - ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
4
  - gradientai/Llama-3-8B-Instruct-Gradient-1048k
5
  - ArliAI/Llama-3.1-8B-ArliAI-Formax-v1.0
6
- - Sao10K/L3-8B-Niitama-v1
7
  - Sao10K/L3-8B-Stheno-v3.3-32K
8
  - tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b
9
  - Sao10K/L3-8B-Tamamo-v1
10
- - vicgalle/Roleplay-Hermes-3-Llama-3.1-8B
11
  library_name: transformers
12
  tags:
13
  - mergekit
@@ -39,6 +39,85 @@ Smooth Sampl: 0.18
39
 
40
  ### Merge Theory
41
 
42
- This sucked
43
 
44
- ### Config
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  - ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
4
  - gradientai/Llama-3-8B-Instruct-Gradient-1048k
5
  - ArliAI/Llama-3.1-8B-ArliAI-Formax-v1.0
6
+ - Sao10K/L3.1-8B-Niitama-v1.1
7
  - Sao10K/L3-8B-Stheno-v3.3-32K
8
  - tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b
9
  - Sao10K/L3-8B-Tamamo-v1
10
+ - Edgerunners/Lyraea-large-llama-3.1
11
  library_name: transformers
12
  tags:
13
  - mergekit
 
39
 
40
  ### Merge Theory
41
 
42
+ This sucked. Repalce RP Hermes back with Edgerunners Lyraea and swapped Niitama with L3.1 Niitama.
43
 
44
+ ### Config
45
+
46
+ ```yaml
47
+ slices:
48
+ - sources:
49
+ - layer_range: [0, 16]
50
+ model: ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
51
+ - sources:
52
+ - layer_range: [16, 32]
53
+ model: gradientai/Llama-3-8B-Instruct-Gradient-1048k
54
+ parameters:
55
+ int8_mask: true
56
+ merge_method: passthrough
57
+ dtype: float32
58
+ out_dtype: bfloat16
59
+ name: formax.ext
60
+ ---
61
+ models:
62
+ - model: formax.ext
63
+ parameters:
64
+ weight: 1.1
65
+ base_model: ArliAI/Llama-3.1-8B-ArliAI-Formax-v1.0
66
+ parameters:
67
+ normalize: false
68
+ int8_mask: true
69
+ merge_method: dare_linear
70
+ dtype: float32
71
+ out_dtype: bfloat16
72
+ tokenizer_source: base
73
+ name: formaxext.3.1
74
+ ---
75
+ models:
76
+ - model: Sao10K/L3-8B-Niitama-v1
77
+ parameters:
78
+ weight: 0.5
79
+ - model: Sao10K/L3-8B-Stheno-v3.3-32K
80
+ parameters:
81
+ weight: 0.6
82
+ base_model: tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b
83
+ parameters:
84
+ normalize: false
85
+ int8_mask: true
86
+ merge_method: dare_linear
87
+ dtype: float32
88
+ out_dtype: bfloat16
89
+ tokenizer_source: base
90
+ name: siith.3.1
91
+ ---
92
+ models:
93
+ - model: siith.3.1
94
+ - model: Sao10K/L3-8B-Tamamo-v1
95
+ base_model: Edgerunners/Lyraea-large-llama-3.1
96
+ parameters:
97
+ normalize: false
98
+ int8_mask: true
99
+ merge_method: model_stock
100
+ dtype: float32
101
+ out_dtype: bfloat16
102
+ name: siithamol3.1
103
+ ---
104
+ models:
105
+ - model: siithamol3.1
106
+ parameters:
107
+ weight: [0.5, 0,8, 0.8, 0.9, 1]
108
+ density: 0.9
109
+ gamma: 0.01
110
+ - model: formaxext.3.1
111
+ parameters:
112
+ weight: [0.5, 0.2, 0.2, 0.1, 0]
113
+ density: 0.9
114
+ gamma: 0.01
115
+ base_model: siithamol3.1
116
+ parameters:
117
+ normalize: false
118
+ int8_mask: true
119
+ merge_method: breadcrumbs_ties
120
+ dtype: float32
121
+ out_dtype: bfloat16
122
+ name: siithamov3
123
+ ```