Casual-Autopsy
commited on
Commit
•
fd9439c
1
Parent(s):
7b66b68
Update README.md
Browse files
README.md
CHANGED
@@ -177,6 +177,23 @@ The following models were included in the merge:
|
|
177 |
* [Nitral-AI/Hathor_Stable-v0.2-L3-8B](https://huggingface.co/Nitral-AI/Hathor_Stable-v0.2-L3-8B)
|
178 |
* [Sao10K/L3-8B-Stheno-v3.1](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.1)
|
179 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
180 |
## Secret Sauce
|
181 |
|
182 |
The following YAML configurations were used to produce this model:
|
@@ -319,17 +336,4 @@ models:
|
|
319 |
merge_method: task_arithmetic
|
320 |
base_model: Casual-Autopsy/Umbral-Mind-3
|
321 |
dtype: bfloat16
|
322 |
-
```
|
323 |
-
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
324 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Casual-Autopsy__L3-Umbral-Mind-RP-v2.0-8B)
|
325 |
-
|
326 |
-
| Metric |Value|
|
327 |
-
|-------------------|----:|
|
328 |
-
|Avg. |25.76|
|
329 |
-
|IFEval (0-Shot) |71.23|
|
330 |
-
|BBH (3-Shot) |32.49|
|
331 |
-
|MATH Lvl 5 (4-Shot)|10.12|
|
332 |
-
|GPQA (0-shot) | 4.92|
|
333 |
-
|MuSR (0-shot) | 5.55|
|
334 |
-
|MMLU-PRO (5-shot) |30.26|
|
335 |
-
|
|
|
177 |
* [Nitral-AI/Hathor_Stable-v0.2-L3-8B](https://huggingface.co/Nitral-AI/Hathor_Stable-v0.2-L3-8B)
|
178 |
* [Sao10K/L3-8B-Stheno-v3.1](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.1)
|
179 |
|
180 |
+
## [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
181 |
+
|
182 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Casual-Autopsy__L3-Umbral-Mind-RP-v2.0-8B)
|
183 |
+
|
184 |
+
*Explaination for AI RP newbies:** IFEval is the most important evaluation for RP AIs as it determines how well it can follow OOC, Lorebooks, and most importantly character cards.
|
185 |
+
The rest don't matter. At least not nearly as much as IFEval.
|
186 |
+
|
187 |
+
| Metric |Value|
|
188 |
+
|-------------------|----:|
|
189 |
+
|Avg. |25.76|
|
190 |
+
|IFEval (0-Shot) |71.23|
|
191 |
+
|BBH (3-Shot) |32.49|
|
192 |
+
|MATH Lvl 5 (4-Shot)|10.12|
|
193 |
+
|GPQA (0-shot) | 4.92|
|
194 |
+
|MuSR (0-shot) | 5.55|
|
195 |
+
|MMLU-PRO (5-shot) |30.26|
|
196 |
+
|
197 |
## Secret Sauce
|
198 |
|
199 |
The following YAML configurations were used to produce this model:
|
|
|
336 |
merge_method: task_arithmetic
|
337 |
base_model: Casual-Autopsy/Umbral-Mind-3
|
338 |
dtype: bfloat16
|
339 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|