Casual-Autopsy commited on
Commit
fd9439c
1 Parent(s): 7b66b68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -14
README.md CHANGED
@@ -177,6 +177,23 @@ The following models were included in the merge:
177
  * [Nitral-AI/Hathor_Stable-v0.2-L3-8B](https://huggingface.co/Nitral-AI/Hathor_Stable-v0.2-L3-8B)
178
  * [Sao10K/L3-8B-Stheno-v3.1](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.1)
179
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
180
  ## Secret Sauce
181
 
182
  The following YAML configurations were used to produce this model:
@@ -319,17 +336,4 @@ models:
319
  merge_method: task_arithmetic
320
  base_model: Casual-Autopsy/Umbral-Mind-3
321
  dtype: bfloat16
322
- ```
323
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
324
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Casual-Autopsy__L3-Umbral-Mind-RP-v2.0-8B)
325
-
326
- | Metric |Value|
327
- |-------------------|----:|
328
- |Avg. |25.76|
329
- |IFEval (0-Shot) |71.23|
330
- |BBH (3-Shot) |32.49|
331
- |MATH Lvl 5 (4-Shot)|10.12|
332
- |GPQA (0-shot) | 4.92|
333
- |MuSR (0-shot) | 5.55|
334
- |MMLU-PRO (5-shot) |30.26|
335
-
 
177
  * [Nitral-AI/Hathor_Stable-v0.2-L3-8B](https://huggingface.co/Nitral-AI/Hathor_Stable-v0.2-L3-8B)
178
  * [Sao10K/L3-8B-Stheno-v3.1](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.1)
179
 
180
+ ## [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
181
+
182
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Casual-Autopsy__L3-Umbral-Mind-RP-v2.0-8B)
183
+
184
+ *Explaination for AI RP newbies:** IFEval is the most important evaluation for RP AIs as it determines how well it can follow OOC, Lorebooks, and most importantly character cards.
185
+ The rest don't matter. At least not nearly as much as IFEval.
186
+
187
+ | Metric |Value|
188
+ |-------------------|----:|
189
+ |Avg. |25.76|
190
+ |IFEval (0-Shot) |71.23|
191
+ |BBH (3-Shot) |32.49|
192
+ |MATH Lvl 5 (4-Shot)|10.12|
193
+ |GPQA (0-shot) | 4.92|
194
+ |MuSR (0-shot) | 5.55|
195
+ |MMLU-PRO (5-shot) |30.26|
196
+
197
  ## Secret Sauce
198
 
199
  The following YAML configurations were used to produce this model:
 
336
  merge_method: task_arithmetic
337
  base_model: Casual-Autopsy/Umbral-Mind-3
338
  dtype: bfloat16
339
+ ```