leaderboard-pr-bot commited on
Commit
cfa6a68
1 Parent(s): 9ff20f5

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +118 -9
README.md CHANGED
@@ -1,12 +1,7 @@
1
  ---
2
- License: agpl-3.0
3
- Language:
4
- - En
5
- Pipeline_tag: text-generation
6
- Base_model: google/gemma-2-9b
7
- Tags:
8
- - Chat
9
  license: agpl-3.0
 
 
10
  datasets:
11
  - NewEden/Claude-Instruct-5K
12
  - anthracite-org/kalo-opus-instruct-22k-no-refusal
@@ -16,8 +11,108 @@ datasets:
16
  - Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
17
  - anthracite-org/kalo_opus_misc_240827
18
  - anthracite-org/kalo_misc_part2
19
- tags:
20
- - chat
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  ---
22
 
23
  ![](https://huggingface.co/Delta-Vector/Odin-9B/resolve/main/FinalOdin9B.jpg)
@@ -207,3 +302,17 @@ The training was done for 4 epochs. We used 8 x [H100s](https://www.nvidia.com/
207
  ## Safety
208
 
209
  Nein.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
 
 
 
2
  license: agpl-3.0
3
+ tags:
4
+ - chat
5
  datasets:
6
  - NewEden/Claude-Instruct-5K
7
  - anthracite-org/kalo-opus-instruct-22k-no-refusal
 
11
  - Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
12
  - anthracite-org/kalo_opus_misc_240827
13
  - anthracite-org/kalo_misc_part2
14
+ License: agpl-3.0
15
+ Language:
16
+ - En
17
+ Pipeline_tag: text-generation
18
+ Base_model: google/gemma-2-9b
19
+ Tags:
20
+ - Chat
21
+ model-index:
22
+ - name: Odin-9B
23
+ results:
24
+ - task:
25
+ type: text-generation
26
+ name: Text Generation
27
+ dataset:
28
+ name: IFEval (0-Shot)
29
+ type: HuggingFaceH4/ifeval
30
+ args:
31
+ num_few_shot: 0
32
+ metrics:
33
+ - type: inst_level_strict_acc and prompt_level_strict_acc
34
+ value: 36.92
35
+ name: strict accuracy
36
+ source:
37
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
38
+ name: Open LLM Leaderboard
39
+ - task:
40
+ type: text-generation
41
+ name: Text Generation
42
+ dataset:
43
+ name: BBH (3-Shot)
44
+ type: BBH
45
+ args:
46
+ num_few_shot: 3
47
+ metrics:
48
+ - type: acc_norm
49
+ value: 34.83
50
+ name: normalized accuracy
51
+ source:
52
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
53
+ name: Open LLM Leaderboard
54
+ - task:
55
+ type: text-generation
56
+ name: Text Generation
57
+ dataset:
58
+ name: MATH Lvl 5 (4-Shot)
59
+ type: hendrycks/competition_math
60
+ args:
61
+ num_few_shot: 4
62
+ metrics:
63
+ - type: exact_match
64
+ value: 12.54
65
+ name: exact match
66
+ source:
67
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
68
+ name: Open LLM Leaderboard
69
+ - task:
70
+ type: text-generation
71
+ name: Text Generation
72
+ dataset:
73
+ name: GPQA (0-shot)
74
+ type: Idavidrein/gpqa
75
+ args:
76
+ num_few_shot: 0
77
+ metrics:
78
+ - type: acc_norm
79
+ value: 12.19
80
+ name: acc_norm
81
+ source:
82
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
83
+ name: Open LLM Leaderboard
84
+ - task:
85
+ type: text-generation
86
+ name: Text Generation
87
+ dataset:
88
+ name: MuSR (0-shot)
89
+ type: TAUR-Lab/MuSR
90
+ args:
91
+ num_few_shot: 0
92
+ metrics:
93
+ - type: acc_norm
94
+ value: 17.56
95
+ name: acc_norm
96
+ source:
97
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
98
+ name: Open LLM Leaderboard
99
+ - task:
100
+ type: text-generation
101
+ name: Text Generation
102
+ dataset:
103
+ name: MMLU-PRO (5-shot)
104
+ type: TIGER-Lab/MMLU-Pro
105
+ config: main
106
+ split: test
107
+ args:
108
+ num_few_shot: 5
109
+ metrics:
110
+ - type: acc
111
+ value: 33.85
112
+ name: accuracy
113
+ source:
114
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Delta-Vector/Odin-9B
115
+ name: Open LLM Leaderboard
116
  ---
117
 
118
  ![](https://huggingface.co/Delta-Vector/Odin-9B/resolve/main/FinalOdin9B.jpg)
 
302
  ## Safety
303
 
304
  Nein.
305
+
306
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
307
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Delta-Vector__Odin-9B)
308
+
309
+ | Metric |Value|
310
+ |-------------------|----:|
311
+ |Avg. |24.65|
312
+ |IFEval (0-Shot) |36.92|
313
+ |BBH (3-Shot) |34.83|
314
+ |MATH Lvl 5 (4-Shot)|12.54|
315
+ |GPQA (0-shot) |12.19|
316
+ |MuSR (0-shot) |17.56|
317
+ |MMLU-PRO (5-shot) |33.85|
318
+