Files changed (1) hide show
  1. README.md +122 -7
README.md CHANGED
@@ -1,19 +1,121 @@
1
  ---
2
- license: other
3
- license_name: qwen
4
- license_link: >-
5
- https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT
6
  language:
7
  - en
8
  - zh
 
9
  library_name: transformers
10
- pipeline_tag: text-generation
11
- inference: false
12
  tags:
13
  - mistral
14
  - qwen
15
  - qwen1.5
16
  - qwen2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ---
18
  This is the Mistral version of [Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) model by Alibaba Cloud.
19
  The original codebase can be found at: (https://github.com/hiyouga/LLaMA-Factory/blob/main/tests/llamafy_qwen.py).
@@ -82,4 +184,17 @@ hf-causal (pretrained=Qwen1.5-7B-Chat_mistral), limit: None, provide_description
82
  | | |mc2 |0.5161|± |0.0159|
83
  |winogrande | 0|acc |0.6306|± |0.0136|
84
  ```
85
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
2
  language:
3
  - en
4
  - zh
5
+ license: other
6
  library_name: transformers
 
 
7
  tags:
8
  - mistral
9
  - qwen
10
  - qwen1.5
11
  - qwen2
12
+ license_name: qwen
13
+ license_link: https://github.com/QwenLM/Qwen/blob/main/Tongyi%20Qianwen%20LICENSE%20AGREEMENT
14
+ pipeline_tag: text-generation
15
+ inference: false
16
+ model-index:
17
+ - name: Qwen1.5-7B-Chat_mistral
18
+ results:
19
+ - task:
20
+ type: text-generation
21
+ name: Text Generation
22
+ dataset:
23
+ name: AI2 Reasoning Challenge (25-Shot)
24
+ type: ai2_arc
25
+ config: ARC-Challenge
26
+ split: test
27
+ args:
28
+ num_few_shot: 25
29
+ metrics:
30
+ - type: acc_norm
31
+ value: 24.49
32
+ name: normalized accuracy
33
+ source:
34
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
35
+ name: Open LLM Leaderboard
36
+ - task:
37
+ type: text-generation
38
+ name: Text Generation
39
+ dataset:
40
+ name: HellaSwag (10-Shot)
41
+ type: hellaswag
42
+ split: validation
43
+ args:
44
+ num_few_shot: 10
45
+ metrics:
46
+ - type: acc_norm
47
+ value: 26.69
48
+ name: normalized accuracy
49
+ source:
50
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
51
+ name: Open LLM Leaderboard
52
+ - task:
53
+ type: text-generation
54
+ name: Text Generation
55
+ dataset:
56
+ name: MMLU (5-Shot)
57
+ type: cais/mmlu
58
+ config: all
59
+ split: test
60
+ args:
61
+ num_few_shot: 5
62
+ metrics:
63
+ - type: acc
64
+ value: 25.78
65
+ name: accuracy
66
+ source:
67
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
68
+ name: Open LLM Leaderboard
69
+ - task:
70
+ type: text-generation
71
+ name: Text Generation
72
+ dataset:
73
+ name: TruthfulQA (0-shot)
74
+ type: truthful_qa
75
+ config: multiple_choice
76
+ split: validation
77
+ args:
78
+ num_few_shot: 0
79
+ metrics:
80
+ - type: mc2
81
+ value: 52.33
82
+ source:
83
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
84
+ name: Open LLM Leaderboard
85
+ - task:
86
+ type: text-generation
87
+ name: Text Generation
88
+ dataset:
89
+ name: Winogrande (5-shot)
90
+ type: winogrande
91
+ config: winogrande_xl
92
+ split: validation
93
+ args:
94
+ num_few_shot: 5
95
+ metrics:
96
+ - type: acc
97
+ value: 53.67
98
+ name: accuracy
99
+ source:
100
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
101
+ name: Open LLM Leaderboard
102
+ - task:
103
+ type: text-generation
104
+ name: Text Generation
105
+ dataset:
106
+ name: GSM8k (5-shot)
107
+ type: gsm8k
108
+ config: main
109
+ split: test
110
+ args:
111
+ num_few_shot: 5
112
+ metrics:
113
+ - type: acc
114
+ value: 0.0
115
+ name: accuracy
116
+ source:
117
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Minami-su/Qwen1.5-7B-Chat_mistral
118
+ name: Open LLM Leaderboard
119
  ---
120
  This is the Mistral version of [Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) model by Alibaba Cloud.
121
  The original codebase can be found at: (https://github.com/hiyouga/LLaMA-Factory/blob/main/tests/llamafy_qwen.py).
 
184
  | | |mc2 |0.5161|± |0.0159|
185
  |winogrande | 0|acc |0.6306|± |0.0136|
186
  ```
187
+ ```
188
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
189
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Minami-su__Qwen1.5-7B-Chat_mistral)
190
+
191
+ | Metric |Value|
192
+ |---------------------------------|----:|
193
+ |Avg. |30.49|
194
+ |AI2 Reasoning Challenge (25-Shot)|24.49|
195
+ |HellaSwag (10-Shot) |26.69|
196
+ |MMLU (5-Shot) |25.78|
197
+ |TruthfulQA (0-shot) |52.33|
198
+ |Winogrande (5-shot) |53.67|
199
+ |GSM8k (5-shot) | 0.00|
200
+