GuiCas commited on
Commit
fe1c913
1 Parent(s): 9c336ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +172 -7
README.md CHANGED
@@ -1,22 +1,187 @@
1
  ---
2
  language:
3
- - en
4
- license: apache-2.0
5
  tags:
6
  - text-generation-inference
7
  - transformers
8
  - unsloth
9
- - mistral
10
- - gguf
11
  base_model: unsloth/Phi-3-mini-4k-instruct-bnb-4bit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
  # Uploaded model
15
 
16
  - **Developed by:** GuiCas
17
- - **License:** apache-2.0
18
  - **Finetuned from model :** unsloth/Phi-3-mini-4k-instruct-bnb-4bit
19
 
20
- This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
-
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  language:
3
+ - pt
4
+ license: mit
5
  tags:
6
  - text-generation-inference
7
  - transformers
8
  - unsloth
9
+ - phi
 
10
  base_model: unsloth/Phi-3-mini-4k-instruct-bnb-4bit
11
+ datasets:
12
+ - dominguesm/Canarim-Instruct-PTBR-Dataset
13
+ model-index:
14
+ - name: Phituguese_FP16
15
+ results:
16
+ - task:
17
+ type: text-generation
18
+ name: Text Generation
19
+ dataset:
20
+ name: ENEM Challenge (No Images)
21
+ type: eduagarcia/enem_challenge
22
+ split: train
23
+ args:
24
+ num_few_shot: 3
25
+ metrics:
26
+ - type: acc
27
+ value: 49.97
28
+ name: accuracy
29
+ source:
30
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
31
+ name: Open Portuguese LLM Leaderboard
32
+ - task:
33
+ type: text-generation
34
+ name: Text Generation
35
+ dataset:
36
+ name: BLUEX (No Images)
37
+ type: eduagarcia-temp/BLUEX_without_images
38
+ split: train
39
+ args:
40
+ num_few_shot: 3
41
+ metrics:
42
+ - type: acc
43
+ value: 43.25
44
+ name: accuracy
45
+ source:
46
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
47
+ name: Open Portuguese LLM Leaderboard
48
+ - task:
49
+ type: text-generation
50
+ name: Text Generation
51
+ dataset:
52
+ name: OAB Exams
53
+ type: eduagarcia/oab_exams
54
+ split: train
55
+ args:
56
+ num_few_shot: 3
57
+ metrics:
58
+ - type: acc
59
+ value: 38.13
60
+ name: accuracy
61
+ source:
62
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
63
+ name: Open Portuguese LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: Assin2 RTE
69
+ type: assin2
70
+ split: test
71
+ args:
72
+ num_few_shot: 15
73
+ metrics:
74
+ - type: f1_macro
75
+ value: 74.75
76
+ name: f1-macro
77
+ source:
78
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
79
+ name: Open Portuguese LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: Assin2 STS
85
+ type: eduagarcia/portuguese_benchmark
86
+ split: test
87
+ args:
88
+ num_few_shot: 15
89
+ metrics:
90
+ - type: pearson
91
+ value: 71.93
92
+ name: pearson
93
+ source:
94
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
95
+ name: Open Portuguese LLM Leaderboard
96
+ - task:
97
+ type: text-generation
98
+ name: Text Generation
99
+ dataset:
100
+ name: FaQuAD NLI
101
+ type: ruanchaves/faquad-nli
102
+ split: test
103
+ args:
104
+ num_few_shot: 15
105
+ metrics:
106
+ - type: f1_macro
107
+ value: 43.97
108
+ name: f1-macro
109
+ source:
110
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
111
+ name: Open Portuguese LLM Leaderboard
112
+ - task:
113
+ type: text-generation
114
+ name: Text Generation
115
+ dataset:
116
+ name: HateBR Binary
117
+ type: ruanchaves/hatebr
118
+ split: test
119
+ args:
120
+ num_few_shot: 25
121
+ metrics:
122
+ - type: f1_macro
123
+ value: 57.34
124
+ name: f1-macro
125
+ source:
126
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
127
+ name: Open Portuguese LLM Leaderboard
128
+ - task:
129
+ type: text-generation
130
+ name: Text Generation
131
+ dataset:
132
+ name: PT Hate Speech Binary
133
+ type: hate_speech_portuguese
134
+ split: test
135
+ args:
136
+ num_few_shot: 25
137
+ metrics:
138
+ - type: f1_macro
139
+ value: 60.48
140
+ name: f1-macro
141
+ source:
142
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
143
+ name: Open Portuguese LLM Leaderboard
144
+ - task:
145
+ type: text-generation
146
+ name: Text Generation
147
+ dataset:
148
+ name: tweetSentBR
149
+ type: eduagarcia/tweetsentbr_fewshot
150
+ split: test
151
+ args:
152
+ num_few_shot: 25
153
+ metrics:
154
+ - type: f1_macro
155
+ value: 61.11
156
+ name: f1-macro
157
+ source:
158
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=GuiCas/Phituguese_FP16
159
+ name: Open Portuguese LLM Leaderboard
160
  ---
161
 
162
  # Uploaded model
163
 
164
  - **Developed by:** GuiCas
165
+ - **License:** mit
166
  - **Finetuned from model :** unsloth/Phi-3-mini-4k-instruct-bnb-4bit
167
 
 
 
168
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
169
+
170
+
171
+ # Open Portuguese LLM Leaderboard Evaluation Results
172
+
173
+ Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/GuiCas/Phituguese_FP16) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
174
+
175
+ | Metric | Value |
176
+ |--------------------------|---------|
177
+ |Average |**55.66**|
178
+ |ENEM Challenge (No Images)| 49.97|
179
+ |BLUEX (No Images) | 43.25|
180
+ |OAB Exams | 38.13|
181
+ |Assin2 RTE | 74.75|
182
+ |Assin2 STS | 71.93|
183
+ |FaQuAD NLI | 43.97|
184
+ |HateBR Binary | 57.34|
185
+ |PT Hate Speech Binary | 60.48|
186
+ |tweetSentBR | 61.11|
187
+