indiejoseph commited on
Commit
345c4d2
1 Parent(s): e13c3b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -14
README.md CHANGED
@@ -153,17 +153,4 @@ output = tokenizer.decode(output[0], skip_special_tokens=True)
153
 
154
  The model is intended to use for Cantonese language understanding and generation tasks, it may not be suitable for other Chinese languages. The model is trained on a diverse range of Cantonese text, including news, Wikipedia, and textbooks, it may not be suitable for informal or dialectal Cantonese, it may contain bias and misinformation, please use it with caution.
155
 
156
- We found the model is not well trained on the updated Hong Kong knowledge, it may due to the corpus is not large enough to brainwash the original model. We will continue to improve the model and corpus in the future.
157
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
158
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_hon9kon9ize__CantoneseLLM-6B-preview202402)
159
-
160
- | Metric |Value|
161
- |---------------------------------|----:|
162
- |Avg. |56.93|
163
- |AI2 Reasoning Challenge (25-Shot)|55.63|
164
- |HellaSwag (10-Shot) |75.80|
165
- |MMLU (5-Shot) |63.07|
166
- |TruthfulQA (0-shot) |42.26|
167
- |Winogrande (5-shot) |74.11|
168
- |GSM8k (5-shot) |30.71|
169
-
 
153
 
154
  The model is intended to use for Cantonese language understanding and generation tasks, it may not be suitable for other Chinese languages. The model is trained on a diverse range of Cantonese text, including news, Wikipedia, and textbooks, it may not be suitable for informal or dialectal Cantonese, it may contain bias and misinformation, please use it with caution.
155
 
156
+ We found the model is not well trained on the updated Hong Kong knowledge, it may due to the corpus is not large enough to brainwash the original model. We will continue to improve the model and corpus in the future.