Update README.md

README.md

@@ -1,3 +1,36 @@
----
-license: llama3
----
+---
+license: llama3
+datasets:
+- yuyijiong/Long-Instruction-with-Paraphrasing
+language:
+- zh
+- en
+library_name: peft
+pipeline_tag: text-generation
+---
+
+# Llama3-8b-chinese-chat-32k 
+
+## 训练方式
+
+* 使用 NTK-aware 方法扩展上下文长度至32k
+
+* 以 [shenzhi-wang/Llama3-8B-Chinese-Chat](https://huggingface.co/shenzhi-wang/Llama3-8B-Chinese-Chat) 为基础
+在 [Long-Instruction-with-Paraphrasing](https://huggingface.co/datasets/yuyijiong/Long-Instruction-with-Paraphrasing)
+数据集上，使用 QLora 微调 1 epoch。
+
+
+## 长上下文表现
+相比原始版本，拥有更强的长上下文能力
+
+### LongBench (en)
+| model                     | hotpotqa  | multifieldqa_en| passage_retrieval_en|qmsum| trec|
+|---------------------------|-----------|--|--|--|--|
+| llama3-chinese-8b         | 45.88     |50.56|68.0|22.52|73.0|
+| llama3-8b-chinese-chat-32k| **47.64** |49.98|**100.0**|**25.13**|**75.0**|
+
+### LongBench (zh)
+| model                             | dureader  | multifieldqa_zh| passage_retrieval_zh|qmsum| trec|
+|-----------------------------------|-----------|--|--|--|--|
+| llama3-8b-chinese-chat            | 29.08     |58.4|93.5|22.52|73.0|
+| llama3-8b-chinese-chat-32k | **32.31** |**58.66**|82.5|**25.13**|**75.0**|