Upload folder using huggingface_hub

README.md

@@ -6,9 +6,10 @@ pipeline_tag: text-generation
 <div align="center">
 <h1>Llama-3-8B-Instruct-80K-QLoRA</h1>
 
-[<a href="https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon/new/docs/llama3-8b-instruct-qlora-80k.md">Blog</a>]
+<a href="https://github.com/FlagOpen/FlagEmbedding/tree/master/Long_LLM/activation_beacon/new/docs/llama3-8b-instruct-qlora-80k.md">[Data&Code]</a>
 </div>
 
+We extend the context length of Llama-3-8B-Instruct to 80K using QLoRA and 3.5K long-context training data synthesized from GPT-4. The entire training cycle is super efficient, which takes 8 hours on a 8xA800 (80G) machine. Yet, the resulted model achieves remarkable performance on a series of downstream long-context evaluation benchmarks.
 
 
 # Evaluation
@@ -43,17 +44,17 @@ We evaluate the model on [InfiniteBench](https://arxiv.org/pdf/2402.13718.pdf) u
 ## Topic Retrieval
 We evaluate the model on [Topic Retrieval](https://lmsys.org/blog/2023-06-29-longchat/) task with `[5,10,15,20,25,30,40,50,60,70]` topics.
 
-<img src="data/topic.png"></img>
+<img src="data/topic_retrieval.png"></img>
 
 
 ## MMLU
 We evaluate the model's zero-shot performance on MMLU benchmark as a reflection of its short-context capability.
 
-|Model|||||
-|:-:|:-:|:-:|:-:|:-:|
-|[meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)||
-|[gradientai/Llama-3-8B-Instruct-262k](https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k)||
-|[Llama-3-8B-Instruct-80K-QLoRA]()||
+|Model|STEM|Social Sciences|Humanities|Others|Avg|
+|:-:|:-:|:-:|:-:|:-:|:-:|
+|[meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)|0.5387|0.7566|0.6944|0.6975|0.6591|
+|[gradientai/Llama-3-8B-Instruct-262k](https://huggingface.co/NousResearch/Yarn-Mistral-7b-128k)|0.5210|0.7326|0.6715|0.6980|0.6434|
+|[Llama-3-8B-Instruct-80K-QLoRA]()|0.5310|0.7324|0.6732|0.6879|0.6444|
 
 # Environment
 ```bash