victordiao
commited on
Commit
•
cde761c
1
Parent(s):
124c0fe
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,66 @@
|
|
1 |
---
|
2 |
-
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
inference: false
|
3 |
---
|
4 |
+
|
5 |
+
# Robin Model Card
|
6 |
+
|
7 |
+
## Model Details
|
8 |
+
|
9 |
+
Robin is a series of models finetuned from LLaMA on several high-quality data.
|
10 |
+
|
11 |
+
- **Developed by:** [LMFlow](https://github.com/OptimalScale/LMFlow/)
|
12 |
+
- **Model type:** An auto-regressive language model based on the transformer architecture.
|
13 |
+
- **License:** Non-commercial license
|
14 |
+
- **Finetuned from model:** [LLaMA](https://arxiv.org/abs/2302.13971).
|
15 |
+
|
16 |
+
### Model Sources
|
17 |
+
|
18 |
+
- **Repository:** https://github.com/OptimalScale/LMFlow/
|
19 |
+
- **Blog:** https://medium.com/@hkust.ml/robin-v2-launches-achieves-unparalleled-performance-on-openllm-4f6886e822c1
|
20 |
+
- **Paper:** https://arxiv.org/abs/2306.12420
|
21 |
+
- **Demo:** https://lmflow.com/
|
22 |
+
|
23 |
+
## Uses
|
24 |
+
|
25 |
+
Robin is primarily utilized for conducting research on extensive language models and chatbots, catering to users specializing in natural language processing, machine learning, and artificial intelligence research.
|
26 |
+
|
27 |
+
## How to Get Started with the Model
|
28 |
+
|
29 |
+
We provide four kinds of demos including:
|
30 |
+
|
31 |
+
- Online Service: If you don't want to run any code and just want to try our models, we deploy our instruction-tuned LLaMA you to have a try.
|
32 |
+
- Colab Chatbot (shell): An interactive shell-based chatbot for you to easily deploy a chatbot on colab.
|
33 |
+
- Colab Chatbot (web): An interactive web-based chatbot for you to easily deploy your own chatbot on colab.
|
34 |
+
- Local Deploy: We also provide a way for you to deploy your model/chatbot locally, which means you can deploy much larger model than previous three methods if you have enough resource.
|
35 |
+
|
36 |
+
Please refer to https://github.com/OptimalScale/LMFlow#demos
|
37 |
+
|
38 |
+
## Training Details
|
39 |
+
|
40 |
+
|
41 |
+
Expanding upon the initial idea of self-instruct techniques, we incorporated several different data sources and build a new dataset called [LMFlow Dataset](http://lmflow.org:5000/lmflow_data.tar.gz).
|
42 |
+
The new training split is created by merging the following datasets:
|
43 |
+
- ShareGPT: randomly sample 50K English data and 10K Chinese data from ShareGPT.
|
44 |
+
- GPT-4-LLM: 52K English data from GPT-4-LLM.
|
45 |
+
- BELLE: randomly sample 80K Chinese data from BELLE.
|
46 |
+
|
47 |
+
See more details in the "Instruction Tuning" section in our [paper](https://arxiv.org/pdf/2306.12420.pdf).
|
48 |
+
|
49 |
+
## Evaluation
|
50 |
+
|
51 |
+
Robin is evaluated with [LMFlow Benchmark](https://blog.gopenai.com/lmflow-benchmark-an-automatic-evaluation-framework-for-open-source-llms-ef5c6f142418).
|
52 |
+
See more details in this [paper](https://arxiv.org/pdf/2306.12420.pdf).
|
53 |
+
|
54 |
+
## Citation
|
55 |
+
If you find this repository useful, please consider giving ⭐ and citing our [paper](https://arxiv.org/abs/2306.12420):
|
56 |
+
|
57 |
+
```
|
58 |
+
@misc{lmflow,
|
59 |
+
author = {Shizhe Diao and Rui Pan and Hanze Dong and KaShun Shum and Jipeng Zhang and Wei Xiong and Tong Zhang},
|
60 |
+
title = {LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models},
|
61 |
+
year = {2023},
|
62 |
+
publisher = {GitHub},
|
63 |
+
journal = {GitHub repository},
|
64 |
+
howpublished = {\url{https://optimalscale.github.io/LMFlow/}},
|
65 |
+
}
|
66 |
+
```
|