|
--- |
|
license: other |
|
license_name: license.md |
|
license_link: LICENSE |
|
--- |
|
# Model Card for CodeFuse-CodeGeeX2-6B |
|
<p align="center"> |
|
<img src="https://modelscope.cn/api/v1/models/codefuse-ai/CodeFuse-CodeGeeX2-6B/repo?Revision=master&FilePath=LOGO.jpg&View=true" width="800"/> |
|
<p> |
|
|
|
[[中文]](#chinese) [[English]](#english) |
|
|
|
|
|
<a id="english"></a> |
|
|
|
## Model Description |
|
|
|
CodeFuse-CodeGeeX2-6B is a 6B Code-LLM finetuned by LoRA of multiple code tasks on the base model CodeGeeX2. |
|
|
|
<br> |
|
|
|
## News and Updates |
|
|
|
🔥🔥 2023-11-10 CodeFuse-CodeGeeX2-6B has been released, achieving a pass@1 (greedy decoding) score of 45.12% on HumanEval, which is a 9.22% increase compared to CodeGeeX2 35.9%. |
|
|
|
🔥🔥 2023-10-20 CodeFuse-QWen-14B technical documentation has been released. For those interested, please refer to the CodeFuse article on our WeChat official account via the provided link.(https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw) |
|
|
|
🔥🔥 2023-10-16 CodeFuse-QWen-14B has been released, achieving a pass@1 (greedy decoding) score of 48.78% on HumanEval, which is a 16% increase compared to Qwen-14b's 32.3%. |
|
|
|
🔥🔥 2023-09-27 CodeFuse-StarCoder-15B has been released, achieving a pass@1 (greedy decoding) score of 54.9% on HumanEval, which is a 21% increase compared to StarCoder's 33.6%. |
|
|
|
🔥🔥🔥 2023-09-26 We are pleased to announce the release of the [4-bit quantized version](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary) of [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary). Despite the quantization process, the model still achieves a remarkable 73.8% accuracy (greedy decoding) on the HumanEval pass@1 metric. |
|
|
|
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary) has achived 74.4% of pass@1 (greedy decoding) on HumanEval, which is SOTA results for openspurced LLMs at present. |
|
|
|
<br> |
|
|
|
## Code Community |
|
|
|
**Homepage**: 🏡 https://github.com/codefuse-ai (**Please give us your support with a Star🌟 + Fork🚀 + Watch👀**) |
|
|
|
+ If you wish to fine-tune the model yourself, you can visit ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨ |
|
|
|
+ If you wish to deploy the model yourself, you can visit ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨ |
|
|
|
+ If you wish to see a demo of the model, you can visit ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨ |
|
|
|
<br> |
|
|
|
## Performance |
|
|
|
|
|
| Model | HumanEval(pass@1) | Date | |
|
|:----------------------------|:-----------------:|:-------:| |
|
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 | |
|
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 | |
|
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 | |
|
| GPT-4(zero-shot) | 67.0% | 2023.3 | |
|
| PanGu-Coder2 15B | 61.6% | 2023.8 | |
|
| CodeLlama-34b-Python | 53.7% | 2023.8 | |
|
| CodeLlama-34b | 48.8% | 2023.8 | |
|
| GPT-3.5(zero-shot) | 48.1% | 2022.11 | |
|
| OctoCoder | 46.2% | 2023.8 | |
|
| StarCoder-15B | 33.6% | 2023.5 | |
|
| Qwen-14b | 32.3% | 2023.10 | |
|
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 | |
|
| **CodeFuse-QWen-14B** | **48.78%** | 2023.10 | |
|
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 | |
|
|
|
|
|
<br> |
|
|
|
## Requirements |
|
|
|
* python>=3.8 |
|
* pytorch>=2.0.0 |
|
* transformers==4.33.2 |
|
* Sentencepiece |
|
* CUDA 11.4 |
|
<br> |
|
|
|
## Inference String Format |
|
|
|
The inference string is a concatenated string formed by combining conversation data(system, human and bot contents) in the training data format. It is used as input during the inference process. |
|
Here is an example format of the concatenated string: |
|
|
|
```python |
|
""" |
|
<s>system |
|
System instruction |
|
<s>human |
|
Human 1st round input |
|
<s>bot |
|
Bot 1st round output<|endoftext|> |
|
<s>human |
|
Human 2nd round input |
|
<s>bot |
|
Bot 2nd round output<|endoftext|> |
|
... |
|
... |
|
... |
|
<s>human |
|
Human nth round input |
|
<s>bot |
|
{Bot output to be genreated}<|endoftext|> |
|
""" |
|
``` |
|
|
|
When applying inference, you always make your input string end with "\<s\>bot" to ask the model generating answers. |
|
|
|
|
|
## Quickstart |
|
|
|
|
|
```bash |
|
pip install transformers cpm_kernels -U |
|
pip install -r requirements.txt |
|
``` |
|
|
|
```python |
|
import torch |
|
from transformers import ( |
|
AutoTokenizer, |
|
AutoModel, |
|
) |
|
tokenizer = AutoTokenizer.from_pretrained('codefuse-ai/CodeFuse-CodeGeeX2-6B', trust_remote_code=True) |
|
tokenizer.padding_side = "left" |
|
# try 4bit loading if cuda memory not enough |
|
model = AutoModel.from_pretrained(model_dir, |
|
trust_remote_code=True, |
|
load_in_4bit=False, |
|
device_map="auto", |
|
torch_dtype=torch.bfloat16) |
|
model.eval() |
|
|
|
HUMAN_ROLE_START_TAG = "<s>human\n" |
|
BOT_ROLE_START_TAG = "<s>bot\n" |
|
|
|
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}" |
|
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda") |
|
outputs = model.generate( |
|
inputs=inputs["input_ids"], |
|
attention_mask=inputs["attention_mask"], |
|
max_new_tokens=512, |
|
top_p=0.95, |
|
temperature=0.1, |
|
do_sample=True, |
|
eos_token_id=tokenizer.eos_token_id, |
|
pad_token_id=tokenizer.pad_token_id |
|
) |
|
|
|
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True) |
|
print(gen_text[0]) |
|
``` |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<a id="chinese"></a> |
|
|
|
## 模型简介 |
|
|
|
CodeFuse-CodeGeeX2-6B 是一个通过LoRA对基座模型CodeGeeeX2进行多代码任务微调的代码大模型。 |
|
<br> |
|
|
|
## 新闻 |
|
|
|
🔥🔥 2023-11-10 开源了CodeFuse-CodeGeeX2-6B模型,在HumanEval pass@1(greedy decoding)上可以达到48.12%, 比CodeGeeX2提高了9.22%的代码能力(HumanEval) |
|
|
|
🔥🔥 2023-10-20 公布了CodeFuse-QWen-14B技术文档,感兴趣详见微信公众号CodeFuse文章:https://mp.weixin.qq.com/s/PCQPkvbvfxSPzsqjOILCDw |
|
|
|
🔥🔥 2023-10-16开源了CodeFuse-QWen-14B模型,在HumanEval pass@1(greedy decoding)上可以达到48.78%, 比Qwen-14b提高了16%的代码能力(HumanEval) |
|
|
|
🔥🔥 2023-09-27开源了CodeFuse-StarCoder-15B模型,在HumanEval pass@1(greedy decoding)上可以达到54.9%, 比StarCoder提高了21%的代码能力(HumanEval) |
|
|
|
🔥🔥🔥 2023-09-26 [CodeFuse-CodeLlama-34B 4bits](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B-4bits/summary)量化版本发布,量化后模型在HumanEval pass@1指标为73.8% (贪婪解码)。 |
|
|
|
🔥🔥🔥 2023-09-11 [CodeFuse-CodeLlama-34B](https://modelscope.cn/models/codefuse-ai/CodeFuse-CodeLlama-34B/summary)发布,HumanEval pass@1指标达到74.4% (贪婪解码), 为当前开源SOTA。 |
|
|
|
<br> |
|
|
|
## 代码社区 |
|
**大本营**: 🏡 https://github.com/codefuse-ai (**请支持我们的项目Star🌟 + Fork🚀 + Watch👀**) |
|
|
|
+ 如果您想自己微调该模型,可以访问 ✨[MFTCoder](https://github.com/codefuse-ai/MFTCoder)✨✨ |
|
|
|
+ 如果您想自己部署该模型,可以访问 ✨[FasterTransformer4CodeFuse](https://github.com/codefuse-ai/FasterTransformer4CodeFuse)✨✨ |
|
|
|
+ 如果您想观看该模型示例,可以访问 ✨[CodeFuse Demo](https://github.com/codefuse-ai/codefuse)✨✨ |
|
|
|
<br> |
|
|
|
|
|
## 评测表现 |
|
|
|
### 代码 |
|
|
|
|
|
| 模型 | HumanEval(pass@1) | 日期 | |
|
|:----------------------------|:-----------------:|:-------:| |
|
| **CodeFuse-CodeLlama-34B** | **74.4%** | 2023.9 | |
|
|**CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023.9 | |
|
| WizardCoder-Python-34B-V1.0 | 73.2% | 2023.8 | |
|
| GPT-4(zero-shot) | 67.0% | 2023.3 | |
|
| PanGu-Coder2 15B | 61.6% | 2023.8 | |
|
| CodeLlama-34b-Python | 53.7% | 2023.8 | |
|
| CodeLlama-34b | 48.8% | 2023.8 | |
|
| GPT-3.5(zero-shot) | 48.1% | 2022.11 | |
|
| OctoCoder | 46.2% | 2023.8 | |
|
| StarCoder-15B | 33.6% | 2023.5 | |
|
| Qwen-14b | 32.3% | 2023.10 | |
|
| **CodeFuse-StarCoder-15B** | **54.9%** | 2023.9 | |
|
| **CodeFuse-QWen-14B** | **48.78%** | 2023.8 | |
|
| **CodeFuse-CodeGeeX2-6B** | **45.12%** | 2023.11 | |
|
|
|
|
|
## Requirements |
|
|
|
* python>=3.8 |
|
* pytorch>=2.0.0 |
|
* transformers==4.33.2 |
|
* Sentencepiece |
|
* CUDA 11.4 |
|
<br> |
|
|
|
## 推理数据格式 |
|
|
|
推理数据为模型在训练数据格式下拼接的字符串形式,它也是推理时输入prompt拼接的方式: |
|
|
|
```python |
|
""" |
|
<s>system |
|
这是System指令 |
|
<s>human |
|
这是第1轮用户输入的问题 |
|
<s>bot |
|
这是第1轮模型生成的内容<|endoftext|> |
|
<s>human |
|
这是第2轮用户输入的问题 |
|
<s>bot |
|
这是第2轮模型生成的内容<|endoftext|> |
|
... |
|
... |
|
... |
|
<s>human |
|
这是第n轮用户输入的问题 |
|
<s>bot |
|
{模型现在要生成的内容}<|endoftext|> |
|
""" |
|
``` |
|
|
|
推理时,请确保拼接的prompt字符串以"\<s\>bot\n"结尾,引导模型生成回答。 |
|
|
|
## 快速使用 |
|
|
|
|
|
```bash |
|
pip install transformers cpm_kernels -U |
|
pip install -r requirements.txt |
|
``` |
|
|
|
```python |
|
import torch |
|
from transformers import ( |
|
AutoTokenizer, |
|
AutoModel, |
|
) |
|
tokenizer = AutoTokenizer.from_pretrained('codefuse-ai/CodeFuse-CodeGeeX2-6B', trust_remote_code=True) |
|
tokenizer.padding_side = "left" |
|
# try 4bit loading if cuda memory not enough |
|
model = AutoModel.from_pretrained(model_dir, |
|
trust_remote_code=True, |
|
load_in_4bit=False, |
|
device_map="auto", |
|
torch_dtype=torch.bfloat16) |
|
model.eval() |
|
|
|
HUMAN_ROLE_START_TAG = "<s>human\n" |
|
BOT_ROLE_START_TAG = "<s>bot\n" |
|
|
|
text = f"{HUMAN_ROLE_START_TAG}write a python function of quick sort.\n{BOT_ROLE_START_TAG}" |
|
inputs = tokenizer(text, return_tensors='pt', padding=True, add_special_tokens=False).to("cuda") |
|
outputs = model.generate( |
|
inputs=inputs["input_ids"], |
|
attention_mask=inputs["attention_mask"], |
|
max_new_tokens=512, |
|
top_p=0.95, |
|
temperature=0.1, |
|
do_sample=True, |
|
eos_token_id=tokenizer.eos_token_id, |
|
pad_token_id=tokenizer.pad_token_id |
|
) |
|
|
|
gen_text = tokenizer.batch_decode(outputs[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True) |
|
print(gen_text[0]) |
|
``` |