metadata
title: TransGPT-7b
emoji: 📚
colorFrom: gray
colorTo: red
language:
- zh
tags:
- chatglm
- pytorch
- zh
- Text2Text-Generation
license: other
widget:
- text: 我想了解如何申请和更新驾驶证？
TransGPT
发布中文TransGPT（7B）模型
test case:
| input_text | predict |
|---|---|
| 我想了解如何申请和更新驾驶证？ | xx |
Usage
First, you pass your input through the transformer model, then you get the generated sentence.
Install package:
pip install sentencepiece
pip install transformers>=4.28.0
import torch
import transformers
from transformers import LlamaTokenizer, LlamaForCausalLM
def generate_prompt(text):
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{text}
### Response:"""
checkpoint="DUOMO-Lab/TransGPT-v0"
tokenizer = LlamaTokenizer.from_pretrained(checkpoint)
model = LlamaForCausalLM.from_pretrained(checkpoint).half().cuda()
model.eval()
text = '我想了解如何申请和更新驾驶证？'
prompt = generate_prompt(text)
input_ids = tokenizer.encode(prompt, return_tensors='pt').to('cuda')
with torch.no_grad():
output_ids = model.generate(
input_ids=input_ids,
max_new_tokens=1024,
temperature=1,
top_k=20,
top_p=0.9,
repetition_penalty=1.15
).cuda()
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(output.replace(text, '').strip())
output:
我想了解如何申请和更新驾驶证？
模型来源
release合并后的模型权重。
HuggingFace版本权重（.bin文件）可用于：
- 使用Transformers进行训练和推理
- 使用text-generation-webui搭建界面
PyTorch版本权重（.pth文件）可用于：
- 使用llama.cpp工具进行量化和部署
模型文件组成：
TransGPT
config.json
generation_config.json
pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
pytorch_model.bin.index.json
special_tokens_map.json
tokenizer.json
tokenizer.model
tokenizer_config.json
硬件要求：14G显存
微调数据集
- ～xx万条文本数据集（用于领域内预训练）：DUOMO-Lab/TransGPT-pt
- ～5.6万条对话数据（用于微调）:finetune_data
如果需要训练LLaMA模型，请参考https://github.com/DUOMO/TransGPT
Citation
@software{TransGPT,
author = {Wang Peng},
title = {DUOMO/TransGPT},
year = {2023},
url = {https://github.com/DUOMO/TransGPT},
}