File size: 3,992 Bytes
93c1d72 090e573 93c1d72 090e573 93c1d72 e032123 93c1d72 e032123 93c1d72 e032123 93c1d72 e032123 93c1d72 e032123 93c1d72 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
---
library_name: transformers
tags:
- llama2
- deutsch
- german
- seedbox
license: llama2
datasets:
- seedboxai/multitask_german_examples_32k
language:
- de
- en
pipeline_tag: text-generation
---
# KafkaLM-7B-DARE_TIES-LaserRMT-QLoRA-DPO-v0.5
**KafkaLM 7b** is a Mistral 7b model - further pre-trained on a large German dataset from Björn Plüster and LAION. [leo-mistral-hessianai-7b](https://huggingface.co/LeoLM/leo-mistral-hessianai-7b) - which was finetuned on an ensemble of popular high-quality open-source instruction sets (translated from English to German).
KafkaLM 7b is a [Seedbox](https://huggingface.co/seedboxai) project trained by [Dennis Dickmann](https://huggingface.co/doubledsbv).
**Why Kafka?**
The models are proficient, yet creative, and have some tendencies to linguistically push boundaries 😊
## THE MODEL CAN BE TESTET HERE [Kafka-7B HF Space](https://huggingface.co/spaces/doubledsbv/Kafka-7B-DARE-TIES-QLoRa-LaserRMT-DPO)
## Model Details
The purpose of releasing the **KafkaLM series** is to contribute to the German AI community with a set of fine-tuned LLMs that are easy to use in everyday applications across a variety of tasks.
The main goal was to provide LLMs proficient in German, especially to be used in German-speaking business contexts where English alone is not sufficient.
## DPO Training with laserRMT w/ Q-Lora
Based on the brilliant work from [laserRMT](https://github.com/cognitivecomputations/laserRMT/) team, I used the SNR implementation for identifying candiate layers to be used for the DPO training.
### Dataset
I used a 8k filtered version of the following [seedboxai/multitask_german_examples_32k](https://huggingface.co/datasets/seedboxai/multitask_german_examples_32k)
### Prompt Format
This model follows the subsequent prompt format:
```
<|system|>
Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert und präzise, ohne dabei relevante Fakten auszulassen.</s>
<|user|>
Welche Möglichkeiten der energetischen Sanierung habe ich neben Solar und Energiespeicher?</s>
<|assistant|>
```
## 🧩 Configuration
```yaml
models:
- model: mistralai/Mistral-7B-v0.1
# no parameters necessary for base model
- model: seedboxai/KafkaLM-7B-German-V0.1
parameters:
density: 0.65
weight: 0.50
- model: mlabonne/Monarch-7B
parameters:
density: 0.60
weight: 0.30
- model: mayflowergmbh/Wiedervereinigung-7b-dpo-laser
parameters:
density: 0.60
weight: 0.20
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
int8_mask: true
dtype: bfloat16
random_seed: 0
```
## 💻 Usage (fast vLLM inference example)
```python
!pip install -qU vllm
import torch
from vllm import LLM, SamplingParams
sampling_params = SamplingParams(
temperature=0.7,
top_p=0.95,
top_k=50,
max_tokens=512,
)
llm = LLM(model="doubledsbv/KafkaLM-7B-DARE_TIES-DPO-v0.5-AWQ", quantization = "awq", dtype=torch.float16)
def generate_prompt(input, sys_prompt = None):
prompt = ''
if not sys_prompt:
sys_prompt = "Du bist ein freundlicher und hilfsbereiter KI-Assistent. Du beantwortest Fragen faktenorientiert, präzise und ausführlich."
prompt += f"<|system|>\n{sys_prompt.strip()}</s>\n"
prompt += f"<|user|>\n{input.strip()}</s>\n"
prompt += f"<|assistant|>\n"
return prompt
outputs = llm.generate(generate_prompt("Was ist der Unterschied zwischen Ironie und Sarkasmus?"), sampling_params)
primt(outputs[0].outputs[0].text.strip())
```
## Disclaimer
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model.
This model should only be used for research purposes. The original Llama2 license and all restrictions of datasets used to train this model apply. |