|
--- |
|
base_model: |
|
- Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss |
|
- jondurbin/bagel-dpo-8x7b-v0.2 |
|
- NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss |
|
license: cc-by-nc-4.0 |
|
tags: |
|
- not-for-all-audiences |
|
- nsfw |
|
- mergekit |
|
- merge |
|
- exllamav2 |
|
--- |
|
|
|
# Undi95/BagelMix-8x7B 3.5bpw |
|
Exllama quant of [Undi95/BagelMix-8x7B](https://huggingface.co/Undi95/BagelMix-8x7B) |
|
|
|
You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too) |
|
|
|
|
|
### Prompt format: |
|
ChatML? maybe? Unclear |
|
|
|
``` |
|
<|im_start|>system |
|
{sysprompt}<|im_end|> |
|
<|im_start|>user |
|
{input}<|im_end|> |
|
<|im_start|>assistant |
|
{output}<|im_end|> |
|
``` |
|
|
|
### Contact |
|
Kooten on discord. |
|
|
|
|
