Quant Details
EXL2 quant of Undi's merge using Bluemoon-Light's Nemo parquet. Cache 8bit seems more stable than 4bit in my testing with Nemo based models.
https://huggingface.co/Undi95/Lumimaid-Magnum-12B
https://huggingface.co/datasets/ParasiticRogue/Bluemoon-Light
Original Model Card Information
Merge of Lumimaid and Magnum as requested by some.
I used the new DELLA merge method in mergekit and added a finetune of Nemo only on Claude input, trained on 16k ctx, in the mix.
Prompt template: Mistral
<s>[INST] {input} [/INST] {output}</s>
- Downloads last month
- 19
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.