Hugging Face's logo

tmpupload
/
superhot-30b-8k-no-rlhf-test-GGML

Model card Files Community
superhot-30b-8k-no-rlhf-test-GGML / README.md
3v324v23's picture
3v324v23
Model upload.
fb6f09d
preview code
|
raw
history blame
685 Bytes

superhot-30b-8k-no-rlhf-test-GGML

Merged base LLaMA and LoRA with this: https://github.com/tloen/alpaca-lora

Base LLaMA 30B: https://huggingface.co/huggyllama/llama-30b

SuperHOT 30B 8k no-rlhf-test LoRA: https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test

BASE_MODEL=huggyllama_llama-30b LORA=kaiokendev_superhot-30b-8k-no-rlhf-test python export_hf_checkpoint.py

Converted and quantized with llama.cpp commit 447ccbe:

python convert.py superhot-30b-8k-safetensors --outtype f32 --outfile superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin
./bin/quantize superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin superhot-30b-8k-no-rlhf-test.ggmlv3.Q2_K.bin Q2_K