| # superhot-30b-8k-no-rlhf-test-GGML | |
| Merged base LLaMA and LoRA with this: | |
| https://github.com/tloen/alpaca-lora | |
| Base LLaMA 30B: | |
| https://huggingface.co/huggyllama/llama-30b | |
| SuperHOT 30B 8k no-rlhf-test LoRA: | |
| https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test | |
| ``` sh | |
| BASE_MODEL=huggyllama_llama-30b LORA=kaiokendev_superhot-30b-8k-no-rlhf-test python export_hf_checkpoint.py | |
| ``` | |
| Converted and quantized with llama.cpp commit `447ccbe`: | |
| ``` sh | |
| python convert.py superhot-30b-8k-safetensors --outtype f32 --outfile superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin | |
| ./bin/quantize superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin superhot-30b-8k-no-rlhf-test.ggmlv3.Q2_K.bin Q2_K | |
| ``` | |