|
--- |
|
license: other |
|
--- |
|
# superhot-30b-8k-no-rlhf-test-GGML |
|
|
|
**Note: LLAMA_ROPE_SCALE from PR [#1967](https://github.com/ggerganov/llama.cpp/pull/1967) needs to be set to 0.25** |
|
|
|
Merged base LLaMA and LoRA with this: |
|
https://github.com/tloen/alpaca-lora |
|
|
|
Base LLaMA 30B: |
|
https://huggingface.co/huggyllama/llama-30b |
|
|
|
SuperHOT 30B 8k no-rlhf-test LoRA: |
|
https://huggingface.co/kaiokendev/superhot-30b-8k-no-rlhf-test |
|
|
|
``` sh |
|
BASE_MODEL=huggyllama_llama-30b LORA=kaiokendev_superhot-30b-8k-no-rlhf-test python export_hf_checkpoint.py |
|
``` |
|
|
|
Converted and quantized with llama.cpp commit `447ccbe`: |
|
|
|
``` sh |
|
python convert.py superhot-30b-8k-safetensors --outtype f32 --outfile superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin |
|
./bin/quantize superhot-30b-8k-no-rlhf-test.ggmlv3.f32.bin superhot-30b-8k-no-rlhf-test.ggmlv3.Q2_K.bin Q2_K |
|
``` |
