What are the different files for?
#9
by
Arya123456
- opened
I am a beginner at this and was wondering which of these files is to be used in the Oobabooga WebUI? Or do I need all of them? Thanks for your help :)
gpt4-x-alpasta-30b-128g-4bit.safetensors
gpt4-x-alpasta-30b-4bit.safetensors
gpt4-x-alpasta-30b-ggml-q4_1.bin
gpt4-x-alpasta-30b-ggml-q5_0.bin
gpt4-x-alpasta-30b-ggml-q5_1.bin
I think you use the highest one your card/VRAM can support. 5_1 would be the best for the BINS, but if you can use the safetensor without the 128g in it that would be ideal, as it won't exceed your VRAM is how I understand it.
Thank you Clevnumb!