miqu-1-70b-Requant-b2007-iMat-c32_ch400-IQ3_XXS.gguf Errors
#2
by
snombler
- opened
Hey, friend. Thanks for the quants.
As mentioned above miqu-1-70b-Requant-b2007-iMat-c32_ch400-IQ3_XXS.gguf failed to load in both llama.cpp and koboldcpp.
Checked with the latest koboldcpp and llama.ccp b2013 (latest as of posting).
Ah wait, the 3bits are in a later commit, I assumed the filename was the required commit so I thought 2013 would include them but it doesn't.
Koboldcpp error is expected as well.
Closing.
snombler
changed discussion status to
closed
Sorry, I expected IQ3_XXS to be merged and be the b2007, and it was merged a bit later!
It now works in LlamaCPP master, and Lostruins might need to adapt KCPP to allow it to work as well because I tried to make an experimental build with it and failed.