cant see the Q5_K_M gguf quant
I can see K_S and 0_quants, but as we all know, most tests show K_M will be closest to 6_K, as it loses the least information while having similar size to k_m and performance. So, I request the authors to bring that to the list too.
i think this will be the best for users having 12gb vram card when combined with Q8 t5xxl
We do have the logic for K_M quants, I just need to go through and test what keys to keep in higher precision for _M quants for them to actually be an improvement instead of just a sidegrade lol.
@Ai11Ali hey Ali i'm using q8 which is the most close to fp16 @sameyourGPU what ever the 6_K better than q5_K_m , i made q4_K_m "dev" for lower gpu and cpu users here https://civitai.com/models/661102?modelVersionId=751359 check this also special ediotion of q5_k_m https://civitai.com/models/682369?modelVersionId=764898