new exl2 quant
Hi, can you make a new quant of this model on exl2-2? I will be very grateful, I really like this model
I'll add it to the list.
@LoneStriker how are you doing new 2_2 quants? I am not really seeing any recent change in exllamav2 code that would implement this. I also don't see discussions about it on the issues github page. Is it public code? Can I just do git pull, build exllamav2package again and then it will use the new method when I run convert.py script? Can I re-use measurement.json files and still get better ppl?
It's on the experimental branch. Turbo's still making changes to it at the moment. Once it's stable, it'll be merged into main. No measurements needed for the new quant method, though the quants themselves may take a bit longer than before (after measurement.)