https://huggingface.co/instructkr/lynn-7b-alpha
iMatrix quantization (preferably, of course) of this Mistral 0.2 based model, thank you again mradermacher.
Sure! This will be my first imatrix quant of a 7b, even - I normally don't bother imatrix quants for models < 20B.
Makes sense and I agree that it isn't worth it overall, but since this is a 32K context length model, it being iMatrix quantified makes it so that I have less to bother with the perplexity of the model in lower quantization in thus trading it for more context- since I do have low VRAM to play around and context length requires more the higher you go.
Good point :) In any case, the imatrix has been prepared and a server has been allocated, so expect the quants to show up within 24 hours at most (first the static ones followed by weighted ones).