Congrats.
#4
by
Nexesenex
- opened
On Ooba Exllama, it works with a 4096 context on a RTX 3090.
Superhot 8k allows it also, but the Airoboros 33b GPTQ SH8k feels like a 13b model due to the increase in perplexity. This one finally feels like a 33b.
It also works very well with SillyTavern and the Arena presets in terms of coherence !
I can't wait to get a second 3090 and test an eventual 65b version !
Many thanks and much gratitude !