An attempt to make @ParasiticRogue model a tad better on longer contexts. I just ran the script, all credit for the merge goes to my friend.
Exl2 quants already being updated by him: https://huggingface.co/ParasiticRogue/RP-Stew-v2.5-34B-exl2-4.65
An attempt to make @ParasiticRogue model a tad better on longer contexts. I just ran the script, all credit for the merge goes to my friend.
Exl2 quants already being updated by him: https://huggingface.co/ParasiticRogue/RP-Stew-v2.5-34B-exl2-4.65