General discussion and feedback.
@Test157t -- Not sure if it interests you, considering their license...
This model is intended for personal use only, other use is not permitted.
...but this model here is holding kind of well at the 12-16K range, the writing is interesting... It is SUPER consistent with formatting and focuses on dialogues, I'll say that.
I can't push it to 32K here but that's their tested range.
Using the new IQ4_NL/Q4_K_M/S-imatrix quants. Testing.
Update:
- kunocchini-128k still holds better.
Yeah sadly for me, using q8, very low consistency, very random. I like the assertiveness, but it is a bit dumb and all over the place to me, I still prefer EndlessRP (follow the plot a little bit better). Even a super simple NSFW scene, it gets confused quickly :/
@Varkoyote - Cheers!
very low consistency, very random. I like the assertiveness, but it is a bit dumb and all over the place to me
I used the Q4 and Q5 and didn't experience this level of incoherency.
I still prefer EndlessRP
Hey, that's solid.
But just curious... Have you tested this model here using their ST Presets? It's very important to use them because it has a very peculiar prompting format, that the presets try to match as close as possible of course.
Here:
https://huggingface.co/dreamgen/opus-v1.2-7b/tree/main/configs/silly_tavern/v2
Thank you, I'll try! What sampler settings do you recommend for this one please?
Sampling params
For story-writing and role-play, I recommend "Min P" based sampling with min_p in the range [0.01, 0.1] and with temperature in the range [0.5, 1.5], depending on your preferences. A good starting point would be min_p=0.1; temperature=0.8.
You may also benefit from setting presence, frequency and repetition penalties, especially at lower temperatures.
@Varkoyote I think this should be good: