Surprisingly, i prefer the original Lyra4-Gutenberg-12B version over this one.
#3
by
BigBeavis
- opened
It works fine in general, but it just feels like it's less precise about following the details and sticking to the character. I also see a lot more formatting issues with this one than with the original. I don't know if it's because you combined the two datasets or because the newer dataset is of lower quality, or something else. Either way, i think i'll be sticking with the v1 for now.
Thanks for your feedback. I did use ChatML so it's possible the extra training steps compounded the formatting issues. I'll take another look at the new dataset for any issues.