It seems the model doesnt work well on lower temperatures
#2
by
fernandofernandes
- opened
Hi!
It seems that Zephyr-beta is running through some issues while dealing with low temperatures. The output easily gets corrupted after a relatively small number of tokens (~500). I don't know if other people are experiencing it as I am.
top_p = 0.95
temperature < 0.1
Any idea?
If more people is having same issue, I can paste more details over here.
Best,
Fernando
That looks like it might be an issue with mistral (the base model). Others have had issues with repetition, see https://huggingface.co/mistralai/Mistral-7B-v0.1/discussions/29 . Turning up the temp helped them