Have some errors in the oob webui instruct model
While running in chat mode, it seems to work fine, but in instruct mode, the output is broken，outputting random repeating characters, for example:
USER:hello
ASSISTANT:picture.s.S.s.K.S.s.s.S.S.S.S.s.s.S.s.S.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s
I am using oob webui (snapshot-2023-12-10) ,lzlv_70b_fp16_hf-4.65bpw-h6-exl2 works fine, is something wrong with the model or just the current version of webui doesn't have good support for exl2-2？
If it works for chat but not instruct, there's nothing wrong with the loader. You best bet for debugging is to use the notebook and put in the raw prompt template to verify that things work as expected.
You mean this?
It doesn't look right.
I added the verbose parameter, and strangely, I put the chat mode input into the notebook as is, and it also produces meaningless repetitions, or stops after outputting a bunch of garbled characters
I tried exui and had the same problem 🥹
I compiled exllamav2 myself and everything worked fine, thanks for the reply!