GGML format
Hi,
How can I convert this model to GGML ?
Thanks
@TheBloke As far as I can tell this is an open reproduction of LLaMA. Could you convert it to GGML/GPTQ pls? Would be great to benchmark this. And also the other two released by salesforce
I've been looking into this and would love to do it. The problem is while it uses Llama for the weights, it uses a custom tokenizer (tiktoken). So GGML support won't be possible until they add specific support for that.
Otherwise I believe it will just produce gibberish, because the text you enter and the model outputs will be converted to/from the wrong token ids.
I'll give it a test later today but I'm pretty sure it can't work atm.
Ah I see makes sense. Thanks for the rapid reply.
I see GPT-2 name in the tokenizer, maybe the trick is using existing GPT-2 GGML tokenizer example?