What tokenizer is used ?

#3
by Aruno - opened

Hello,

Thank you for your model. I could use it for inference and fine-tuning without problem.
However, could you provide more information about tokenizer used ?
It is a GPTNeoXTokenizer, but how did you train it ?

Have a nice day

Sign up or log in to comment