Fast tokenizer

#39
by soominc - opened

Are there any plans to add a fast tokenizer or a script to convert to fast tokenizer? Thank you!

Sign up or log in to comment