Adding a fast tokenizer

#10

by Narsil HF staff - opened May 22

←

Narsil

May 22

No thorough check was made yet.

YShow

May 25

hi, thank you for your contribution. We have supported the fast tokenizer

Narsil

May 28

Thanks !

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Cannot merge

This branch has merge conflicts in the following files:

· Sign up or log in to comment