Enable LlamaTokenizerFast and AutoTokenizer to load in seconds rather than 5 minutes.

#2
Files changed (1) hide show
  1. tokenizer.json +0 -0
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff