Problem with tokenizer
Hi,
I get this problem:
'''
ValueError Traceback (most recent call last)
in <cell line: 13>()
11
12 feature_extractor = AutoFeatureExtractor.from_pretrained('vukpetar/trocr-small-photomath')
---> 13 tokenizer = AutoTokenizer.from_pretrained("vukpetar/trocr-small-photomath")
14 model = VisionEncoderDecoderModel.from_pretrained('vukpetar/trocr-small-photomath')
15 pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
4 frames
/usr/local/lib/python3.9/dist-packages/transformers/tokenization_utils_fast.py in init(self, *args, **kwargs)
118 fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
119 else:
--> 120 raise ValueError(
121 "Couldn't instantiate the backend tokenizer from one of: \n"
122 "(1) a tokenizers library serialization file, \n"
ValueError: Couldn't instantiate the backend tokenizer from one of:
(1) a tokenizers library serialization file,
(2) a slow tokenizer instance to convert or
(3) an equivalent slow tokenizer class to instantiate and convert.
You need to have sentencepiece installed to convert a slow tokenizer to a fast one.
'''