2021twitchfr-conv-bert-small-mlm / tokenizer_config.json
Francois Vieille
add redondant tokeniezr for widget
3d92e58
raw
history blame contribute delete
287 Bytes
{"sep_token": "<sep>", "pad_token": "<pad>", "cls_token": "<cls>", "mask_token": "<mask>", "special_tokens_map_file": "/mnt/twitchat/models/convbert-small-hf/special_tokens_map.json", "name_or_path": "/mnt/twitchat/models/convbert-small-hf", "tokenizer_class": "PreTrainedTokenizerFast"}