prem-1B / added_tokens.json
goku's picture
Upload tokenizer
207f1fa verified
raw
history blame
106 Bytes
{
"<|end_header_id|>": 32001,
"<|eot_id|>": 32002,
"<|start_header_id|>": 32000,
"[PAD]": 32003
}