LiteLlama-460M-1T / tokenizer_config.json

Commit History

Update tokenizer_config.json
2f26834
verified

yuntian-deng commited on

change bos/eos/pad to <|endoftext|>
842cea9

da03 commited on

Upload tokenizer
e7bacef
verified

yuntian-deng commited on