zirui3
/

llm-zh-tokenizer

Model card Files Files and versions Community

llm-zh-tokenizer / README.md

zirui3's picture

Update README.md

fe523d6 over 1 year ago

|

history blame contribute delete

88 Bytes

	---
	license: cc-by-nc-4.0
	---

	Chinese tokenizer trained on Baike by using BPE algorithm