error in tokenizer

#5
by Algaliarept - opened

Bracket in tokenizer json makes it unreadable is that wanted? -> ({ "version": "1.0", "truncation": null, "padding": null, "added_tokens": [ Line 1

cpu_and_mobile/cpu-int4-rtn-block-32/tokenizer.json

Microsoft org

Thanks! It was a typo and has been fixed.

gugarosa changed discussion status to closed

Sign up or log in to comment