Fix extremely long model_max_length
#1
by
cnut1648
- opened
Fix model_max_length to be 4k
The tokenizer length is set super high so that it never truncates when tokenizing -- it's deceptively named but this is the correct value
emozilla
changed pull request status to
closed