max_position_embeddings should be 4096, not 4069.
max_position_embeddings
4096
4069
Thank you! We will change it later
· Sign up or log in to comment