metadata
license: gpl-3.0
language:
- en
library_name: transformers
This model uses the LTG-BERT architecture. The model was trained on a combination of the BabyLM Dataset, the TinyStories Dataset, and generated data, in accordance with the rules of the Stric track, and the 100M word budget.
The model was trained with 128 token sequence length
Hyperparameters used and evaluation scores will follow in a subsequent update.