---
language:
- ms
---
# Pretrain BASE 512 masking context length DebertaV2 on Malaysian text

Special thanks to https://github.com/aisyahrzk for pretraining DebertaV2 Base.

WanDB at https://wandb.ai/aisyahrazak/deberta-base?nw=nwuseraisyahrazak