---
datasets:
- infCapital/vnnews_corpus_100K
language:
- vi
---

## Base Model: LLaMa2 7B Chat HF
+ Extend vocab to 44,800 for better Vietnamese understanding
+ Continual Pre-Train with >2B tokens Vietnamese
+ Trainning profile: LoRa (rank=32, alpha=128, 16fp), 1 epoch, block size = 512. Takes 300GPU Hours x RXT4090 24GB

## Can be better use for
+ Futher training / Fine-tuning for Vietnamese tasks