README.md

---
license: apache-2.0
---

This repo contains SlovenianGPT - the best open-source base 7B LLM for Slovenian language developed by Aleksa Gordić (see [LinkedIn announcement](https://www.linkedin.com/posts/aleksagordic_sloveniangpt-7b-done-training-after-7-days-activity-7171819162397204480-8U4A?utm_source=share)).

If you're interested in more powerful instruct models for Slovenian languages feel free to reach out via email (surname.name at gmail com)/[LinkedIn](https://www.linkedin.com/in/aleksagordic).

<img src="https://cdn-uploads.huggingface.co/production/uploads/6169e6f39e1ad4ff8735710b/0vTXni9ZIGU0ltC4MLzBu.png" alt="SlovenianGPT" width="450"/>

SlovenianGPT eval results compared to Mistral 7B, LLaMA 2 7b, and Gemma (also see this [LinkedIn post](https://www.linkedin.com/posts/aleksagordic_happy-to-announce-slovenian-llm-eval-and-activity-7173332520757772288-RB0c?utm_source=share) for more info):

<img src="https://cdn-uploads.huggingface.co/production/uploads/6169e6f39e1ad4ff8735710b/bP3DS7-Zr0CR7oTBAWLn1.png" alt="SlovenianGPT eval results" width="750"/>

Instruct-SlovenianGPT eval results ([LinkedIn post](https://www.linkedin.com/posts/aleksagordic_very-happy-to-announce-sloveniangpt-instruct-activity-7181638113197248512-5jvs?utm_source=share)):

<img src="https://cdn-uploads.huggingface.co/production/uploads/6169e6f39e1ad4ff8735710b/EHEh71HvT7eAMMtzROrjm.png" alt="SlovenianGPT eval results" width="750"/>

Eval was computed using https://github.com/gordicaleksa/slovenian-llm-eval

The model was trained on tens of billions of Slovenian language tokens and is based off of [Mistral 7B](https://huggingface.co/mistralai/Mistral-7B-v0.1).

## Notes

1) SlovenianGPT is a base model and therefore does not have any moderation mechanisms.

2) Since it's a base model it won't follow your instructions as it's just a powerful autocomplete engine.

3) If you want an access to much more powerful Slovenian LLMs - feel free to reach out to me via email (surname.name at gmail com)/[LinkedIn](https://www.linkedin.com/in/aleksagordic).

# Credits

The data for the project was obtained with the help of [Nikola Ljubešić](https://nljubesi.github.io/).

**Also a big thank you to the following individuals:**
- [**Aleksander Segedi**](https://www.linkedin.com/in/aleksander-segedi-08430936/) - for help around bookkeeping!

## Citation

```
@article{SlovenianGPT,
  author    = "Gordić Aleksa",
  title     = "SlovenianGPT - an open-source LLM for Slovenian language",
  year      = "2024"
  howpublished = {\url{https://huggingface.co/gordicaleksa/SlovenianGPT}},
}
```