|
--- |
|
language: "tr" |
|
tags: |
|
- turkish |
|
- tr |
|
- gpt2-tr |
|
- gpt2-turkish |
|
--- |
|
# 🇹🇷 Turkish GPT-2 Model |
|
|
|
In this repository I release GPT-2 model, that was trained on various texts for Turkish. |
|
|
|
The model is meant to be an entry point for fine-tuning on other texts. |
|
|
|
## Training corpora |
|
|
|
I used a Turkish corpora that is taken from oscar-corpus. |
|
|
|
It was possible to create byte-level BPE with Tokenizers library of Huggingface. |
|
|
|
With the Tokenizers library, I created a 52K byte-level BPE vocab based on the training corpora. |
|
|
|
After creating the vocab, I could train the GPT-2 for Turkish on two 2080TI over the complete training corpus (five epochs). |
|
|
|
Logs during training: |
|
https://tensorboard.dev/experiment/3AWKv8bBTaqcqZP5frtGkw/#scalars |
|
|
|
## Model weights |
|
|
|
Both PyTorch and Tensorflow compatible weights are available. |
|
|
|
| Model | Downloads |
|
| --------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
| `redrussianarmy/gpt2-turkish-cased` | [`config.json`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/config.json) • [`merges.txt`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/merges.txt) • [`pytorch_model.bin`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/pytorch_model.bin) • [`special_tokens_map.json`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/special_tokens_map.json) • [`tf_model.h5`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/tf_model.h5) • [`tokenizer_config.json`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/tokenizer_config.json) • [`traning_args.bin`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/training_args.bin) • [`vocab.json`](https://huggingface.co/redrussianarmy/gpt2-turkish-cased/resolve/main/vocab.json) |
|
|
|
## Using the model |
|
|
|
The model itself can be used in this way: |
|
|
|
``` python |
|
from transformers import AutoTokenizer, AutoModelWithLMHead |
|
tokenizer = AutoTokenizer.from_pretrained("redrussianarmy/gpt2-turkish-cased") |
|
model = AutoModelWithLMHead.from_pretrained("redrussianarmy/gpt2-turkish-cased") |
|
``` |
|
|
|
Here's an example that shows how to use the great Transformers Pipelines for generating text: |
|
|
|
``` python |
|
from transformers import pipeline |
|
pipe = pipeline('text-generation', model="redrussianarmy/gpt2-turkish-cased", |
|
tokenizer="redrussianarmy/gpt2-turkish-cased", config={'max_length':800}) |
|
text = pipe("Akşamüstü yolda ilerlerken, ")[0]["generated_text"] |
|
print(text) |
|
``` |
|
|
|
### How to clone the model repo? |
|
``` |
|
git lfs install |
|
git clone https://huggingface.co/redrussianarmy/gpt2-turkish-cased |
|
``` |
|
|
|
## Contact (Bugs, Feedback, Contribution and more) |
|
For questions about the GPT2-Turkish model, just open an issue [here](https://github.com/redrussianarmy/gpt2-turkish/issues) 🤗 |
