Update README.md
Browse files
README.md
CHANGED
@@ -12,11 +12,15 @@ inference: false
|
|
12 |
|
13 |
# T5-base pre-trained on cleaned Dutch mC4 π³π±
|
14 |
|
15 |
-
A t5
|
16 |
|
17 |
-
|
18 |
-
For a
|
|
|
19 |
the **[Netherformer π°](https://huggingface.co/spaces/flax-community/netherformer)** example application!
|
|
|
|
|
|
|
20 |
|
21 |
## Tokenizer
|
22 |
|
|
|
12 |
|
13 |
# T5-base pre-trained on cleaned Dutch mC4 π³π±
|
14 |
|
15 |
+
A [T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) v1.1 base model pre-trained from scratch on [Dutch mC4](https://huggingface.co/datasets/yhavinga/mc4_nl_cleaned). This model achieves an evaluation accuracy of 0,78 and loss 0,96.
|
16 |
|
17 |
+
* Pre-trained T5 models need to be finetuned before they can be used for downstream tasks, therefore the inference widget on the right has been turned off.
|
18 |
+
* For a fine-tuned version for summarization, see [yhavinga/t5-v1.1-base-dutch-cnn-test](https://huggingface.co/yhavinga/t5-v1.1-base-dutch-cnn-test).
|
19 |
+
* For a demo of the Dutch CNN summarization models, head over to the Hugging Face Spaces for
|
20 |
the **[Netherformer π°](https://huggingface.co/spaces/flax-community/netherformer)** example application!
|
21 |
+
* T5 paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)
|
22 |
+
|
23 |
+
![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67)
|
24 |
|
25 |
## Tokenizer
|
26 |
|