zhichao yang
commited on
Commit
•
0f5ed99
1
Parent(s):
cd814cf
Update README.md
Browse files
README.md
CHANGED
@@ -1,11 +1,15 @@
|
|
1 |
---
|
|
|
|
|
|
|
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
|
5 |
## Introduction
|
6 |
[Google's LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://arxiv.org/pdf/2112.07916.pdf).
|
7 |
|
8 |
-
This is an unofficial longt5-large-16384-pubmed-10k_steps checkpoint. I.e., this is a large configuration of the LongT5 model with a transient-global attention fine-tuned on pubmed summarization dataset for 10,000 training steps.
|
9 |
|
10 |
## Results and Fine-tuning Details
|
11 |
|
|
|
1 |
---
|
2 |
+
language: en
|
3 |
+
datasets:
|
4 |
+
- ccdv/pubmed-summarization
|
5 |
+
|
6 |
license: apache-2.0
|
7 |
---
|
8 |
|
9 |
## Introduction
|
10 |
[Google's LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://arxiv.org/pdf/2112.07916.pdf).
|
11 |
|
12 |
+
This is an unofficial longt5-large-16384-pubmed-10k_steps checkpoint. I.e., this is a large configuration of the LongT5 model with a transient-global attention fine-tuned on [pubmed summarization dataset](https://huggingface.co/datasets/ccdv/pubmed-summarization) for 10,000 training steps.
|
13 |
|
14 |
## Results and Fine-tuning Details
|
15 |
|