clean up
Browse files
README.md
CHANGED
|
@@ -68,13 +68,15 @@ parameters:
|
|
| 68 |
|
| 69 |
> NOTE: this is still a work-in-progress (WIP) and not completed/converged by any means, but sharing to maybe save some time for others :)
|
| 70 |
|
| 71 |
-
##
|
|
|
|
|
|
|
| 72 |
|
| 73 |
- July 4, 2022: add checkpoint with 6 additional epochs of training with the dataset summary outputs filtered to 1024 **tokens**, resolving prior issue of short summaries.
|
| 74 |
|
| 75 |
## About
|
| 76 |
|
| 77 |
-
- a checkpoint of [Stancld/longt5-tglobal-large-16384-pubmed-3k_steps](https://huggingface.co/Stancld/longt5-tglobal-large-16384-pubmed-3k_steps) trained on `kmfoda/booksum` for about
|
| 78 |
- max input lengths during training vary between 8192 and 16384 tokens depending on GPU availability. This checkpoint was **trained with 16384 tokens as the max input length for the final eight epochs**
|
| 79 |
|
| 80 |
|
|
|
|
| 68 |
|
| 69 |
> NOTE: this is still a work-in-progress (WIP) and not completed/converged by any means, but sharing to maybe save some time for others :)
|
| 70 |
|
| 71 |
+
## Updates
|
| 72 |
+
|
| 73 |
+
_As I make updates to this WIP checkpoint I will post a note here._
|
| 74 |
|
| 75 |
- July 4, 2022: add checkpoint with 6 additional epochs of training with the dataset summary outputs filtered to 1024 **tokens**, resolving prior issue of short summaries.
|
| 76 |
|
| 77 |
## About
|
| 78 |
|
| 79 |
+
- a checkpoint of [Stancld/longt5-tglobal-large-16384-pubmed-3k_steps](https://huggingface.co/Stancld/longt5-tglobal-large-16384-pubmed-3k_steps) trained on `kmfoda/booksum` for about 26 epochs
|
| 80 |
- max input lengths during training vary between 8192 and 16384 tokens depending on GPU availability. This checkpoint was **trained with 16384 tokens as the max input length for the final eight epochs**
|
| 81 |
|
| 82 |
|