SnakyMcSnekFace
/

Psyfighter2-13B-vore

Text Generation

Not-For-All-Audiences

text-generation-inference

Model card Files Files and versions Community

SnakyMcSnekFace commited on Apr 7

Commit

00ea061

•

1 Parent(s): 19bd185

Update README.md

Files changed (1) hide show

README.md +2 -0

README.md CHANGED Viewed

@@ -92,6 +92,8 @@ Training parameters:
 - Sample size: 768 tokens
 - Samples per epoch: 47420
 - Number of epochs: 2
 - First epoch: Learning rate = 3e-4, 1000 steps warmup, cosine schedule
 - Second epoch: Learning rate = 1e-4, 256 steps warmup, inverse sqrt schedule

 - Sample size: 768 tokens
 - Samples per epoch: 47420
 - Number of epochs: 2
+- Batch size: 1
+- Gradient accumulation steps: 16
 - First epoch: Learning rate = 3e-4, 1000 steps warmup, cosine schedule
 - Second epoch: Learning rate = 1e-4, 256 steps warmup, inverse sqrt schedule