Update README.md
Browse files
README.md
CHANGED
|
@@ -80,7 +80,7 @@ Hyperparameters:
|
|
| 80 |
- learning rate: 3% linear warmup, with a peak of 3e-5 and cosine decay
|
| 81 |
- epochs: 2
|
| 82 |
- batch size: 64
|
| 83 |
-
- context length:
|
| 84 |
- DPO beta: 0.1
|
| 85 |
|
| 86 |
## Limitations of `phi-2-dpo`
|
|
|
|
| 80 |
- learning rate: 3% linear warmup, with a peak of 3e-5 and cosine decay
|
| 81 |
- epochs: 2
|
| 82 |
- batch size: 64
|
| 83 |
+
- context length: 1024
|
| 84 |
- DPO beta: 0.1
|
| 85 |
|
| 86 |
## Limitations of `phi-2-dpo`
|