Update README.md
Browse files
README.md
CHANGED
@@ -117,3 +117,31 @@ The following hyperparameters were used during training:
|
|
117 |
- Pytorch 2.1.2+cu121
|
118 |
- Datasets 2.18.0
|
119 |
- Tokenizers 0.15.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
117 |
- Pytorch 2.1.2+cu121
|
118 |
- Datasets 2.18.0
|
119 |
- Tokenizers 0.15.1
|
120 |
+
|
121 |
+
## Citation
|
122 |
+
|
123 |
+
If you find Zephyr 141B-A35B is useful in your work, please cite the ORPO paper:
|
124 |
+
|
125 |
+
```
|
126 |
+
@misc{hong2024orpo,
|
127 |
+
title={ORPO: Monolithic Preference Optimization without Reference Model},
|
128 |
+
author={Jiwoo Hong and Noah Lee and James Thorne},
|
129 |
+
year={2024},
|
130 |
+
eprint={2403.07691},
|
131 |
+
archivePrefix={arXiv},
|
132 |
+
primaryClass={cs.CL}
|
133 |
+
}
|
134 |
+
```
|
135 |
+
|
136 |
+
You may also wish to cite the creators of this model:
|
137 |
+
|
138 |
+
```
|
139 |
+
@misc{zephyr_141b,
|
140 |
+
author = {Alvaro Bartolome and Jiwoo Hong and Noah Lee and Lewis Tunstall},
|
141 |
+
title = {Zephyr 141B A35B},
|
142 |
+
year = {2024},
|
143 |
+
publisher = {Hugging Face},
|
144 |
+
journal = {Hugging Face repository},
|
145 |
+
howpublished = {\url{https://huggingface.co/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1}}
|
146 |
+
}
|
147 |
+
```
|