Commit
•
4e0c2b0
1
Parent(s):
1592da4
Update README.md
Browse files
README.md
CHANGED
@@ -167,7 +167,7 @@ model-index:
|
|
167 |
---
|
168 |
|
169 |
<div align="center">
|
170 |
-
<img src="https://cdn-uploads.huggingface.co/production/uploads/60420dccc15e823a685f2b03/CuMO3IjJfymC94_5qd15T.png"
|
171 |
</div>
|
172 |
|
173 |
# Model Card for Notus 7B v1
|
@@ -186,6 +186,8 @@ This model **wouldn't have been possible without the amazing [Alignment Handbook
|
|
186 |
|
187 |
Notus models are intended to be used as assistants via chat-like applications, and are evaluated with Chat (MT-Bench, AlpacaEval) and Academic (Open LLM Leaderboard) benchmarks for a direct comparison with the original Zephyr dDPO model and other 7B models.
|
188 |
|
|
|
|
|
189 |
## Model Details
|
190 |
|
191 |
### Model Description
|
@@ -320,6 +322,8 @@ Results from [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/o
|
|
320 |
| Zephyr 7B dDPO (HuggingFaceH4/zephyr-7b-beta) | 52.15 | 62.03 | 84.36 | 61.07 | **57.45** | 77.74 | 12.74 | **9.66** |
|
321 |
| argilla/notus-7b-v1 | **52.89** | **64.59** | **84.78** | **63.03** | 54.37 | **79.4** | **15.16** | 8.91 |
|
322 |
|
|
|
|
|
323 |
## Training Details
|
324 |
|
325 |
### Training Hardware
|
|
|
167 |
---
|
168 |
|
169 |
<div align="center">
|
170 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/60420dccc15e823a685f2b03/CuMO3IjJfymC94_5qd15T.png"/>
|
171 |
</div>
|
172 |
|
173 |
# Model Card for Notus 7B v1
|
|
|
186 |
|
187 |
Notus models are intended to be used as assistants via chat-like applications, and are evaluated with Chat (MT-Bench, AlpacaEval) and Academic (Open LLM Leaderboard) benchmarks for a direct comparison with the original Zephyr dDPO model and other 7B models.
|
188 |
|
189 |
+
> **Why Notus?**: Notus name comes from the ancient Greek god Notus, as a wink to Zephyr, which comes from the ancient Greek god Zephyrus; with the difference that Notus is the god of the south wind, and Zephyr the god of the west wind. More information at https://en.wikipedia.org/wiki/Anemoi.
|
190 |
+
|
191 |
## Model Details
|
192 |
|
193 |
### Model Description
|
|
|
322 |
| Zephyr 7B dDPO (HuggingFaceH4/zephyr-7b-beta) | 52.15 | 62.03 | 84.36 | 61.07 | **57.45** | 77.74 | 12.74 | **9.66** |
|
323 |
| argilla/notus-7b-v1 | **52.89** | **64.59** | **84.78** | **63.03** | 54.37 | **79.4** | **15.16** | 8.91 |
|
324 |
|
325 |
+
⚠️ A data contamination issue has been reported recently by Mistral AI, which led other researchers to explore the contamination within other datasets, and since UltraFeedback (the dataset this model has been fine-tuned on), the TruthfulQA results may be affected, so the score achieved is not realistic. See https://twitter.com/natolambert/status/1730364108078469513.
|
326 |
+
|
327 |
## Training Details
|
328 |
|
329 |
### Training Hardware
|