Update README.md

README.md

@@ -1 +1,69 @@
-GGUF files for [Magic-Dolphin-7b](https://huggingface.co/InferenceIllusionist/Magic-Dolphin-7b)
+---
+license: apache-2.0
+---
+GGUF files for [Magic-Dolphin-7b](https://huggingface.co/InferenceIllusionist/Magic-Dolphin-7b)
+
+---
+# Magic-Dolphin-7b
+<img src="https://huggingface.co/InferenceIllusionist/Magic-Dolphin-7b/resolve/main/magic-dolphin.jfif" width="500"/>
+
+For GGUF files please look [here](https://huggingface.co/InferenceIllusionist/Magic-Dolphin-7b-GGUF)
+
+A linear merge of:
+- [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
+- [Locutusque/Hyperion-1.5-Mistral-7B](https://huggingface.co/Locutusque/Hyperion-1.5-Mistral-7B)
+- [ibm/merlinite-7b](https://huggingface.co/ibm/merlinite-7b)
+
+
+These three models showed excellent acumen in technical topics so I wanted to see how they would behave together in a merge. Several different ratios were tested before this release, in the end a higher weighting for merlinite-7b helped smooth out some edges. This model is a test of how LAB tuning is impacted by merges with models leveraging DPO.
+
+### Benchmark Performance
+| Name | Avg. | ARC | HellaSwag | MMLU | TruthfulQA | Winograde | GSM8K |
+| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
+| <b>Magic-Dolphin-7b</b> | <u><b>67.48</b></u> | 65.78 | 85.61 | 64.64 | 58.01 | <u><b>79.64</b></u> | <u><b>51.18</b></u> |
+| dolphin-2.6-mistral-7b-dpo-laser | 67.28 | 66.3 | 85.73 | 63.16 | 61.71 | 79.16 | 47.61 |
+| merlinite-7b | N/A | 63.99 | 84.37 | 64.88 | N/A | 78.24 | N/A |
+| Hyperion-1.5-Mistral-7B | 61.43 | 60.49 | 83.64 | 63.57 | 41.78 | 78.61 | 40.49 |
+
+This was my first experiment with merging models so any feedback is greatly appreciated.
+
+Uses Alpaca template.
+
+<p align="center">
+
+</p>
+
+<b>Sample Question</b>
+<img src="https://huggingface.co/InferenceIllusionist/Magic-Dolphin-7b/resolve/main/magic-dolphin.JPG" width="750"/>
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
+* [Locutusque/Hyperion-1.5-Mistral-7B](https://huggingface.co/Locutusque/Hyperion-1.5-Mistral-7B)
+* [ibm/merlinite-7b](https://huggingface.co/ibm/merlinite-7b)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: models/dolphin-2.6-mistral-7b-dpo-laser
+    parameters:
+      weight: 1.0
+  - model: models/Hyperion-1.5-Mistral-7B
+    parameters:
+      weight: 0.3
+  - model: models/merlinite-7b
+    parameters:
+      weight: 0.5
+merge_method: linear
+dtype: float16
+
+```