Update README.md

README.md

@@ -1,45 +1,137 @@
 ---
-base_model: []
+license: llama3.1
+language:
+- en
 library_name: transformers
 tags:
 - mergekit
 - merge
-
+base_model:
+- meta-llama/Meta-Llama-3.1-70B-Instruct
+- NousResearch/Hermes-3-Llama-3.1-70B
+- abacusai/Dracarys-Llama-3.1-70B-Instruct
+- VAGOsolutions/Llama-3.1-SauerkrautLM-70b-Instruct
 ---
-# Brinebreath-Llama-70B
-
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
-
-## Merge Details
-### Merge Method
-
-This model was merged using the SLERP merge method.
-
-### Models Merged
-
-The following models were included in the merge:
-* /Volumes/external/VAGOsolutions_Llama-3-SauerkrautLM-70b-Instruct
-* /Volumes/external2/models/Drake-Llama-3.1-70B
-
-### Configuration
-
-The following YAML configuration was used to produce this model:
-
-```yaml
-base_model: /Volumes/external/VAGOsolutions_Llama-3-SauerkrautLM-70b-Instruct
-dtype: bfloat16
-merge_method: slerp
-parameters:
-  t:
-  - filter: self_attn
-    value: [0.0, 0.5, 0.3, 0.7, 1.0]
-  - filter: mlp
-    value: [1.0, 0.5, 0.7, 0.3, 0.0]
-  - value: 0.5
-slices:
-- sources:
-  - layer_range: [0, 80]
-    model: /Volumes/external/VAGOsolutions_Llama-3-SauerkrautLM-70b-Instruct
-  - layer_range: [0, 80]
-    model: /Volumes/external2/models/Drake-Llama-3.1-70B
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/649dc85249ae3a68334adcc6/yDDOz1fsWfSviCGtCh3f3.png)
+**Brinebreath-Llama-3.1-70B**
+=====================================
+
+I made this since I started having some problems with Cathallama. This seems to behave well.
+
+**Notable Performance**
+
+* 7% overall success rate increase on MMLU-PRO over LLaMA 3.1 70b at Q4_0
+* Strong performance in MMLU-PRO categories overall
+* Great performance during manual testing
+
+**Creation workflow**
+=====================
+**Models merged**
+* meta-llama/Meta-Llama-3.1-70B-Instruct
+* NousResearch/Hermes-3-Llama-3.1-70B
+* abacusai/Dracarys-Llama-3.1-70B-Instruct
+* VAGOsolutions/Llama-3.1-SauerkrautLM-70b-Instruct
+
 ```
+flowchart TD
+    A[Hermes 3] -->|Merge with| B[Meta-Llama-3.1]
+    C[Dracarys] -->|Merge with| D[Meta-Llama-3.1]
+    B -->| | E[Merge]
+    D -->| | E[Merge]
+    G[SauerkrautLM] -->|Merge with| E[Merge]
+    E[Merge] -->| | F[Brinebreath]
+```
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/649dc85249ae3a68334adcc6/3cjOUfghMD2GvxL7a3SOh.png)
+
+**Testing**
+=====================
+
+**Hyperparameters**
+---------------
+
+* **Temperature**: 0.0 for automated, 0.9 for manual
+* **Penalize repeat sequence**: 1.05
+* **Consider N tokens for penalize**: 256
+* **Penalize repetition of newlines**
+* **Top-K sampling**: 40
+* **Top-P sampling**: 0.95
+* **Min-P sampling**: 0.05
+
+**LLaMAcpp Version**
+------------------
+
+* b3527-2-g2d5dd7bb
+* -fa -ngl -1 -ctk f16 --no-mmap
+
+**Tested Files**
+------------------
+
+* Brinebreath-Llama-3.1-70B.Q4_0.gguf
+* Meta-Llama-3.1-70B-Instruct.Q4_0.gguf
+
+
+**Manual testing**
+
+| Category | Test Case | Brinebreath-Llama-3.1-70B.Q4_0.gguf | Meta-Llama-3.1-70B-Instruct.Q4_0.gguf |
+| --- | --- | --- | --- |
+| **Common Sense** | Ball on cup | OK | OK | 
+|  | Big duck small horse | OK | OK |
+|  | Killers | OK | OK |
+|  | Strawberry r's | <span style="color: red;">KO</span> | <span style="color: red;">KO</span> |
+|  | 9.11 or 9.9 bigger | <span style="color: red;">KO</span> | <span style="color: red;">KO</span> |
+|  | Dragon or lens | <span style="color: red;">KO</span> | <span style="color: red;">KO</span> |
+|  | Shirts | OK | <span style="color: red;">KO</span> |
+|  | Sisters | OK | <span style="color: red;">KO</span> |
+|  | Jane faster | OK | OK |
+| **Programming** | JSON | OK | OK |
+|  | Python snake game | OK | <span style="color: red;">KO</span> |
+| **Math** | Door window combination | OK | <span style="color: red;">KO</span> |
+| **Smoke** | Poem |  OK | OK |
+|  | Story |  OK | OK |
+
+*Note: See [sample_generations.txt](https://huggingface.co/gbueno86/Brinebreath-Llama-3.1-70B/blob/main/sample_generations.txt) on the main folder of the repo for the raw generations.*
+
+**MMLU-PRO**
+
+| Model | Success % |
+| --- | --- |
+| Brinebreath-3.1-70B.Q4_0.gguf | **49.0%** |
+| Meta-Llama-3.1-70B-Instruct.Q4_0.gguf | 42.0% |
+
+
+| MMLU-PRO category| Brinebreath-3.1-70B.Q4_0.gguf | Meta-Llama-3.1-70B-Instruct.Q4_0.gguf |
+| --- | --- | --- |
+| Business | **45.0%** | 40.0% |
+| Law | **40.0%** | 35.0% |
+| Psychology | **85.0%** | 80.0% |
+| Biology | **80.0%** | 75.0% |
+| Chemistry | **50.0%** | 45.0% |
+| History | **65.0%** | 60.0% |
+| Other | **55.0%** | 50.0% |
+| Health | **70.0%** | 65.0% |
+| Economics | **80.0%** | 75.0% |
+| Math | **35.0%** | 30.0% |
+| Physics | **45.0%** | 40.0% |
+| Computer Science | **60.0%** | 55.0% |
+| Philosophy | **50.0%** | 45.0% |
+| Engineering | **45.0%** | 40.0% |
+
+Note: MMLU-PRO Overall tested with 100 questions. Categories testes with 20 questions from each category.
+
+**PubmedQA**
+
+ Model Name | Success% |
+| --- | --- |
+| Brinebreath-3.1-70B.Q4_0.gguf| **71.00%** |
+| Meta-Llama-3.1-70B-Instruct.Q4_0.gguf | 68.00% |
+
+
+Note: PubmedQA tested with 100 questions.
+
+
+**Request**
+--------------
+If you are hiring in the EU or can sponsor a visa, PM me :D