File size: 5,302 Bytes
bb8ac5a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 |
---
license: apache-2.0
tags:
- merge
- TIES
- Llama3
- BestMix
- Chemistry
- Einstein
- instruction-following
- conversational
- long-form-generation
- scientific
base_model:
- bunnycore/Best-Mix-Llama-3.1-8B
---
[](https://hf.co/QuantFactory)
# QuantFactory/Llama3.1-BestMix-Chem-Einstein-8B-GGUF
This is quantized version of [ZeroXClem/Llama3.1-BestMix-Chem-Einstein-8B](https://huggingface.co/ZeroXClem/Llama3.1-BestMix-Chem-Einstein-8B) created using llama.cpp
# Original Model Card
# **ZeroXClem/Llama3.1-BestMix-Chem-Einstein-8B**
**Llama3.1-BestMix-Chem-Einstein-8B** is an innovative, meticulously blended model designed to excel in **instruction-following**, **chemistry-focused tasks**, and **long-form conversational generation**. This model fuses the **best qualities** of multiple Llama3-based architectures, making it highly versatile for both general and specialized tasks. 💻🧠✨
## 🌟 **Family Tree**
This model is the result of merging the following:
- [**bunnycore/Best-Mix-Llama-3.1-8B**](https://huggingface.co/bunnycore/Best-Mix-Llama-3.1-8B): A balanced blend of top Llama models, optimized for general performance across reasoning, instruction-following, and math.
- [**USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B**](https://huggingface.co/USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B): A model specialized in **scientific knowledge** and **chemistry**, excelling in chemistry benchmarks.
- [**Weyaxi/Einstein-v6.1-Llama3-8B**](https://huggingface.co/Weyaxi/Einstein-v6.1-Llama3-8B): Fine-tuned for **long-form generation**, **conversation-heavy tasks**, and optimized with cutting-edge techniques for efficient memory usage and fast performance.
---
## 🧬 **Model Lineage**
### **A: bunnycore/Best-Mix-Llama-3.1-8B**
- A masterful **blend** of several Llama3 models like **Aurora_faustus**, **TitanFusion**, and **OpenMath2**.
- Provides a **balanced performance** in a variety of tasks such as reasoning, math, and instruction-following.
- Key contributor to the **overall versatility** of the merged model.
### **B: USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B**
- Specializes in **chemistry** and **scientific knowledge**, outperforming many larger models in **chemistry benchmarks**.
- Adds **scientific rigor** and domain-specific expertise to the merged model, making it perfect for scientific and academic tasks.
### **C: Weyaxi/Einstein-v6.1-Llama3-8B**
- Fine-tuned on a wide range of **instructive** and **conversational datasets** like **WizardLM**, **Alpaca**, and **ShareGPT**.
- Optimized for **long-form text generation** and enhanced with **xformers attention** and **flash attention** techniques for better performance.
- Key player in **dialogue-based tasks** and **long conversation generation**.
---
## 🛠️ **Merge Details**
This model was merged using the **TIES merge method**, ensuring a smooth integration of the key strengths from each contributing model. Here's the configuration used:
```yaml
yaml
Copy code
models:
- model: bunnycore/Best-Mix-Llama-3.1-8B
parameters:
density: [1, 0.7, 0.5]
weight: 1.0
- model: USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B
parameters:
density: 0.6
weight: [0.3, 0.7, 1.0]
- model: Weyaxi/Einstein-v6.1-Llama3-8B
parameters:
density: 0.4
weight:
- filter: mlp
value: 0.5
- filter: self_attn
value: 0.7
- value: 0.5
merge_method: ties
base_model: bunnycore/Best-Mix-Llama-3.1-8B
parameters:
normalize: true
int8_mask: true
dtype: float16
```
---
## 🎯 **Key Features & Capabilities**
### **1. Instruction Following & General Reasoning**:
With the foundation of **Best-Mix**, this model excels in **general-purpose reasoning**, instruction-following, and tasks that require high adaptability.
### **2. Scientific & Chemistry Expertise**:
Thanks to the contribution from **KALE-LM-Chem**, this model shines in **scientific research**, particularly **chemistry-focused tasks**, making it ideal for academic and research purposes.
### **3. Long-Form & Conversational Mastery**:
With **Einstein-v6.1**, the model handles **long-form generation** effortlessly, excelling in extended conversations and structured dialogue applications.
---
## 🚀 **Performance Benchmarks**
While still in its early stages, **Llama3.1-BestMix-Chem-Einstein-8B** is expected to perform well across a variety of benchmarks, including:
- **Chemistry-focused benchmarks** (KALE-LM-Chem)
- **Instruction-following tasks** (Best-Mix)
- **Conversational AI** and **long-form text generation** (Einstein-v6.1)
Further testing and evaluation will continue to refine this model's capabilities.
---
## 📜 **License**
This model is open-sourced under the **Apache-2.0 License**, allowing free use and modification with proper attribution.
---
## 💡 **Tags**
- `merge`
- `TIES`
- `BestMix`
- `Chemistry`
- `Einstein`
- `instruction-following`
- `long-form-generation`
- `conversational`
---
|