File size: 2,044 Bytes
882e6de 9f1c00a 882e6de 8f441f8 332484e ce12c2b 3ace9c2 ce12c2b 58b2534 882e6de 62e0ed0 276ffe5 e3dc942 7bc24b1 50c0bab bd257b5 a630151 ce12c2b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
---
base_model:
- appvoid/palmer-002-32k
- raidhon/coven_tiny_1.1b_32k_orpo_alpha
- appvoid/palmer-003
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
---
<style>
@import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap');
</style>
<div style="background-color: #101010; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;">
#### Introducing palmer-004
We are thrilled to unveil palmer-004, the fourth iteration in our esteemed palmer series, designed to push the boundaries of performance and context window capacity. Merging the best of palmer-003 with the expansive capabilities of palmer-002-32k, palmer-004 is a testament to the power of merging similar trained models, think of it as palmer-003 with 32k context with minimal performance degradation.
| Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande |
|---------------|-------|-------|-----------|--------|------------|
| tinyllama-3t | 0.2577| 0.3029| 0.5935 | 0.7329 | 0.5959 |
| palmer-003 | 0.2523| 0.3439| 0.6208 | 0.7524 | 0.6590 |
| palmer-004 | 0.2601| 0.3456| 0.6138 | 0.7443 | 0.6511 |
Crafted using the TIES merge method, palmer-004 elevates its predecessor's prowess to a remarkable 32k context window without needing to train on additional data. This model doesn't just aim to match; it surpasses, outperforming the current state-of-the-art coven_tiny_1.1b_32k_orpo_alpha on several benchmarks. As of June 1, 2024, palmer-004 stands as the 2nd best 1b overall model as well as the best 32k 1B model to fine-tune from on Hugging Face, setting a new standard for excellence in the open-source AI community.
If you are using Windows, you can now support me by purchasing a copy of [quark, a cool text editor](https://appvoid.itch.io/quark) that uses language models. Still in beta but super cool still.
</div> |