File size: 3,929 Bytes
a342fc7 fe2f907 a272980 a342fc7 a272980 a342fc7 a272980 a342fc7 77102a7 a342fc7 a272980 a342fc7 a272980 a342fc7 6f0937d a342fc7 4a05214 a342fc7 a272980 a342fc7 a272980 a342fc7 a272980 a342fc7 a272980 a342fc7 1e9daa8 a272980 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
license: other
language:
- en
pipeline_tag: text-generation
---
# llama-3-neural-chat-v2.2-8b
<!-- Provide a quick summary of what the model is/does. -->
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
I fine-tuned llama-3 8B on an approach similar to Intel's neural chat language model. I have slightly modified the data sources so it is stronger in coding, math, and writing. I use both SFT and DPO-Positive.
DPO-Positive dramatically improves performance over DPO.
- **Developed by:** Locutusque
- **Model type:** Built with Meta Llama 3
- **Language(s) (NLP):** Many?
- **License:** Llama 3 license https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE
## Quants
GGUF: https://huggingface.co/bartowski/llama-3-neural-chat-v2.2-8B-GGUF
ExLlamaV2: https://huggingface.co/bartowski/llama-3-neural-chat-v2.2-8B-exl2
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
This model has great performance in writing, coding, and math.
## Training Data
Recipe information will be coming soon. This language model's recipe is similar to Intel's Neural Chat.
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
Conversational AI. This model is also very uncensored, it will respond to pretty much any request regardless of the system prompt, use at your own risk.
## Evaluations
| Tasks |Version| Filter |n-shot| Metric |Value | |Stderr|
|---------------------------------|-------|----------------|-----:|-----------|-----:|---|-----:|
|truthfulqa_mc2 | 2|none | 0|acc |0.5232|± |0.0151|
|gsm8k | 3|strict-match | 5|exact_match|0.5974|± |0.0135|
| | |flexible-extract| 5|exact_match|0.5974|± |0.0135|
|agieval_nous |N/A |none | 0|acc_norm |0.3841|± |0.0094|
| | |none | 0|acc |0.3802|± |0.0094|
| - agieval_aqua_rat | 1|none | 0|acc |0.2598|± |0.0276|
| | |none | 0|acc_norm |0.2520|± |0.0273|
| - agieval_logiqa_en | 1|none | 0|acc |0.3441|± |0.0186|
| | |none | 0|acc_norm |0.3687|± |0.0189|
| - agieval_lsat_ar | 1|none | 0|acc |0.2217|± |0.0275|
| | |none | 0|acc_norm |0.2348|± |0.0280|
| - agieval_lsat_lr | 1|none | 0|acc |0.3882|± |0.0216|
| | |none | 0|acc_norm |0.3824|± |0.0215|
| - agieval_lsat_rc | 1|none | 0|acc |0.4944|± |0.0305|
| | |none | 0|acc_norm |0.5019|± |0.0305|
| - agieval_sat_en | 1|none | 0|acc |0.6650|± |0.0330|
| | |none | 0|acc_norm |0.6553|± |0.0332|
| - agieval_sat_en_without_passage| 1|none | 0|acc |0.3981|± |0.0342|
| | |none | 0|acc_norm |0.3981|± |0.0342|
| - agieval_sat_math | 1|none | 0|acc |0.3500|± |0.0322|
| | |none | 0|acc_norm |0.3318|± |0.0318| |