|
--- |
|
base_model: euclaise/Memphis-CoT-3B |
|
datasets: |
|
- euclaise/TinyCoT |
|
- euclaise/reddit-instruct |
|
- sablo/oasst2_curated |
|
license: cc-by-sa-3.0 |
|
language: |
|
- en |
|
model_creator: euclaise |
|
model_name: Memphis-CoT-3B |
|
model_type: stablelm_epoch |
|
inference: false |
|
tags: |
|
- supertrainer2000 |
|
- human-data |
|
- stablelm_epoch |
|
pipeline_tag: text-generation |
|
prompt_template: | |
|
{{system_message}} |
|
### User: |
|
{{prompt}} |
|
### Assistant: |
|
|
|
quantized_by: brittlewis12 |
|
--- |
|
|
|
# Memphis-CoT-3B GGUF |
|
|
|
 |
|
|
|
Original model: [Memphis-CoT-3B](https://huggingface.co/euclaise/Memphis-CoT-3B) |
|
Model creator: [euclaise](https://huggingface.co/euclaise) |
|
|
|
This repo contains GGUF format model files for euclaise’s Memphis-CoT-3B, updated for the latest training run as of 2/2/24. |
|
|
|
> Memphis-CoT is a finetune of StableLM 3b 4e1t on TinyCoT, along with reddit-instruct (subset to 5000 examples, excluding posts with brackets in the title) and a curated subset of oasst2. |
|
|
|
|
|
### What is GGUF? |
|
|
|
GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. |
|
Converted using llama.cpp b2022 ([8f8ddfc](https://github.com/ggerganov/llama.cpp/commits/8f8ddfcfadc830b936318c3ea9fe2e8e3365aa85)) |
|
|
|
### Prompt template: |
|
|
|
``` |
|
{{system_message}} |
|
### User: |
|
{{prompt}} |
|
### Assistant: |
|
``` |
|
|
|
or Tiny CoT: |
|
``` |
|
### User: |
|
{{prompt}} |
|
### Rationale: |
|
[...] |
|
### Answer: |
|
``` |
|
|
|
--- |
|
|
|
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac! |
|
|
|
 |
|
|
|
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device: |
|
- create & save **Characters** with custom system prompts & temperature settings |
|
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)! |
|
- make it your own with custom **Theme colors** |
|
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming! |
|
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)! |
|
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date |
|
|
|
--- |
|
|
|
## Original Model Evaluations: |
|
|
|
| Model | Size | Data | Method | GSM8K (5-shot) | AGIEval (English/Nous subset, acc_norm) | BIG Bench Hard (CoT, few-shot*) | |
|
|:-----------------------------------------------------------------------|--------|:--------------------|---------------|:---------------|:----------------------------------------|:------------------------------ | |
|
| [StableLM 3B Base](https://hf.co/stabilityai/stablelm-3b-4e1t) | 3B | Base | Base | 2.05% | 25.14% | 36.75% | |
|
| [StableHermes 3B](https://hf.co/cxllin/StableHermes-3b) | 3B | GPT | SFT | 3.64% | 24.31% | *37.28%* | |
|
| [MPT 7B Instruct](https://hf.co/mosaicml/mpt-7b-instruct) | **7B** | **Human**+Anthropic | SFT | 2.05% | 24.12% | 11.01% | |
|
| [OpenLLaMA 7B v2 open-instruct](http://hf.co/VMware/open-llama-7b-v2-open-instruct) | **7B** | **Human** (nearly: ecqa is an exception) | SFT | 8.64% | 23.21% | 29.84% | |
|
| [StableLM Zephyr 3B](https://hf.co/stabilityai/stablelm-zephyr-3b) | 3B | GPT | DPO | possibly contaminated (45.72%) | **33.31%** | 0.91% | |
|
| [**Memphis-CoT 3B**](https://hf.co/euclaise/memphis-cot-3b) | 3B | **Human** | Self-teaching | **13.8%** | *26.24%* | **38.24%** | |
|
|
|
*5-shot, as performed automatically by LM Evaluation Harness bbh_cot_fewshot even with num_fewshot=0 |
|
|
|
> Memphis outperforms other primarily-human-data models that are over twice its size, along with SFT models of its size, and trades with the Zephyr DPO model. That said, Zephyr uses synthetic data, and *much* more of it. |
