uploaded readme

README.md

@@ -0,0 +1,185 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+karakuri-lm-7b-apm-v0.1 - GGUF
+- Model creator: https://huggingface.co/karakuri-ai/
+- Original model: https://huggingface.co/karakuri-ai/karakuri-lm-7b-apm-v0.1/
+
+
+| Name | Quant method | Size |
+| ---- | ---- | ---- |
+| [karakuri-lm-7b-apm-v0.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q2_K.gguf) | Q2_K | 3.24GB |
+| [karakuri-lm-7b-apm-v0.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.IQ3_XS.gguf) | IQ3_XS | 3.54GB |
+| [karakuri-lm-7b-apm-v0.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.IQ3_S.gguf) | IQ3_S | 3.71GB |
+| [karakuri-lm-7b-apm-v0.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q3_K_S.gguf) | Q3_K_S | 3.71GB |
+| [karakuri-lm-7b-apm-v0.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.IQ3_M.gguf) | IQ3_M | 3.82GB |
+| [karakuri-lm-7b-apm-v0.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q3_K.gguf) | Q3_K | 4.07GB |
+| [karakuri-lm-7b-apm-v0.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q3_K_M.gguf) | Q3_K_M | 4.07GB |
+| [karakuri-lm-7b-apm-v0.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q3_K_L.gguf) | Q3_K_L | 4.39GB |
+| [karakuri-lm-7b-apm-v0.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.IQ4_XS.gguf) | IQ4_XS | 4.48GB |
+| [karakuri-lm-7b-apm-v0.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q4_0.gguf) | Q4_0 | 4.67GB |
+| [karakuri-lm-7b-apm-v0.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.IQ4_NL.gguf) | IQ4_NL | 4.69GB |
+| [karakuri-lm-7b-apm-v0.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q4_K_S.gguf) | Q4_K_S | 4.7GB |
+| [karakuri-lm-7b-apm-v0.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q4_K.gguf) | Q4_K | 4.96GB |
+| [karakuri-lm-7b-apm-v0.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q4_K_M.gguf) | Q4_K_M | 4.96GB |
+| [karakuri-lm-7b-apm-v0.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q4_1.gguf) | Q4_1 | 5.12GB |
+| [karakuri-lm-7b-apm-v0.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q5_0.gguf) | Q5_0 | 5.57GB |
+| [karakuri-lm-7b-apm-v0.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q5_K_S.gguf) | Q5_K_S | 5.57GB |
+| [karakuri-lm-7b-apm-v0.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q5_K.gguf) | Q5_K | 5.72GB |
+| [karakuri-lm-7b-apm-v0.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q5_K_M.gguf) | Q5_K_M | 5.72GB |
+| [karakuri-lm-7b-apm-v0.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q5_1.gguf) | Q5_1 | 6.02GB |
+| [karakuri-lm-7b-apm-v0.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q6_K.gguf) | Q6_K | 6.53GB |
+| [karakuri-lm-7b-apm-v0.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/karakuri-ai_-_karakuri-lm-7b-apm-v0.1-gguf/blob/main/karakuri-lm-7b-apm-v0.1.Q8_0.gguf) | Q8_0 | 8.45GB |
+
+
+
+
+Original model description:
+---
+library_name: transformers
+license: gemma
+datasets:
+- OpenAssistant/oasst2
+- nvidia/HelpSteer
+language:
+- en
+- ja
+tags:
+- gemma
+- steerlm
+base_model: google/gemma-7b
+---
+
+# KARAKURI LM 7B APM v0.1
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** [KARAKURI Inc.](https://about.karakuri.ai/)
+- **Model type:** Causal decoder-only transformer language model
+- **Languages**: Primarily English
+- **License:** [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
+- **Finetuned from model:** [google/gemma-7b](https://huggingface.co/google/gemma-7b)
+- **Contact**: For questions and comments about the model, please email `[email protected]`
+
+## Usage
+
+KARAKURI LM 7B APM v0.1 is a attribute prediction model that rates model responses on various aspects that makes a response desirable.
+
+Given a conversation with multiple turns between user and assistant, the model rates the following attributes (between 0 and 4) for every assistant turn.
+
+- helpfulness: Overall helpfulness of the response to the prompt.
+- correctness: Inclusion of all pertinent facts without errors.
+- coherence: Consistency and clarity of expression.
+- complexity: Intellectual depth required to write response (i.e. whether the response can be written by anyone with basic language competency or requires deep domain expertise).
+- verbosity: Amount of detail included in the response, relative to what is asked for in the prompt.
+- quality: Perceived goodness of response.
+- toxicity: Undesirable elements such as vulgar, harmful or potentially biased response.
+- humor: Sense of humor within response.
+- creativity: Willingness to generate non-conventional response.
+
+The first five are derived from HelpSteer, while the remaining four are derived from OASST2.
+
+You can run the model using the 🤗 Transformers:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "karakuri-ai/karakuri-lm-7b-apm-v0.1"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype="auto",
+    device_map="auto",
+)
+
+messages = [
+    {"role": "user", "content": "Hello!"},
+    {"role": "assistant", "content": "Hello! How can I help you today?"},
+]
+tokenizer.apply_chat_template(
+    messages,
+    label="helpsteer",
+    tokenize=False,
+    add_generation_prompt=True,
+)
+# <bos>[INST] Hello! [/INST] Hello! How can I help you today? [ATTR_1]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    label="helpsteer",
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+outputs = model.generate(input_ids, max_new_tokens=32)
+tokenizer.decode(outputs[0][input_ids.shape[-1]:])
+#  helpfulness: 2 correctness: 1 coherence: 2 complexity: 1 verbosity: 1 [/ATTR_1]<eos>
+
+messages += [
+    {"role": "label", "content": "helpfulness: 2 correctness: 1 coherence: 2 complexity: 1 verbosity: 1"},
+    {"role": "user", "content": "Thank you!"},
+    {"role": "assistant", "content": "You're welcome! I'm happy to help however I can."},
+]
+tokenizer.apply_chat_template(
+    messages,
+    label="helpsteer",
+    tokenize=False,
+    add_generation_prompt=True,
+)
+# <bos>[INST] Hello! [/INST] Hello! How can I help you today? [ATTR_1] helpfulness: 2 correctness: 1 coherence: 2 complexity: 1 verbosity: 1 [/ATTR_1]<eos>[INST] Thank you! [/INST] You're welcome! I'm happy to help however I can. [ATTR_1]
+
+messages = [
+    {"role": "user", "content": "Hello!"},
+    {"role": "assistant", "content": "Hello! How can I help you today?"},
+]
+tokenizer.apply_chat_template(
+    messages,
+    label="oasst",
+    tokenize=False,
+    add_generation_prompt=True,
+)
+# <bos>[INST] Hello! [/INST] Hello! How can I help you today? [ATTR_2]
+
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    label="oasst",
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+outputs = model.generate(input_ids, max_new_tokens=32)
+tokenizer.decode(outputs[0][input_ids.shape[-1]:])
+#  quality: 3 toxicity: 1 humor: 1 creativity: 1 [/ATTR_2]<eos>
+```
+
+## Training Details
+
+### Training Data
+
+- [OASST2](https://huggingface.co/datasets/OpenAssistant/oasst2)
+- [HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)
+
+### Training Infrastructure
+
+- **Hardware**: The model was trained on single node of an Amazon EC2 trn1.32xlarge instance.
+- **Software**: We use code based on [neuronx-nemo-megatron](https://github.com/aws-neuron/neuronx-nemo-megatron).
+
+## Citation
+
+```
+@misc{karakuri_lm_7b_apm_v01,
+	author       = { {KARAKURI} {I}nc. },
+	title        = { {KARAKURI} {LM} 7{B} {APM} v0.1 },
+	year         = { 2024 },
+	url          = { https://huggingface.co/karakuri-ai/karakuri-lm-7b-apm-v0.1 },
+	publisher    = { Hugging Face },
+    journal      = { Hugging Face repository }
+}
+```
+