Add details

README.md

@@ -1,51 +1,124 @@
 ---
-license: other
-base_model: lewtun/zephyr-7b-gemma-sft
+license: mit
+base_model: HuggingFaceH4/zephyr-7b-gemma-sft
 tags:
 - alignment-handbook
 - trl
 - dpo
 - generated_from_trainer
-- trl
-- dpo
-- generated_from_trainer
 datasets:
 - argilla/dpo-mix-7k
+pipeline_tag: text-generation
 model-index:
-- name: zephyr-7b-gemma-dpo
-  results: []
+- name: zephyr-7b-gemma
+  results:
+  # MT-Bench (taken from model card)
+  - task: 
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MT-Bench
+      type: unknown
+    metrics:
+       - type: unknown
+         name: score
+         value: 7.81
+    source:
+      url: https://huggingface.co/spaces/lmsys/mt-bench
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+<img src="https://huggingface.co/HuggingFaceH4/zephyr-7b-gemma/resolve/main/thumbnail.png" alt="Zephyr 7B Gemma Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
-# zephyr-7b-gemma-dpo
+# Model Card for Zephyr 7B Gemma
 
-This model is a fine-tuned version of [lewtun/zephyr-7b-gemma-sft](https://huggingface.co/lewtun/zephyr-7b-gemma-sft) on the argilla/dpo-mix-7k dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.4695
-- Rewards/chosen: -3.3746
-- Rewards/rejected: -4.9715
-- Rewards/accuracies: 0.7188
-- Rewards/margins: 1.5970
-- Logps/rejected: -459.4853
-- Logps/chosen: -429.9115
-- Logits/rejected: 86.4684
-- Logits/chosen: 92.8200
+Zephyr is a series of language models that are trained to act as helpful assistants. Zephyr 7B Gemma is the third model in the series, and is a fine-tuned version of [`google/gemma-7b`](https://huggingface.co/google/gemma-7b) that was trained on on a mix of publicly available, synthetic datasets using Direct Preference Optimization (DPO). You can reproduce the training of this model via the recipe provided in the [Alignment Handbook](https://github.com/huggingface/alignment-handbook).
 
 ## Model description
 
-More information needed
+- **Model type:** A 7B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
+- **Language(s) (NLP):** Primarily English
+- **License:** MIT
+- **Finetuned from model:** [google/gemma-7b](https://huggingface.co/google/gemma-7b)
+
+### Model Sources
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** https://github.com/huggingface/alignment-handbook
+- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat
+
+## Performance
+
+At the time of release, Zephyr 7B Gemma is the highest ranked 7B chat model on the [MT-Bench](https://huggingface.co/spaces/lmsys/mt-bench) and [AlpacaEval](https://tatsu-lab.github.io/alpaca_eval/) benchmarks:
+
+
+
+In particular, on several categories of MT-Bench, Zephyr-7B-β has strong performance compared to larger open models like Llama2-Chat-70B:
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6200d0a443eb0913fa2df7cc/raxvt5ma16d7T23my34WC.png)
+
+However, on more complex tasks like coding and mathematics, Zephyr-7B-β lags behind proprietary models and more research is needed to close the gap.
 
 ## Intended uses & limitations
 
-More information needed
+The model was initially fine-tuned on a filtered and preprocessed of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. 
+We then further aligned the model with [🤗 TRL's](https://github.com/huggingface/trl) `DPOTrainer` on the [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset, which contains 64k prompts and model completions that are ranked by GPT-4. As a result, the model can be used for chat and you can check out our [demo](https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat) to test its capabilities. 
+
+You can find the datasets used for training Zephyr-7B-β [here](https://huggingface.co/collections/HuggingFaceH4/zephyr-7b-6538c6d6d5ddd1cbb1744a66)
+
+Here's how you can run the model using the `pipeline()` function from 🤗 Transformers:
+
+```python
+# Install transformers from source - only needed for versions <= v4.38.1
+# pip install git+https://github.com/huggingface/transformers.git
+# pip install accelerate
+
+import torch
+from transformers import pipeline
+
+pipe = pipeline("text-generation", model="HuggingFaceH4/zephyr-7b-gemma", torch_dtype=torch.bfloat16, device_map="auto")
+
+# We use ChatML to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
+messages = [
+    {
+        "role": "system",
+        "content": "You are a friendly chatbot who always responds in the style of a pirate",
+    },
+    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
+]
+prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
+# <|system|>
+# You are a friendly chatbot who always responds in the style of a pirate.</s>
+# <|user|>
+# How many helicopters can a human eat in one sitting?</s>
+# <|assistant|>
+# Ah, me hearty matey! But yer question be a puzzler! A human cannot eat a helicopter in one sitting, as helicopters are not edible. They be made of metal, plastic, and other materials, not food!
+```
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+Zephyr 7B Gemma has not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model (`google/gemma-7b`), however it is likely to have included a mix of Web data and technical sources like books and code. See the [StarCoder2 model card](https://huggingface.co/bigcode/starcoder2-15b) for an example of this.
+
 
 ## Training and evaluation data
 
-More information needed
 
-## Training procedure
+This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-gemma-sft](https://huggingface.co/HuggingFaceH4/zephyr-7b-gemma-sft) on the argilla/dpo-mix-7k dataset.
+
+It achieves the following results on the evaluation set:
+- Loss: 0.4695
+- Rewards/chosen: -3.3746
+- Rewards/rejected: -4.9715
+- Rewards/accuracies: 0.7188
+- Rewards/margins: 1.5970
+- Logps/rejected: -459.4853
+- Logps/chosen: -429.9115
+- Logits/rejected: 86.4684
+- Logits/chosen: 92.8200
 
 ### Training hyperparameters