Update README.md

README.md

@@ -7,7 +7,7 @@ tags: []
 
 This repository contains the results of our merged rag relevance PEFT adapter model.
 
-### Classification Performance
+### RAG Relevance Classification Metrics
 
 Our merged model achieves the following performance on a binary classification task:
 
@@ -22,6 +22,45 @@ Our merged model achieves the following performance on a binary classification t
 weighted avg       0.75      0.75      0.75       200
 ```
 
+### Model Usage
+For best results, we recommend starting with the following prompting strategy (and encourage tweaks as you see fit):
+
+```python
+def format_input_classification(query, text):
+    input = f"""
+      You are comparing a reference text to a question and trying to determine if the reference text
+  contains information relevant to answering the question. Here is the data:
+      [BEGIN DATA]
+      ************
+      [Question]: {query}
+      ************
+      [Reference text]: {text}
+      ************
+      [END DATA]
+  Compare the Question above to the Reference text. You must determine whether the Reference text
+  contains information that can answer the Question. Please focus on whether the very specific
+  question can be answered by the information in the Reference text.
+  Your response must be single word, either "relevant" or "unrelated",
+  and should not contain any text or characters aside from that word.
+  "unrelated" means that the reference text does not contain an answer to the Question.
+  "relevant" means the reference text contains an answer to the Question."""
+    return input
+
+
+text = format_input_classification("What is quanitzation?",
+  "Quantization is a method to reduce the memory footprint")
+messages = [
+    {"role": "user", "content": text}
+]
+
+pipe = pipeline(
+    "text-generation",
+    model=base_model,
+    model_kwargs={"attn_implementation": attn_implementation, "torch_dtype": torch.float16},
+    tokenizer=tokenizer,
+)
+```
+
 ### Comparison with Other Models
 
 We compared our merged model's performance on the RAG Eval benchmark against several other state-of-the-art language models: