Update README.md

README.md

@@ -1,18 +1,18 @@
----
-base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
-language:
-- en
-license: apache-2.0
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- llama
-- trl
-datasets:
-- TIGER-Lab/MathInstruct
-library_name: transformers
----
+---
+base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
+language:
+- en
+license: mit
+tags:
+- text-generation-inference
+- transformers
+- unsloth
+- llama
+- trl
+datasets:
+- TIGER-Lab/MathInstruct
+library_name: transformers
+---
 
 # Model Card: Math Homework Solver
 
@@ -44,7 +44,8 @@ To start using the Math Homework Solver model, follow these steps:
    ```
    
 2. **Run inference**
-   Make a new file named main.py and run this code in it:
+
+   1. This method is recommended as its reliabel and accurate:
    ```python
    from unsloth import FastLanguageModel
    import torch
@@ -54,7 +55,7 @@ To start using the Math Homework Solver model, follow these steps:
 
    # Load the model
    model, tokenizer = FastLanguageModel.from_pretrained(
-           model_name = "Math_Homework_Solver_Llama318B", # The dir where the repo is cloned or "\\" for root
+           model_name = "Math_Homework_Solver_Llama318B/model_adapters", # The dir where the repo is cloned or "\\" for root
            max_seq_length = 2048,
            dtype = None,
            load_in_4bit = True,
@@ -81,6 +82,45 @@ To start using the Math Homework Solver model, follow these steps:
    text_streamer = TextStreamer(tokenizer)
    _ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 512)
    ```
+   
+   2. Another way to run inference is to use the merged adapters (not recommend as it gives inaccurate/different answers):
+   ```python
+   from transformers import LlamaForCausalLM, AutoTokenizer
+
+   # Load the model
+   model = LlamaForCausalLM.from_pretrained(
+       "justsomerandomdude264/Science_Homework_Solver_Llama318B",
+       device_map="auto"
+   )
+   
+   # Load the tokenizer
+   tokenizer = AutoTokenizer.from_pretrained("justsomerandomdude264/Science_Homework_Solver_Llama318B")
+
+   # Set the inputs up
+   qa_template = """Question: {}
+   Answer: {}"""
+   
+   inputs = tokenizer(
+   [
+       qa_template.format(
+           "find the force of an object with 5kg of mass and 3.9ms2 acceleration", # instruction
+           "", # output - leave this blank for generation!
+       )
+   ], return_tensors = "pt").to("cuda")
+   
+   # Do a forward pass
+   outputs = model.generate(**inputs, max_new_tokens = 128, use_cache = True)
+   raw_output = str(tokenizer.batch_decode(outputs))
+   
+   # Formtting the string
+   # Removing the list brackets and splitting the string by newline characters
+   formatted_string = raw_output.strip("[]").replace("<|begin_of_text|>", "").replace("<|eot_id|>", "").strip("''").split("\\n")
+
+   # Print the lines one by one
+   for line in formatted_string:
+     print(line)
+  ```
+  
 
 ## Citation