add: llama for education

README.md

@@ -7,19 +7,19 @@ tags:
 - sft
 - generated_from_trainer
 model-index:
-- name: outputs
+- name: llama-edu
   results: []
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# outputs
+# llama-edu
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.8215
-- Model Preparation Time: 0.0166
+- Loss: 0.7934
+- Model Preparation Time: 0.0161
 
 ## Model description
 
@@ -54,20 +54,21 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss | Model Preparation Time |
 |:-------------:|:------:|:----:|:---------------:|:----------------------:|
-| 2.1179        | 1.1765 | 20   | 2.0786          | 0.0166                 |
-| 1.1724        | 2.3529 | 40   | 1.2198          | 0.0166                 |
-| 0.9532        | 3.5294 | 60   | 0.9806          | 0.0166                 |
-| 0.851         | 4.7059 | 80   | 0.8941          | 0.0166                 |
-| 0.8294        | 5.8824 | 100  | 0.8571          | 0.0166                 |
-| 0.7754        | 7.0588 | 120  | 0.8378          | 0.0166                 |
-| 0.748         | 8.2353 | 140  | 0.8258          | 0.0166                 |
-| 0.7357        | 9.4118 | 160  | 0.8224          | 0.0166                 |
+| 2.2418        | 1.0256 | 20   | 2.0564          | 0.0161                 |
+| 1.2511        | 2.0513 | 40   | 1.2006          | 0.0161                 |
+| 0.9906        | 3.0769 | 60   | 0.9552          | 0.0161                 |
+| 0.8046        | 4.1026 | 80   | 0.8716          | 0.0161                 |
+| 0.7893        | 5.1282 | 100  | 0.8376          | 0.0161                 |
+| 0.8012        | 6.1538 | 120  | 0.8167          | 0.0161                 |
+| 0.7734        | 7.1795 | 140  | 0.8038          | 0.0161                 |
+| 0.78          | 8.2051 | 160  | 0.7970          | 0.0161                 |
+| 0.7576        | 9.2308 | 180  | 0.7939          | 0.0161                 |
 
 
 ### Framework versions
 
 - PEFT 0.12.0
-- Transformers 4.44.0
+- Transformers 4.44.2
 - Pytorch 2.1.2
 - Datasets 2.21.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "o_proj",
     "up_proj",
     "down_proj",
-    "k_proj",
-    "o_proj",
     "v_proj",
     "q_proj",
-    "gate_proj"
+    "gate_proj",
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d0ea907eab6d40559ce9a423f2297c8ac366c66a13d69aa42eec515d31edb59b
+oid sha256:8a5a3d7ea99698316cfe9750f9d69a4c1e3d16da3156448d4a6dfabcc5076dfa
 size 167832240

runs/Aug29_11-54-18_98b9c91667d9/events.out.tfevents.1724932510.98b9c91667d9.24.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64777f7f55a30ce0665d9c53705dd7b7207ccf108a865151972e8a898dabcb21
+size 49164

runs/Aug29_11-54-18_98b9c91667d9/events.out.tfevents.1724937021.98b9c91667d9.24.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cdbe4aa13afbfe5de62c4bcae85aa61f4a410256329f182ba298c02e99eeb2e
+size 425

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e45c6297c96123d2ce89b146263169109927e540ce178b90e34c2267c12daaa9
+oid sha256:83d76460e4773cf68a61695f04a192cec9ea8ab643f1317351e1e190ce4d24ab
 size 5432