sri-lasya/gst-taxing-llm

README.md

@@ -1,9 +1,9 @@
 ---
-license: apache-2.0
+base_model: mistralai/Mistral-7B-v0.3
 library_name: peft
+license: apache-2.0
 tags:
 - generated_from_trainer
-base_model: mistralai/Mistral-7B-v0.3
 model-index:
 - name: mistral_fine_tuned
   results: []
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6744
+- Loss: 1.7497
 
 ## Model description
 
@@ -47,18 +47,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 2.7937        | 0.0926 | 10   | 2.8448          |
-| 2.409         | 0.1852 | 20   | 2.3765          |
-| 1.9925        | 0.2778 | 30   | 2.0637          |
-| 1.8024        | 0.3704 | 40   | 1.9683          |
-| 1.7356        | 0.4630 | 50   | 1.8912          |
-| 1.7182        | 0.5556 | 60   | 1.8409          |
-| 1.7079        | 0.6481 | 70   | 1.8011          |
-| 1.699         | 0.7407 | 80   | 1.7660          |
-| 1.5545        | 0.8333 | 90   | 1.7415          |
-| 1.5352        | 0.9259 | 100  | 1.6744          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.3433        | 0.1   | 10   | 2.8619          |
+| 2.3427        | 0.2   | 20   | 2.3762          |
+| 1.9334        | 0.3   | 30   | 2.0743          |
+| 1.796         | 0.4   | 40   | 1.9626          |
+| 1.8265        | 0.5   | 50   | 1.9076          |
+| 1.7717        | 0.6   | 60   | 1.9241          |
+| 1.597         | 0.7   | 70   | 1.8398          |
+| 1.5922        | 0.8   | 80   | 1.8226          |
+| 1.5468        | 0.9   | 90   | 1.7813          |
+| 1.695         | 1.0   | 100  | 1.7497          |
 
 
 ### Framework versions

adapter_config.json

@@ -21,13 +21,13 @@
   "revision": null,
   "target_modules": [
     "q_proj",
-    "gate_proj",
     "up_proj",
+    "gate_proj",
     "o_proj",
-    "down_proj",
-    "lm_head",
     "v_proj",
-    "k_proj"
+    "k_proj",
+    "lm_head",
+    "down_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5988a3602f0996a10b938f780dc5c64b7eea69f67ecdae3fb79b951d1759f485
+oid sha256:4e8e05ea1613e86c8035471af8058b9d5dcc42a1b9031cf3669c75c9ee6cc601
 size 1217458040