Tatvajsh/Lllama_AHS_V_7.1

README.md

@@ -32,7 +32,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 3e-08
+- learning_rate: 3e-05
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
@@ -40,9 +40,13 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3
+- num_epochs: 5
 - mixed_precision_training: Native AMP
 
+### Training results
+
+
+
 ### Framework versions
 
 - Transformers 4.35.0.dev0

adapter_config.json

@@ -16,14 +16,14 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "base_layer",
-    "o_proj",
-    "v_proj",
     "down_proj",
+    "k_proj",
+    "base_layer",
     "up_proj",
+    "v_proj",
+    "o_proj",
     "q_proj",
-    "gate_proj",
-    "k_proj"
+    "gate_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:eec8b67ef0b857ac1ed155b2d6843d73cd61103f8cc639e3b90b4f187c9b7709
+oid sha256:dda8c2e6bb1d91791cc260964646173c6fc6a20acbeaf8c3fa909afe3ee272b4
 size 101834237

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:23ace4e23d25fc36745f1d7b26a7138d38ef7dd735b9771cee3e7c00662d3078
+oid sha256:54e7ed8815ac546261cf4da1ea2575a5a0b5e32ae11af329a47aca94ee2f7637
 size 4091