Update README.md
Browse files
README.md
CHANGED
@@ -67,7 +67,7 @@ You should:
|
|
67 |
|
68 |
|
69 |
### Message length control
|
70 |
-
Inspired by the previously named "Roleplay" preset in SillyTavern,
|
71 |
version of LimaRP it is possible to append a length modifier to the response instruction
|
72 |
sequence, like this:
|
73 |
|
@@ -111,13 +111,13 @@ training process closer to a full finetune. It's suggested to merge the adapter
|
|
111 |
the base Mistral-7B-v0.1 model.
|
112 |
|
113 |
### Training hyperparameters
|
114 |
-
- learning_rate: 0.
|
115 |
- lr_scheduler_type: cosine
|
|
|
116 |
- lora_r: 256
|
117 |
- lora_alpha: 16
|
118 |
- lora_dropout: 0.05
|
119 |
- lora_target_linear: True
|
120 |
-
- num_epochs: 2
|
121 |
- bf16: True
|
122 |
- tf32: True
|
123 |
- load_in_8bit: True
|
|
|
67 |
|
68 |
|
69 |
### Message length control
|
70 |
+
Inspired by the previously named "Roleplay" preset in SillyTavern, with this
|
71 |
version of LimaRP it is possible to append a length modifier to the response instruction
|
72 |
sequence, like this:
|
73 |
|
|
|
111 |
the base Mistral-7B-v0.1 model.
|
112 |
|
113 |
### Training hyperparameters
|
114 |
+
- learning_rate: 0.001
|
115 |
- lr_scheduler_type: cosine
|
116 |
+
- num_epochs: 2
|
117 |
- lora_r: 256
|
118 |
- lora_alpha: 16
|
119 |
- lora_dropout: 0.05
|
120 |
- lora_target_linear: True
|
|
|
121 |
- bf16: True
|
122 |
- tf32: True
|
123 |
- load_in_8bit: True
|