Hugging Face's logo

LaoRay
/
zephyr-7b-dpo-lora-r16-20k

PEFT
Safetensors
mistral
alignment-handbook
trl
dpo
Generated from Trainer
Model card Files Community
LaoRay commited on
Commit
8c58f70
•
1 Parent(s): a9ef506

Model save

Browse files
Files changed (4) hide show
  1. README.md +24 -25
  2. all_results.json +4 -4
  3. train_results.json +4 -4
  4. trainer_state.json +0 -0
README.md CHANGED
@@ -5,7 +5,6 @@ license: apache-2.0
5
  tags:
6
  - trl
7
  - dpo
8
- - alignment-handbook
9
  - generated_from_trainer
10
  model-index:
11
  - name: zephyr-7b-dpo-lora-r16-20k
@@ -19,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
19
 
20
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
- - Logits/chosen: -2.5568
23
- - Logits/rejected: -2.5135
24
- - Logps/chosen: -362.1219
25
- - Logps/rejected: -395.5133
26
- - Loss: 0.5370
27
- - Rewards/accuracies: 0.7063
28
- - Rewards/chosen: -0.7888
29
- - Rewards/margins: 0.6860
30
- - Rewards/rejected: -1.4748
31
 
32
  ## Model description
33
 
@@ -60,26 +59,26 @@ The following hyperparameters were used during training:
60
 
61
  ### Training results
62
 
63
- | Training Loss | Epoch | Step | Logits/chosen | Logits/rejected | Logps/chosen | Logps/rejected | Validation Loss | Rewards/accuracies | Rewards/chosen | Rewards/margins | Rewards/rejected |
64
- |:-------------:|:-----:|:----:|:-------------:|:---------------:|:------------:|:--------------:|:---------------:|:------------------:|:--------------:|:---------------:|:----------------:|
65
- | 0.6895 | 0.08 | 100 | -2.8901 | -2.8481 | -282.2447 | -247.7537 | 0.6896 | 0.6627 | 0.0099 | 0.0072 | 0.0028 |
66
- | 0.653 | 0.16 | 200 | -2.8742 | -2.8339 | -284.5635 | -257.5692 | 0.6569 | 0.6865 | -0.0133 | 0.0821 | -0.0954 |
67
- | 0.6385 | 0.24 | 300 | -2.8399 | -2.8031 | -310.6566 | -295.5536 | 0.6190 | 0.6905 | -0.2742 | 0.2011 | -0.4752 |
68
- | 0.5689 | 0.32 | 400 | -2.8437 | -2.8083 | -312.9573 | -305.2159 | 0.6027 | 0.6944 | -0.2972 | 0.2747 | -0.5719 |
69
- | 0.5689 | 0.4 | 500 | -2.7560 | -2.7152 | -349.3812 | -355.0662 | 0.5750 | 0.7242 | -0.6614 | 0.4089 | -1.0704 |
70
- | 0.5884 | 0.48 | 600 | -2.6724 | -2.6322 | -352.8877 | -375.1053 | 0.5479 | 0.7123 | -0.6965 | 0.5743 | -1.2708 |
71
- | 0.5366 | 0.56 | 700 | -2.6541 | -2.6144 | -355.7809 | -381.5439 | 0.5462 | 0.7123 | -0.7254 | 0.6097 | -1.3351 |
72
- | 0.542 | 0.64 | 800 | -2.6163 | -2.5757 | -352.4363 | -374.8915 | 0.5451 | 0.7262 | -0.6920 | 0.5766 | -1.2686 |
73
- | 0.5282 | 0.72 | 900 | -2.5716 | -2.5266 | -362.9279 | -390.7825 | 0.5412 | 0.7083 | -0.7969 | 0.6306 | -1.4275 |
74
- | 0.5873 | 0.8 | 1000 | -2.5693 | -2.5254 | -365.5720 | -399.3072 | 0.5369 | 0.7083 | -0.8233 | 0.6894 | -1.5128 |
75
- | 0.5152 | 0.88 | 1100 | -2.5620 | -2.5188 | -357.7025 | -389.9855 | 0.5384 | 0.7143 | -0.7446 | 0.6749 | -1.4196 |
76
- | 0.5213 | 0.96 | 1200 | -2.5568 | -2.5135 | -362.1219 | -395.5133 | 0.5370 | 0.7063 | -0.7888 | 0.6860 | -1.4748 |
77
 
78
 
79
  ### Framework versions
80
 
81
  - PEFT 0.12.0
82
  - Transformers 4.44.0
83
- - Pytorch 2.1.2+cu118
84
  - Datasets 2.21.0
85
  - Tokenizers 0.19.1
 
5
  tags:
6
  - trl
7
  - dpo
 
8
  - generated_from_trainer
9
  model-index:
10
  - name: zephyr-7b-dpo-lora-r16-20k
 
18
 
19
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.5301
22
+ - Rewards/chosen: -0.7870
23
+ - Rewards/rejected: -1.4645
24
+ - Rewards/accuracies: 0.7202
25
+ - Rewards/margins: 0.6775
26
+ - Logps/rejected: -394.4780
27
+ - Logps/chosen: -361.9388
28
+ - Logits/rejected: -2.5045
29
+ - Logits/chosen: -2.5477
30
 
31
  ## Model description
32
 
 
59
 
60
  ### Training results
61
 
62
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
63
+ |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
64
+ | 0.6899 | 0.08 | 100 | 0.6897 | 0.0098 | 0.0028 | 0.6667 | 0.0070 | -247.7543 | -282.2605 | -2.8468 | -2.8890 |
65
+ | 0.6532 | 0.16 | 200 | 0.6569 | -0.0128 | -0.0950 | 0.6885 | 0.0822 | -257.5306 | -284.5143 | -2.8386 | -2.8782 |
66
+ | 0.6372 | 0.24 | 300 | 0.6181 | -0.2381 | -0.4406 | 0.6825 | 0.2026 | -292.0921 | -307.0444 | -2.8033 | -2.8402 |
67
+ | 0.5699 | 0.32 | 400 | 0.6034 | -0.2658 | -0.5383 | 0.6964 | 0.2725 | -301.8563 | -309.8138 | -2.7952 | -2.8319 |
68
+ | 0.5622 | 0.4 | 500 | 0.5688 | -0.5565 | -0.9794 | 0.7143 | 0.4229 | -345.9727 | -338.8872 | -2.6913 | -2.7320 |
69
+ | 0.5826 | 0.48 | 600 | 0.5457 | -0.5456 | -1.1188 | 0.7242 | 0.5732 | -359.9116 | -337.7992 | -2.6523 | -2.6907 |
70
+ | 0.5313 | 0.56 | 700 | 0.5387 | -0.7142 | -1.3304 | 0.7242 | 0.6162 | -381.0734 | -354.6571 | -2.6173 | -2.6586 |
71
+ | 0.5332 | 0.64 | 800 | 0.5386 | -0.7256 | -1.3351 | 0.7183 | 0.6096 | -381.5442 | -355.7965 | -2.5760 | -2.6167 |
72
+ | 0.5334 | 0.72 | 900 | 0.5368 | -0.7061 | -1.3229 | 0.7163 | 0.6168 | -380.3204 | -353.8529 | -2.5574 | -2.5999 |
73
+ | 0.5837 | 0.8 | 1000 | 0.5302 | -0.7953 | -1.4787 | 0.7163 | 0.6834 | -395.8991 | -362.7657 | -2.5273 | -2.5706 |
74
+ | 0.5144 | 0.88 | 1100 | 0.5327 | -0.7410 | -1.4021 | 0.7123 | 0.6611 | -388.2353 | -357.3381 | -2.5162 | -2.5586 |
75
+ | 0.5196 | 0.96 | 1200 | 0.5301 | -0.7870 | -1.4645 | 0.7202 | 0.6775 | -394.4780 | -361.9388 | -2.5045 | -2.5477 |
76
 
77
 
78
  ### Framework versions
79
 
80
  - PEFT 0.12.0
81
  - Transformers 4.44.0
82
+ - Pytorch 2.1.2+cu121
83
  - Datasets 2.21.0
84
  - Tokenizers 0.19.1
all_results.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
  "epoch": 1.0,
3
  "total_flos": 0.0,
4
- "train_loss": 0.0,
5
- "train_runtime": 0.0105,
6
  "train_samples": 20000,
7
- "train_samples_per_second": 1898476.441,
8
- "train_steps_per_second": 118654.778
9
  }
 
1
  {
2
  "epoch": 1.0,
3
  "total_flos": 0.0,
4
+ "train_loss": 0.5873338260650635,
5
+ "train_runtime": 15803.1996,
6
  "train_samples": 20000,
7
+ "train_samples_per_second": 1.266,
8
+ "train_steps_per_second": 0.079
9
  }
train_results.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
  "epoch": 1.0,
3
  "total_flos": 0.0,
4
- "train_loss": 0.0,
5
- "train_runtime": 0.0105,
6
  "train_samples": 20000,
7
- "train_samples_per_second": 1898476.441,
8
- "train_steps_per_second": 118654.778
9
  }
 
1
  {
2
  "epoch": 1.0,
3
  "total_flos": 0.0,
4
+ "train_loss": 0.5873338260650635,
5
+ "train_runtime": 15803.1996,
6
  "train_samples": 20000,
7
+ "train_samples_per_second": 1.266,
8
+ "train_steps_per_second": 0.079
9
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff