Hugging Face's logo

LeeSB
/
zephyr-7b-dpo-qlora

PEFT
TensorBoard
Safetensors
trl
dpo
Generated from Trainer
Model card Files Metrics Community
LeeSB commited on
Commit
da177e2
•
1 Parent(s): cd4df3a

Model save

Browse files
Files changed (6) hide show
  1. README.md +46 -17
  2. adapter_model.safetensors +1 -1
  3. all_results.json +5 -5
  4. runs/Apr07_04-54-57_allennlp-cirrascale-68.reviz.ai2.in/events.out.tfevents.1712490918.allennlp-cirrascale-68.reviz.ai2.in.101444.0 +2 -2
  5. train_results.json +5 -5
  6. trainer_state.json +0 -0
README.md CHANGED
@@ -18,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 1.0143
22
- - Rewards/chosen: -3.1357
23
- - Rewards/rejected: -3.9504
24
- - Rewards/accuracies: 0.5615
25
- - Rewards/margins: 0.8147
26
- - Logps/rejected: -657.2274
27
- - Logps/chosen: -598.0983
28
- - Logits/rejected: -1.6249
29
- - Logits/chosen: -1.7172
30
 
31
  ## Model description
32
 
@@ -62,14 +62,43 @@ The following hyperparameters were used during training:
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
64
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
65
- | 0.4083 | 0.11 | 100 | 0.6636 | -0.0687 | -0.1539 | 0.5595 | 0.0852 | -277.5772 | -291.3970 | -2.6015 | -2.7100 |
66
- | 0.0834 | 0.23 | 200 | 0.7288 | -1.4875 | -1.8701 | 0.5536 | 0.3825 | -449.1904 | -433.2774 | -2.0544 | -2.1544 |
67
- | 0.047 | 0.34 | 300 | 0.9801 | -2.8622 | -3.5385 | 0.5774 | 0.6763 | -616.0311 | -570.7468 | -1.6400 | -1.7298 |
68
- | 0.0411 | 0.46 | 400 | 0.9389 | -2.7119 | -3.4267 | 0.5536 | 0.7148 | -604.8529 | -555.7119 | -1.6178 | -1.7086 |
69
- | 0.0541 | 0.57 | 500 | 1.0554 | -3.1586 | -3.9685 | 0.5575 | 0.8099 | -659.0338 | -600.3828 | -1.6283 | -1.7211 |
70
- | 0.0315 | 0.68 | 600 | 1.0172 | -3.1217 | -3.9425 | 0.5615 | 0.8208 | -656.4329 | -596.6931 | -1.6210 | -1.7132 |
71
- | 0.0209 | 0.8 | 700 | 1.0112 | -3.1270 | -3.9417 | 0.5615 | 0.8147 | -656.3586 | -597.2280 | -1.6241 | -1.7165 |
72
- | 0.0141 | 0.91 | 800 | 1.0143 | -3.1357 | -3.9504 | 0.5615 | 0.8147 | -657.2274 | -598.0983 | -1.6249 | -1.7172 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
 
74
 
75
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 1.0291
22
+ - Rewards/chosen: -5.3085
23
+ - Rewards/rejected: -6.5312
24
+ - Rewards/accuracies: 0.6230
25
+ - Rewards/margins: 1.2227
26
+ - Logps/rejected: -915.3035
27
+ - Logps/chosen: -815.3711
28
+ - Logits/rejected: -1.8910
29
+ - Logits/chosen: -1.9908
30
 
31
  ## Model description
32
 
 
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
64
  |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
65
+ | 0.6668 | 0.03 | 100 | 0.6902 | -0.0428 | -0.0508 | 0.5813 | 0.0079 | -267.2608 | -288.8097 | -2.5925 | -2.6974 |
66
+ | 0.59 | 0.05 | 200 | 0.6712 | -0.1880 | -0.2474 | 0.6052 | 0.0594 | -286.9239 | -303.3236 | -2.6140 | -2.7228 |
67
+ | 0.4648 | 0.08 | 300 | 0.6608 | -0.4719 | -0.6250 | 0.5813 | 0.1531 | -324.6856 | -331.7153 | -2.6547 | -2.7661 |
68
+ | 0.4117 | 0.11 | 400 | 0.6790 | -1.3627 | -1.6607 | 0.6091 | 0.2979 | -428.2497 | -420.7988 | -2.5743 | -2.6840 |
69
+ | 0.3421 | 0.13 | 500 | 0.7134 | -2.3039 | -2.7725 | 0.5893 | 0.4686 | -539.4315 | -514.9171 | -2.2836 | -2.3908 |
70
+ | 0.2386 | 0.16 | 600 | 0.7002 | -2.3266 | -2.9065 | 0.5933 | 0.5798 | -552.8307 | -517.1895 | -2.0689 | -2.1718 |
71
+ | 0.2192 | 0.19 | 700 | 0.7335 | -2.7900 | -3.4049 | 0.6012 | 0.6149 | -602.6772 | -563.5288 | -1.9271 | -2.0255 |
72
+ | 0.1955 | 0.21 | 800 | 0.9052 | -4.8982 | -5.7092 | 0.5813 | 0.8109 | -833.1005 | -774.3487 | -1.7502 | -1.8455 |
73
+ | 0.2622 | 0.24 | 900 | 0.8059 | -3.9311 | -4.7534 | 0.5972 | 0.8223 | -737.5221 | -677.6362 | -1.9382 | -2.0360 |
74
+ | 0.2442 | 0.27 | 1000 | 0.7823 | -3.6824 | -4.4220 | 0.6052 | 0.7395 | -704.3822 | -652.7700 | -1.8754 | -1.9712 |
75
+ | 0.1973 | 0.29 | 1100 | 0.7637 | -3.3697 | -4.0317 | 0.6052 | 0.6620 | -665.3540 | -621.4905 | -1.9451 | -2.0419 |
76
+ | 0.2271 | 0.32 | 1200 | 0.8544 | -4.2788 | -5.1969 | 0.6389 | 0.9181 | -781.8729 | -712.4053 | -1.8583 | -1.9520 |
77
+ | 0.1646 | 0.35 | 1300 | 0.8492 | -3.9728 | -4.7346 | 0.6071 | 0.7618 | -735.6389 | -681.8040 | -2.0860 | -2.1880 |
78
+ | 0.2114 | 0.37 | 1400 | 0.8818 | -4.4350 | -5.3852 | 0.5913 | 0.9502 | -800.7042 | -728.0272 | -2.0593 | -2.1657 |
79
+ | 0.238 | 0.4 | 1500 | 1.1579 | -6.8403 | -8.1286 | 0.5853 | 1.2882 | -1075.0389 | -968.5556 | -1.8808 | -1.9824 |
80
+ | 0.1797 | 0.43 | 1600 | 0.8148 | -3.8153 | -4.7403 | 0.6210 | 0.9250 | -736.2164 | -666.0583 | -2.0400 | -2.1449 |
81
+ | 0.2382 | 0.45 | 1700 | 0.7823 | -3.3666 | -4.2125 | 0.6230 | 0.8459 | -683.4350 | -621.1896 | -2.0495 | -2.1546 |
82
+ | 0.2209 | 0.48 | 1800 | 0.8335 | -3.9836 | -4.9158 | 0.625 | 0.9322 | -753.7604 | -682.8817 | -2.0042 | -2.1072 |
83
+ | 0.172 | 0.51 | 1900 | 1.0104 | -5.2137 | -6.3618 | 0.6032 | 1.1481 | -898.3617 | -805.8920 | -1.9256 | -2.0274 |
84
+ | 0.2414 | 0.53 | 2000 | 0.9241 | -4.5365 | -5.6203 | 0.6151 | 1.0837 | -824.2111 | -738.1791 | -1.9465 | -2.0491 |
85
+ | 0.0881 | 0.56 | 2100 | 1.1385 | -6.7093 | -8.0226 | 0.6071 | 1.3133 | -1064.4432 | -955.4534 | -1.8325 | -1.9293 |
86
+ | 0.1257 | 0.59 | 2200 | 0.9302 | -4.7731 | -5.8420 | 0.6111 | 1.0689 | -846.3864 | -761.8352 | -1.9405 | -2.0411 |
87
+ | 0.1817 | 0.61 | 2300 | 1.0704 | -5.7508 | -7.0037 | 0.6131 | 1.2528 | -962.5499 | -859.6102 | -1.8941 | -1.9935 |
88
+ | 0.2228 | 0.64 | 2400 | 0.9503 | -4.8180 | -5.9316 | 0.625 | 1.1136 | -855.3458 | -766.3225 | -1.9147 | -2.0147 |
89
+ | 0.1957 | 0.67 | 2500 | 1.0221 | -5.2821 | -6.4729 | 0.6171 | 1.1908 | -909.4735 | -812.7320 | -1.8818 | -1.9806 |
90
+ | 0.1727 | 0.69 | 2600 | 1.0749 | -5.6941 | -6.9580 | 0.6111 | 1.2639 | -957.9836 | -853.9392 | -1.8693 | -1.9682 |
91
+ | 0.1364 | 0.72 | 2700 | 0.9818 | -4.8831 | -6.0107 | 0.6171 | 1.1276 | -863.2535 | -772.8399 | -1.9194 | -2.0195 |
92
+ | 0.1612 | 0.75 | 2800 | 1.0124 | -5.1144 | -6.3074 | 0.6210 | 1.1930 | -892.9272 | -795.9686 | -1.8991 | -1.9993 |
93
+ | 0.1884 | 0.77 | 2900 | 0.9946 | -5.0165 | -6.1897 | 0.6290 | 1.1732 | -881.1566 | -786.1763 | -1.9030 | -2.0035 |
94
+ | 0.0901 | 0.8 | 3000 | 1.0242 | -5.2594 | -6.4648 | 0.6210 | 1.2054 | -908.6607 | -810.4657 | -1.8954 | -1.9953 |
95
+ | 0.2163 | 0.83 | 3100 | 1.0258 | -5.2740 | -6.4825 | 0.6230 | 1.2085 | -910.4310 | -811.9224 | -1.8946 | -1.9943 |
96
+ | 0.1651 | 0.85 | 3200 | 1.0311 | -5.3241 | -6.5460 | 0.6230 | 1.2220 | -916.7872 | -816.9316 | -1.8921 | -1.9918 |
97
+ | 0.2249 | 0.88 | 3300 | 1.0313 | -5.3299 | -6.5578 | 0.625 | 1.2279 | -917.9644 | -817.5153 | -1.8902 | -1.9900 |
98
+ | 0.1427 | 0.91 | 3400 | 1.0295 | -5.3145 | -6.5390 | 0.6270 | 1.2245 | -916.0821 | -815.9720 | -1.8897 | -1.9896 |
99
+ | 0.2439 | 0.94 | 3500 | 1.0293 | -5.3098 | -6.5323 | 0.625 | 1.2225 | -915.4187 | -815.5079 | -1.8894 | -1.9892 |
100
+ | 0.1822 | 0.96 | 3600 | 1.0292 | -5.3079 | -6.5305 | 0.625 | 1.2226 | -915.2339 | -815.3108 | -1.8886 | -1.9885 |
101
+ | 0.1565 | 0.99 | 3700 | 1.0291 | -5.3085 | -6.5312 | 0.6230 | 1.2227 | -915.3035 | -815.3711 | -1.8910 | -1.9908 |
102
 
103
 
104
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:afa84cf19713c0a9893c594ccf497e698ddf5b527bc68bb5f7c6ed4fe99a861e
3
  size 671150064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce03c51a64336d81ee92573277b153429a0c239dab8d5725e8ea214ad3159933
3
  size 671150064
all_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 1.0,
3
- "train_loss": 0.12645735846515935,
4
- "train_runtime": 4707.0543,
5
- "train_samples": 14031,
6
- "train_samples_per_second": 2.981,
7
- "train_steps_per_second": 0.186
8
  }
 
1
  {
2
  "epoch": 1.0,
3
+ "train_loss": 0.23124931932427237,
4
+ "train_runtime": 23414.3072,
5
+ "train_samples": 59881,
6
+ "train_samples_per_second": 2.557,
7
+ "train_steps_per_second": 0.16
8
  }
runs/Apr07_04-54-57_allennlp-cirrascale-68.reviz.ai2.in/events.out.tfevents.1712490918.allennlp-cirrascale-68.reviz.ai2.in.101444.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:89cf45ae3f5faa4a2b1eb827f60576b70873d9fc1869fca9b293f66d5bbfded6
3
- size 287759
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74f6df21fe3f156205fb2e6e3fc68a95ca35da490d65caa80416201d7f4b0fae
3
+ size 290865
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 1.0,
3
- "train_loss": 0.12645735846515935,
4
- "train_runtime": 4707.0543,
5
- "train_samples": 14031,
6
- "train_samples_per_second": 2.981,
7
- "train_steps_per_second": 0.186
8
  }
 
1
  {
2
  "epoch": 1.0,
3
+ "train_loss": 0.23124931932427237,
4
+ "train_runtime": 23414.3072,
5
+ "train_samples": 59881,
6
+ "train_samples_per_second": 2.557,
7
+ "train_steps_per_second": 0.16
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff