Weni
/

WeniGPT-Agents-Mistral-1.0.11-SFT-1.0.27-DPO

+---
+library_name: peft
+tags:
+- trl
+- dpo
+- DPO
+- WeniGPT
+- generated_from_trainer
+base_model: Weni/WeniGPT-Agents-Mistral-1.0.11-SFT-merged
+model-index:
+- name: WeniGPT-Agents-Mistral-1.0.11-SFT-1.0.27-DPO
+  results: []
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# WeniGPT-Agents-Mistral-1.0.11-SFT-1.0.27-DPO
+This model is a fine-tuned version of [Weni/WeniGPT-Agents-Mistral-1.0.11-SFT-merged](https://huggingface.co/Weni/WeniGPT-Agents-Mistral-1.0.11-SFT-merged) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0855
+- Rewards/chosen: 1.8718
+- Rewards/rejected: -1.2443
+- Rewards/accuracies: 1.0
+- Rewards/margins: 3.1161
+- Logps/rejected: -191.7649
+- Logps/chosen: -127.9738
+- Logits/rejected: -1.8982
+- Logits/chosen: -1.8696
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 8
+- total_eval_batch_size: 4
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.03
+- training_steps: 180
+- mixed_precision_training: Native AMP
+### Training results
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.5138        | 0.9677 | 30   | 0.4536          | 0.5540         | -0.0631          | 1.0                | 0.6171          | -179.9529      | -141.1520    | -1.8497         | -1.8200       |
+| 0.2915        | 1.9355 | 60   | 0.2489          | 1.1740         | -0.2675          | 1.0                | 1.4414          | -181.9961      | -134.9520    | -1.8636         | -1.8333       |
+| 0.1995        | 2.9032 | 90   | 0.1573          | 1.5574         | -0.6071          | 1.0                | 2.1646          | -185.3929      | -131.1177    | -1.8785         | -1.8481       |
+| 0.1172        | 3.8710 | 120  | 0.1100          | 1.7744         | -0.9240          | 1.0                | 2.6984          | -188.5615      | -128.9482    | -1.8899         | -1.8606       |
+| 0.1162        | 4.8387 | 150  | 0.0910          | 1.8558         | -1.1622          | 1.0                | 3.0180          | -190.9434      | -128.1337    | -1.8967         | -1.8678       |
+| 0.1057        | 5.8065 | 180  | 0.0855          | 1.8718         | -1.2443          | 1.0                | 3.1161          | -191.7649      | -127.9738    | -1.8982         | -1.8696       |
+### Framework versions
+- PEFT 0.10.0
+- Transformers 4.40.0
+- Pytorch 2.1.0+cu118
+- Datasets 2.18.0
+- Tokenizers 0.19.1

adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:52d9679836b3bfb295cae124660ffeb35e167e00b3706a892b629f05c613d484
 size 335604696

 version https://git-lfs.github.com/spec/v1
+oid sha256:ae6d47556149f425a7ebf11a55dcb2c1f6bdfa37f83da1aeab752dbaa5f4fdfb
 size 335604696