Upload folder using huggingface_hub

README.md

@@ -1,99 +1,102 @@
 ---
 license: mit
-library_name: peft
+library_name: "trl"
 tags:
-- trl
-- kto
-- generated_from_trainer
+- KTO
+- WeniGPT
 base_model: HuggingFaceH4/zephyr-7b-beta
 model-index:
-- name: WeniGPT-Agents-Zephyr-1.0.25-KTO
+- name: Weni/WeniGPT-Agents-Zephyr-1.0.25-KTO
   results: []
+language: ['pt']
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+# Weni/WeniGPT-Agents-Zephyr-1.0.25-KTO
 
-# WeniGPT-Agents-Zephyr-1.0.25-KTO
+This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta] on the dataset Weni/wenigpt-agent-1.4.0 with the KTO trainer. It is part of the WeniGPT project for [Weni](https://weni.ai/).
+Description: Experiment with a new tokenizer configuration for chat template of zephyr
 
-This model is a fine-tuned version of [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5
-- Rewards/chosen: -195.8677
-- Rewards/rejected: -165.2624
-- Rewards/margins: -30.6053
-- Kl: 0.0
-- Logps/chosen: -2238.1643
-- Logps/rejected: -1890.7997
+{'eval_loss': 0.5, 'eval_runtime': 169.8846, 'eval_samples_per_second': 2.06, 'eval_steps_per_second': 0.518, 'eval_rewards/chosen': -195.86773681640625, 'eval_rewards/rejected': -165.26242065429688, 'eval_rewards/margins': -30.605329513549805, 'eval_kl': 0.0, 'eval_logps/chosen': -2238.164306640625, 'eval_logps/rejected': -1890.7996826171875, 'epoch': 9.7}
 
-## Model description
+## Intended uses & limitations
 
-More information needed
+This model has not been trained to avoid specific intructions. 
 
-## Intended uses & limitations
+## Training procedure
 
-More information needed
+Finetuning was done on the model HuggingFaceH4/zephyr-7b-beta with the following prompt:
 
-## Training and evaluation data
+```
+---------------------
+System_prompt:
+Agora você se chama {name}, você é {occupation} e seu objetivo é {chatbot_goal}. O adjetivo que mais define a sua personalidade é {adjective} e você se comporta da seguinte forma:
+{instructions_formatted}
 
-More information needed
+Na sua memória você tem esse contexto:
+{context}
+
+Lista de requisitos:
+ - Responda de forma natural, mas nunca fale sobre um assunto fora do contexto.
+ - Nunca traga informações do seu próprio conhecimento.
+ - Repito é crucial que você responda usando apenas informações do contexto.
+ - Nunca mencione o contexto fornecido.
+ - Nunca mencione a pergunta fornecida.
+ - Gere a resposta mais útil possível para a pergunta usando informações do conexto acima.
+ - Nunca elabore sobre o porque e como você fez a tarefa, apenas responda.
+
+
+---------------------
+Question:
+{question}
 
-## Training procedure
+
+---------------------
+Response:
+{answer}
+
+
+---------------------
+
+```
 
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
+- per_device_train_batch_size: 4
+- per_device_eval_batch_size: 4
 - gradient_accumulation_steps: 4
+- num_gpus: 1
 - total_train_batch_size: 16
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.03
-- training_steps: 1470
-- mixed_precision_training: Native AMP
+- optimizer: AdamW
+- lr_scheduler_type: cosine
+- num_steps: 1470
+- quantization_type: bitsandbytes
+- LoRA: ("\n  - bits: 4\n  - use_exllama: True\n  - device_map: auto\n  - use_cache: False\n  - lora_r: 8\n  - lora_alpha: 16\n  - lora_dropout: 0.05\n  - bias: none\n  - target_modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj', 'lm_head', 'embed_tokens']\n  - task_type: CAUSAL_LM",)
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/margins | Kl  | Logps/chosen | Logps/rejected |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:---------------:|:---:|:------------:|:--------------:|
-| 0.6966        | 0.33  | 50   | 0.5063          | -13.4129       | -12.4081         | -1.0049         | 0.0 | -413.6161    | -362.2560      |
-| 0.754         | 0.66  | 100  | 0.5000          | -174.7515      | -145.9646        | -28.7869        | 0.0 | -2027.0018   | -1697.8218     |
-| 0.6274        | 0.99  | 150  | 0.5             | -195.8329      | -165.1599        | -30.6730        | 0.0 | -2237.8149   | -1889.7742     |
-| 0.642         | 1.32  | 200  | 0.5000          | -195.1430      | -164.6777        | -30.4653        | 0.0 | -2230.9163   | -1884.9520     |
-| 0.6241        | 1.65  | 250  | 0.5000          | -195.1471      | -164.6848        | -30.4623        | 0.0 | -2230.9573   | -1885.0226     |
-| 0.7477        | 1.98  | 300  | 0.5             | -195.8677      | -165.2624        | -30.6053        | 0.0 | -2238.1643   | -1890.7997     |
-| 0.8685        | 2.31  | 350  | 0.5             | -195.8568      | -165.2519        | -30.6049        | 0.0 | -2238.0549   | -1890.6946     |
-| 0.693         | 2.64  | 400  | 0.5             | -195.8341      | -165.2328        | -30.6013        | 0.0 | -2237.8274   | -1890.5028     |
-| 0.686         | 2.97  | 450  | 0.5             | -195.8235      | -165.2227        | -30.6008        | 0.0 | -2237.7224   | -1890.4027     |
-| 0.6119        | 3.3   | 500  | 0.5             | -195.8122      | -165.2139        | -30.5983        | 0.0 | -2237.6084   | -1890.3141     |
-| 0.5902        | 3.63  | 550  | 0.5             | -195.8078      | -165.2129        | -30.5949        | 0.0 | -2237.5649   | -1890.3043     |
-| 0.7106        | 3.96  | 600  | 0.5             | -196.2488      | -165.5701        | -30.6787        | 0.0 | -2241.9751   | -1893.8765     |
-| 0.8232        | 4.29  | 650  | 0.5             | -196.2429      | -165.5582        | -30.6847        | 0.0 | -2241.9155   | -1893.7571     |
-| 0.5881        | 4.62  | 700  | 0.5             | -197.1647      | -166.3029        | -30.8618        | 0.0 | -2251.1340   | -1901.2047     |
-| 0.6156        | 4.95  | 750  | 0.5             | -197.1416      | -166.2842        | -30.8573        | 0.0 | -2250.9023   | -1901.0179     |
-| 0.6291        | 5.28  | 800  | 0.5             | -197.1509      | -166.2928        | -30.8580        | 0.0 | -2250.9958   | -1901.1036     |
-| 0.6285        | 5.61  | 850  | 0.5             | -197.1602      | -166.2982        | -30.8620        | 0.0 | -2251.0884   | -1901.1571     |
-| 0.6918        | 5.94  | 900  | 0.5             | -197.1623      | -166.3002        | -30.8621        | 0.0 | -2251.1104   | -1901.1774     |
-| 0.7869        | 6.27  | 950  | 0.5             | -197.1630      | -166.3040        | -30.8591        | 0.0 | -2251.1169   | -1901.2148     |
-| 0.5483        | 6.6   | 1000 | 0.5             | -197.1648      | -166.2998        | -30.8650        | 0.0 | -2251.1345   | -1901.1730     |
-| 0.7744        | 6.93  | 1050 | 0.5             | -197.5333      | -166.5969        | -30.9364        | 0.0 | -2254.8201   | -1904.1442     |
-| 0.9077        | 7.26  | 1100 | 0.5             | -197.5402      | -166.6008        | -30.9394        | 0.0 | -2254.8884   | -1904.1827     |
-| 0.664         | 7.59  | 1150 | 0.5             | -197.2621      | -166.3788        | -30.8832        | 0.0 | -2252.1074   | -1901.9637     |
-| 0.6126        | 7.92  | 1200 | 0.5             | -197.2483      | -166.3705        | -30.8778        | 0.0 | -2251.9697   | -1901.8805     |
-| 0.8377        | 8.25  | 1250 | 0.5             | -197.1308      | -166.2760        | -30.8547        | 0.0 | -2250.7944   | -1900.9357     |
-| 0.6109        | 8.58  | 1300 | 0.5             | -197.1868      | -166.3199        | -30.8669        | 0.0 | -2251.3545   | -1901.3741     |
-| 0.7432        | 8.91  | 1350 | 0.5             | -197.2601      | -166.3793        | -30.8808        | 0.0 | -2252.0879   | -1901.9680     |
-| 0.8664        | 9.24  | 1400 | 0.5             | -197.1278      | -166.2694        | -30.8584        | 0.0 | -2250.7642   | -1900.8693     |
-| 0.7237        | 9.57  | 1450 | 0.5             | -197.125       | -166.2689        | -30.8561        | 0.0 | -2250.7366   | -1900.8641     |
-
-
 ### Framework versions
 
-- PEFT 0.10.0
-- Transformers 4.38.2
-- Pytorch 2.1.0+cu118
-- Datasets 2.18.0
-- Tokenizers 0.15.2
+- transformers==4.38.2
+- datasets==2.18.0
+- peft==0.10.0
+- safetensors==0.4.2
+- evaluate==0.4.1
+- bitsandbytes==0.43
+- huggingface_hub==0.22.2
+- seqeval==1.2.2
+- optimum==1.18.1
+- auto-gptq==0.7.1
+- gpustat==1.1.1
+- deepspeed==0.14.0
+- wandb==0.16.6
+- trl==0.8.1
+- accelerate==0.29.2
+- coloredlogs==15.0.1
+- traitlets==5.14.2
+- autoawq@https://github.com/casper-hansen/AutoAWQ/releases/download/v0.2.4/autoawq-0.2.4+cu118-cp310-cp310-linux_x86_64.whl
+
+### Hardware
+- Cloud provided: runpod.io

checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-100/adapter_config.json

@@ -0,0 +1,36 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "lm_head",
+    "embed_tokens"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f412f4b58c86c1795d0dbf5dc47a10ddbdc92377fd3aa4ac59c68d195543703
+size 1134834064

checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3ee0cb59d1cfd6f7f29e012f60709106a6385d247c460dc502c1caf5ce6576b
+size 172772766

checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9196a1e708bf24d6abba41cce3f8558820acc3e50f9394c5955e29eb41ffea3d
+size 14244

checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a95477309af9566acf8df625eff5b2fe03c3566409932f50c562e95a7b57865
+size 1064

checkpoint-100/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-100/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-100/trainer_state.json

@@ -0,0 +1,114 @@
+{
+  "best_metric": 0.5000000596046448,
+  "best_model_checkpoint": "./zephyr/10-04-24-Weni-WeniGPT-Agents-Zephyr-1.0.25-KTO_Experiment with a new tokenizer configuration for chat template of zephyr-2_max_steps-1470_batch_16_2024-04-10_ppid_9/checkpoint-100",
+  "epoch": 0.6600660066006601,
+  "eval_steps": 50,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.13,
+      "grad_norm": 57.293792724609375,
+      "kl": 0.03853478282690048,
+      "learning_rate": 6.222222222222222e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7078,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 20
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 112.50944519042969,
+      "kl": 3.2648494243621826,
+      "learning_rate": 0.00014666666666666666,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6966,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 40
+    },
+    {
+      "epoch": 0.33,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -413.6161193847656,
+      "eval_logps/rejected": -362.2559509277344,
+      "eval_loss": 0.5063381791114807,
+      "eval_rewards/chosen": -13.412939071655273,
+      "eval_rewards/margins": -1.0048810243606567,
+      "eval_rewards/rejected": -12.408059120178223,
+      "eval_runtime": 170.1826,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 50
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 19.94582748413086,
+      "kl": 0.45922356843948364,
+      "learning_rate": 0.00019887719298245616,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5743,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 60
+    },
+    {
+      "epoch": 0.53,
+      "grad_norm": 79.92957305908203,
+      "kl": 0.0,
+      "learning_rate": 0.0001960701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6108,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 80
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 0.06103940308094025,
+      "kl": 0.0,
+      "learning_rate": 0.00019326315789473686,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.754,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 100
+    },
+    {
+      "epoch": 0.66,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2027.0018310546875,
+      "eval_logps/rejected": -1697.82177734375,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -174.75149536132812,
+      "eval_rewards/margins": -28.786863327026367,
+      "eval_rewards/rejected": -145.96463012695312,
+      "eval_runtime": 170.0562,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 1470,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 100,
+  "total_flos": 0.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae5309801a19049c58de3649400afbb558334e14e33dff69ca022789cf2400ea
+size 5688

checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-1000/adapter_config.json

@@ -0,0 +1,36 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "lm_head",
+    "embed_tokens"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a6735a60251dd555cec76162e1919767b9354745a72e80c1859d1992e67e76a
+size 1134834064

checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92b9e2da41a1603a53b6db76ca19441521bf24e991bafc75689109d861afce43
+size 172772766

checkpoint-1000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3bd3cafcd141485c5526689e7070ba65dab1e4639fbae44141ae41439003c1f
+size 14244

checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7264db995851472cbc0a4e596f81fcf5a6c9d14b6cfc09096e5a48386a62256a
+size 1064

checkpoint-1000/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1000/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-1000/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-1000/trainer_state.json

@@ -0,0 +1,951 @@
+{
+  "best_metric": 0.5,
+  "best_model_checkpoint": "./zephyr/10-04-24-Weni-WeniGPT-Agents-Zephyr-1.0.25-KTO_Experiment with a new tokenizer configuration for chat template of zephyr-2_max_steps-1470_batch_16_2024-04-10_ppid_9/checkpoint-300",
+  "epoch": 6.600660066006601,
+  "eval_steps": 50,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.13,
+      "grad_norm": 57.293792724609375,
+      "kl": 0.03853478282690048,
+      "learning_rate": 6.222222222222222e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7078,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 20
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 112.50944519042969,
+      "kl": 3.2648494243621826,
+      "learning_rate": 0.00014666666666666666,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6966,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 40
+    },
+    {
+      "epoch": 0.33,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -413.6161193847656,
+      "eval_logps/rejected": -362.2559509277344,
+      "eval_loss": 0.5063381791114807,
+      "eval_rewards/chosen": -13.412939071655273,
+      "eval_rewards/margins": -1.0048810243606567,
+      "eval_rewards/rejected": -12.408059120178223,
+      "eval_runtime": 170.1826,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 50
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 19.94582748413086,
+      "kl": 0.45922356843948364,
+      "learning_rate": 0.00019887719298245616,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5743,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 60
+    },
+    {
+      "epoch": 0.53,
+      "grad_norm": 79.92957305908203,
+      "kl": 0.0,
+      "learning_rate": 0.0001960701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6108,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 80
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 0.06103940308094025,
+      "kl": 0.0,
+      "learning_rate": 0.00019326315789473686,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.754,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 100
+    },
+    {
+      "epoch": 0.66,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2027.0018310546875,
+      "eval_logps/rejected": -1697.82177734375,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -174.75149536132812,
+      "eval_rewards/margins": -28.786863327026367,
+      "eval_rewards/rejected": -145.96463012695312,
+      "eval_runtime": 170.0562,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 100
+    },
+    {
+      "epoch": 0.79,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001904561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.95,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 120
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018764912280701756,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6274,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 140
+    },
+    {
+      "epoch": 0.99,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.81494140625,
+      "eval_logps/rejected": -1889.774169921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83285522460938,
+      "eval_rewards/margins": -30.672954559326172,
+      "eval_rewards/rejected": -165.15989685058594,
+      "eval_runtime": 169.8795,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 150
+    },
+    {
+      "epoch": 1.06,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001848421052631579,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6387,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 160
+    },
+    {
+      "epoch": 1.19,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018203508771929826,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8327,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 180
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017922807017543862,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.642,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 200
+    },
+    {
+      "epoch": 1.32,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.916259765625,
+      "eval_logps/rejected": -1884.9520263671875,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14297485351562,
+      "eval_rewards/margins": -30.465293884277344,
+      "eval_rewards/rejected": -164.67767333984375,
+      "eval_runtime": 170.1489,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 200
+    },
+    {
+      "epoch": 1.45,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017642105263157896,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7493,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 220
+    },
+    {
+      "epoch": 1.58,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001736140350877193,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6241,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 240
+    },
+    {
+      "epoch": 1.65,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.957275390625,
+      "eval_logps/rejected": -1885.0225830078125,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14706420898438,
+      "eval_rewards/margins": -30.462318420410156,
+      "eval_rewards/rejected": -164.68475341796875,
+      "eval_runtime": 170.1092,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 250
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017080701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9621,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 260
+    },
+    {
+      "epoch": 1.85,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.000168,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7279,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 280
+    },
+    {
+      "epoch": 1.98,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016519298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7477,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 300
+    },
+    {
+      "epoch": 1.98,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.164306640625,
+      "eval_logps/rejected": -1890.7996826171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.86773681640625,
+      "eval_rewards/margins": -30.605329513549805,
+      "eval_rewards/rejected": -165.26242065429688,
+      "eval_runtime": 170.0647,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 300
+    },
+    {
+      "epoch": 2.11,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016238596491228072,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7111,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 320
+    },
+    {
+      "epoch": 2.24,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015957894736842105,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8685,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 340
+    },
+    {
+      "epoch": 2.31,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.054931640625,
+      "eval_logps/rejected": -1890.694580078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.85682678222656,
+      "eval_rewards/margins": -30.604921340942383,
+      "eval_rewards/rejected": -165.2519073486328,
+      "eval_runtime": 170.0528,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 350
+    },
+    {
+      "epoch": 2.38,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015677192982456142,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6905,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 360
+    },
+    {
+      "epoch": 2.51,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015396491228070175,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.736,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 380
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015115789473684211,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.693,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 400
+    },
+    {
+      "epoch": 2.64,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.827392578125,
+      "eval_logps/rejected": -1890.5028076171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83407592773438,
+      "eval_rewards/margins": -30.601318359375,
+      "eval_rewards/rejected": -165.23275756835938,
+      "eval_runtime": 170.2445,
+      "eval_samples_per_second": 2.056,
+      "eval_steps_per_second": 0.517,
+      "step": 400
+    },
+    {
+      "epoch": 2.77,
+      "grad_norm": 8.788210266175156e-07,
+      "kl": 0.0,
+      "learning_rate": 0.00014835087719298245,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8652,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 420
+    },
+    {
+      "epoch": 2.9,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001455438596491228,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.686,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 440
+    },
+    {
+      "epoch": 2.97,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.722412109375,
+      "eval_logps/rejected": -1890.4027099609375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.82354736328125,
+      "eval_rewards/margins": -30.600812911987305,
+      "eval_rewards/rejected": -165.22274780273438,
+      "eval_runtime": 170.3429,
+      "eval_samples_per_second": 2.055,
+      "eval_steps_per_second": 0.517,
+      "step": 450
+    },
+    {
+      "epoch": 3.04,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00014273684210526318,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6858,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 460
+    },
+    {
+      "epoch": 3.17,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001399298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8479,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 480
+    },
+    {
+      "epoch": 3.3,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013712280701754388,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6119,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 500
+    },
+    {
+      "epoch": 3.3,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.6083984375,
+      "eval_logps/rejected": -1890.3140869140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.81216430664062,
+      "eval_rewards/margins": -30.598268508911133,
+      "eval_rewards/rejected": -165.21388244628906,
+      "eval_runtime": 169.9488,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 500
+    },
+    {
+      "epoch": 3.43,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001343157894736842,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7107,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 520
+    },
+    {
+      "epoch": 3.56,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013150877192982455,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5902,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 540
+    },
+    {
+      "epoch": 3.63,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.56494140625,
+      "eval_logps/rejected": -1890.3043212890625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.80784606933594,
+      "eval_rewards/margins": -30.59491539001465,
+      "eval_rewards/rejected": -165.21290588378906,
+      "eval_runtime": 169.9756,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 550
+    },
+    {
+      "epoch": 3.7,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001287017543859649,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9042,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 560
+    },
+    {
+      "epoch": 3.83,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012589473684210527,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7268,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 580
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012308771929824564,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7106,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 600
+    },
+    {
+      "epoch": 3.96,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.97509765625,
+      "eval_logps/rejected": -1893.87646484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24884033203125,
+      "eval_rewards/margins": -30.67871856689453,
+      "eval_rewards/rejected": -165.57012939453125,
+      "eval_runtime": 169.9427,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 600
+    },
+    {
+      "epoch": 4.09,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012028070175438597,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6829,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 620
+    },
+    {
+      "epoch": 4.22,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011747368421052631,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8232,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 640
+    },
+    {
+      "epoch": 4.29,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.91552734375,
+      "eval_logps/rejected": -1893.757080078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24290466308594,
+      "eval_rewards/margins": -30.684709548950195,
+      "eval_rewards/rejected": -165.55816650390625,
+      "eval_runtime": 169.9605,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 650
+    },
+    {
+      "epoch": 4.36,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011466666666666667,
+      "logps/chosen": -2123.240234375,
+      "logps/rejected": NaN,
+      "loss": 0.6315,
+      "rewards/chosen": -188.09486389160156,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 660
+    },
+    {
+      "epoch": 4.49,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011185964912280702,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 680
+    },
+    {
+      "epoch": 4.62,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010905263157894738,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5881,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 700
+    },
+    {
+      "epoch": 4.62,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134033203125,
+      "eval_logps/rejected": -1901.2047119140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1647491455078,
+      "eval_rewards/margins": -30.8618221282959,
+      "eval_rewards/rejected": -166.30291748046875,
+      "eval_runtime": 169.9704,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 700
+    },
+    {
+      "epoch": 4.75,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010624561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8756,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 720
+    },
+    {
+      "epoch": 4.88,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010343859649122807,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6156,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 740
+    },
+    {
+      "epoch": 4.95,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.90234375,
+      "eval_logps/rejected": -1901.0179443359375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1415557861328,
+      "eval_rewards/margins": -30.857322692871094,
+      "eval_rewards/rejected": -166.28424072265625,
+      "eval_runtime": 169.9616,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 750
+    },
+    {
+      "epoch": 5.02,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010063157894736843,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7376,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 760
+    },
+    {
+      "epoch": 5.15,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.782456140350877e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 780
+    },
+    {
+      "epoch": 5.28,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.501754385964913e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6291,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 800
+    },
+    {
+      "epoch": 5.28,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.995849609375,
+      "eval_logps/rejected": -1901.1036376953125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.15087890625,
+      "eval_rewards/margins": -30.858049392700195,
+      "eval_rewards/rejected": -166.29283142089844,
+      "eval_runtime": 169.941,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 800
+    },
+    {
+      "epoch": 5.41,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.221052631578948e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7167,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 820
+    },
+    {
+      "epoch": 5.54,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.940350877192983e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6285,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 840
+    },
+    {
+      "epoch": 5.61,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.08837890625,
+      "eval_logps/rejected": -1901.1571044921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16017150878906,
+      "eval_rewards/margins": -30.86201286315918,
+      "eval_rewards/rejected": -166.2981719970703,
+      "eval_runtime": 169.9583,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 850
+    },
+    {
+      "epoch": 5.68,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.659649122807018e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7898,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 860
+    },
+    {
+      "epoch": 5.81,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.378947368421053e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8174,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 880
+    },
+    {
+      "epoch": 5.94,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.098245614035088e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6918,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 900
+    },
+    {
+      "epoch": 5.94,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.1103515625,
+      "eval_logps/rejected": -1901.1773681640625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16233825683594,
+      "eval_rewards/margins": -30.86213493347168,
+      "eval_rewards/rejected": -166.30018615722656,
+      "eval_runtime": 170.0642,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 900
+    },
+    {
+      "epoch": 6.07,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.817543859649124e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6965,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 920
+    },
+    {
+      "epoch": 6.2,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.536842105263158e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7869,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 940
+    },
+    {
+      "epoch": 6.27,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.116943359375,
+      "eval_logps/rejected": -1901.21484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16302490234375,
+      "eval_rewards/margins": -30.85906982421875,
+      "eval_rewards/rejected": -166.303955078125,
+      "eval_runtime": 169.9373,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 950
+    },
+    {
+      "epoch": 6.34,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.256140350877193e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6402,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 960
+    },
+    {
+      "epoch": 6.47,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.975438596491229e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8122,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 980
+    },
+    {
+      "epoch": 6.6,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.694736842105264e-05,
+      "logps/chosen": -2150.89111328125,
+      "logps/rejected": NaN,
+      "loss": 0.5483,
+      "rewards/chosen": -190.25399780273438,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1000
+    },
+    {
+      "epoch": 6.6,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134521484375,
+      "eval_logps/rejected": -1901.1729736328125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16481018066406,
+      "eval_rewards/margins": -30.865028381347656,
+      "eval_rewards/rejected": -166.29977416992188,
+      "eval_runtime": 169.9607,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 1470,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 100,
+  "total_flos": 0.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae5309801a19049c58de3649400afbb558334e14e33dff69ca022789cf2400ea
+size 5688

checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-1100/adapter_config.json

@@ -0,0 +1,36 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "lm_head",
+    "embed_tokens"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9a371c9c74dc6e7feb2a05b1dd39007dc16a5f35b76caab9ce2ac483ad7ee46
+size 1134834064

checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaf210564e50826b11ada69542b76a3ce19a4b9afc9b275dcce2cbf6af5c1b54
+size 172772766

checkpoint-1100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c94c375fe5ad2903d244ca6b5cc2a1a6cba4c0c26196f3b9cbd9ddd170bb0b8
+size 14244

checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e174f59377592a49c6fe6f5bccb5b55023170f3201a81a5156b2afa97d5d99e
+size 1064

checkpoint-1100/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-1100/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-1100/trainer_state.json

@@ -0,0 +1,1044 @@
+{
+  "best_metric": 0.5,
+  "best_model_checkpoint": "./zephyr/10-04-24-Weni-WeniGPT-Agents-Zephyr-1.0.25-KTO_Experiment with a new tokenizer configuration for chat template of zephyr-2_max_steps-1470_batch_16_2024-04-10_ppid_9/checkpoint-300",
+  "epoch": 7.260726072607261,
+  "eval_steps": 50,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.13,
+      "grad_norm": 57.293792724609375,
+      "kl": 0.03853478282690048,
+      "learning_rate": 6.222222222222222e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7078,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 20
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 112.50944519042969,
+      "kl": 3.2648494243621826,
+      "learning_rate": 0.00014666666666666666,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6966,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 40
+    },
+    {
+      "epoch": 0.33,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -413.6161193847656,
+      "eval_logps/rejected": -362.2559509277344,
+      "eval_loss": 0.5063381791114807,
+      "eval_rewards/chosen": -13.412939071655273,
+      "eval_rewards/margins": -1.0048810243606567,
+      "eval_rewards/rejected": -12.408059120178223,
+      "eval_runtime": 170.1826,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 50
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 19.94582748413086,
+      "kl": 0.45922356843948364,
+      "learning_rate": 0.00019887719298245616,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5743,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 60
+    },
+    {
+      "epoch": 0.53,
+      "grad_norm": 79.92957305908203,
+      "kl": 0.0,
+      "learning_rate": 0.0001960701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6108,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 80
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 0.06103940308094025,
+      "kl": 0.0,
+      "learning_rate": 0.00019326315789473686,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.754,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 100
+    },
+    {
+      "epoch": 0.66,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2027.0018310546875,
+      "eval_logps/rejected": -1697.82177734375,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -174.75149536132812,
+      "eval_rewards/margins": -28.786863327026367,
+      "eval_rewards/rejected": -145.96463012695312,
+      "eval_runtime": 170.0562,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 100
+    },
+    {
+      "epoch": 0.79,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001904561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.95,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 120
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018764912280701756,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6274,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 140
+    },
+    {
+      "epoch": 0.99,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.81494140625,
+      "eval_logps/rejected": -1889.774169921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83285522460938,
+      "eval_rewards/margins": -30.672954559326172,
+      "eval_rewards/rejected": -165.15989685058594,
+      "eval_runtime": 169.8795,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 150
+    },
+    {
+      "epoch": 1.06,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001848421052631579,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6387,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 160
+    },
+    {
+      "epoch": 1.19,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018203508771929826,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8327,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 180
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017922807017543862,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.642,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 200
+    },
+    {
+      "epoch": 1.32,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.916259765625,
+      "eval_logps/rejected": -1884.9520263671875,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14297485351562,
+      "eval_rewards/margins": -30.465293884277344,
+      "eval_rewards/rejected": -164.67767333984375,
+      "eval_runtime": 170.1489,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 200
+    },
+    {
+      "epoch": 1.45,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017642105263157896,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7493,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 220
+    },
+    {
+      "epoch": 1.58,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001736140350877193,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6241,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 240
+    },
+    {
+      "epoch": 1.65,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.957275390625,
+      "eval_logps/rejected": -1885.0225830078125,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14706420898438,
+      "eval_rewards/margins": -30.462318420410156,
+      "eval_rewards/rejected": -164.68475341796875,
+      "eval_runtime": 170.1092,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 250
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017080701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9621,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 260
+    },
+    {
+      "epoch": 1.85,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.000168,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7279,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 280
+    },
+    {
+      "epoch": 1.98,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016519298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7477,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 300
+    },
+    {
+      "epoch": 1.98,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.164306640625,
+      "eval_logps/rejected": -1890.7996826171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.86773681640625,
+      "eval_rewards/margins": -30.605329513549805,
+      "eval_rewards/rejected": -165.26242065429688,
+      "eval_runtime": 170.0647,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 300
+    },
+    {
+      "epoch": 2.11,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016238596491228072,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7111,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 320
+    },
+    {
+      "epoch": 2.24,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015957894736842105,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8685,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 340
+    },
+    {
+      "epoch": 2.31,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.054931640625,
+      "eval_logps/rejected": -1890.694580078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.85682678222656,
+      "eval_rewards/margins": -30.604921340942383,
+      "eval_rewards/rejected": -165.2519073486328,
+      "eval_runtime": 170.0528,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 350
+    },
+    {
+      "epoch": 2.38,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015677192982456142,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6905,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 360
+    },
+    {
+      "epoch": 2.51,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015396491228070175,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.736,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 380
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015115789473684211,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.693,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 400
+    },
+    {
+      "epoch": 2.64,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.827392578125,
+      "eval_logps/rejected": -1890.5028076171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83407592773438,
+      "eval_rewards/margins": -30.601318359375,
+      "eval_rewards/rejected": -165.23275756835938,
+      "eval_runtime": 170.2445,
+      "eval_samples_per_second": 2.056,
+      "eval_steps_per_second": 0.517,
+      "step": 400
+    },
+    {
+      "epoch": 2.77,
+      "grad_norm": 8.788210266175156e-07,
+      "kl": 0.0,
+      "learning_rate": 0.00014835087719298245,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8652,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 420
+    },
+    {
+      "epoch": 2.9,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001455438596491228,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.686,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 440
+    },
+    {
+      "epoch": 2.97,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.722412109375,
+      "eval_logps/rejected": -1890.4027099609375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.82354736328125,
+      "eval_rewards/margins": -30.600812911987305,
+      "eval_rewards/rejected": -165.22274780273438,
+      "eval_runtime": 170.3429,
+      "eval_samples_per_second": 2.055,
+      "eval_steps_per_second": 0.517,
+      "step": 450
+    },
+    {
+      "epoch": 3.04,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00014273684210526318,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6858,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 460
+    },
+    {
+      "epoch": 3.17,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001399298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8479,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 480
+    },
+    {
+      "epoch": 3.3,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013712280701754388,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6119,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 500
+    },
+    {
+      "epoch": 3.3,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.6083984375,
+      "eval_logps/rejected": -1890.3140869140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.81216430664062,
+      "eval_rewards/margins": -30.598268508911133,
+      "eval_rewards/rejected": -165.21388244628906,
+      "eval_runtime": 169.9488,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 500
+    },
+    {
+      "epoch": 3.43,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001343157894736842,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7107,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 520
+    },
+    {
+      "epoch": 3.56,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013150877192982455,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5902,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 540
+    },
+    {
+      "epoch": 3.63,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.56494140625,
+      "eval_logps/rejected": -1890.3043212890625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.80784606933594,
+      "eval_rewards/margins": -30.59491539001465,
+      "eval_rewards/rejected": -165.21290588378906,
+      "eval_runtime": 169.9756,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 550
+    },
+    {
+      "epoch": 3.7,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001287017543859649,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9042,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 560
+    },
+    {
+      "epoch": 3.83,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012589473684210527,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7268,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 580
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012308771929824564,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7106,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 600
+    },
+    {
+      "epoch": 3.96,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.97509765625,
+      "eval_logps/rejected": -1893.87646484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24884033203125,
+      "eval_rewards/margins": -30.67871856689453,
+      "eval_rewards/rejected": -165.57012939453125,
+      "eval_runtime": 169.9427,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 600
+    },
+    {
+      "epoch": 4.09,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012028070175438597,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6829,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 620
+    },
+    {
+      "epoch": 4.22,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011747368421052631,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8232,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 640
+    },
+    {
+      "epoch": 4.29,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.91552734375,
+      "eval_logps/rejected": -1893.757080078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24290466308594,
+      "eval_rewards/margins": -30.684709548950195,
+      "eval_rewards/rejected": -165.55816650390625,
+      "eval_runtime": 169.9605,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 650
+    },
+    {
+      "epoch": 4.36,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011466666666666667,
+      "logps/chosen": -2123.240234375,
+      "logps/rejected": NaN,
+      "loss": 0.6315,
+      "rewards/chosen": -188.09486389160156,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 660
+    },
+    {
+      "epoch": 4.49,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011185964912280702,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 680
+    },
+    {
+      "epoch": 4.62,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010905263157894738,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5881,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 700
+    },
+    {
+      "epoch": 4.62,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134033203125,
+      "eval_logps/rejected": -1901.2047119140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1647491455078,
+      "eval_rewards/margins": -30.8618221282959,
+      "eval_rewards/rejected": -166.30291748046875,
+      "eval_runtime": 169.9704,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 700
+    },
+    {
+      "epoch": 4.75,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010624561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8756,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 720
+    },
+    {
+      "epoch": 4.88,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010343859649122807,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6156,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 740
+    },
+    {
+      "epoch": 4.95,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.90234375,
+      "eval_logps/rejected": -1901.0179443359375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1415557861328,
+      "eval_rewards/margins": -30.857322692871094,
+      "eval_rewards/rejected": -166.28424072265625,
+      "eval_runtime": 169.9616,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 750
+    },
+    {
+      "epoch": 5.02,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010063157894736843,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7376,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 760
+    },
+    {
+      "epoch": 5.15,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.782456140350877e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 780
+    },
+    {
+      "epoch": 5.28,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.501754385964913e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6291,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 800
+    },
+    {
+      "epoch": 5.28,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.995849609375,
+      "eval_logps/rejected": -1901.1036376953125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.15087890625,
+      "eval_rewards/margins": -30.858049392700195,
+      "eval_rewards/rejected": -166.29283142089844,
+      "eval_runtime": 169.941,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 800
+    },
+    {
+      "epoch": 5.41,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.221052631578948e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7167,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 820
+    },
+    {
+      "epoch": 5.54,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.940350877192983e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6285,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 840
+    },
+    {
+      "epoch": 5.61,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.08837890625,
+      "eval_logps/rejected": -1901.1571044921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16017150878906,
+      "eval_rewards/margins": -30.86201286315918,
+      "eval_rewards/rejected": -166.2981719970703,
+      "eval_runtime": 169.9583,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 850
+    },
+    {
+      "epoch": 5.68,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.659649122807018e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7898,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 860
+    },
+    {
+      "epoch": 5.81,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.378947368421053e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8174,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 880
+    },
+    {
+      "epoch": 5.94,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.098245614035088e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6918,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 900
+    },
+    {
+      "epoch": 5.94,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.1103515625,
+      "eval_logps/rejected": -1901.1773681640625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16233825683594,
+      "eval_rewards/margins": -30.86213493347168,
+      "eval_rewards/rejected": -166.30018615722656,
+      "eval_runtime": 170.0642,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 900
+    },
+    {
+      "epoch": 6.07,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.817543859649124e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6965,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 920
+    },
+    {
+      "epoch": 6.2,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.536842105263158e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7869,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 940
+    },
+    {
+      "epoch": 6.27,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.116943359375,
+      "eval_logps/rejected": -1901.21484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16302490234375,
+      "eval_rewards/margins": -30.85906982421875,
+      "eval_rewards/rejected": -166.303955078125,
+      "eval_runtime": 169.9373,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 950
+    },
+    {
+      "epoch": 6.34,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.256140350877193e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6402,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 960
+    },
+    {
+      "epoch": 6.47,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.975438596491229e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8122,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 980
+    },
+    {
+      "epoch": 6.6,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.694736842105264e-05,
+      "logps/chosen": -2150.89111328125,
+      "logps/rejected": NaN,
+      "loss": 0.5483,
+      "rewards/chosen": -190.25399780273438,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1000
+    },
+    {
+      "epoch": 6.6,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134521484375,
+      "eval_logps/rejected": -1901.1729736328125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16481018066406,
+      "eval_rewards/margins": -30.865028381347656,
+      "eval_rewards/rejected": -166.29977416992188,
+      "eval_runtime": 169.9607,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1000
+    },
+    {
+      "epoch": 6.73,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.414035087719299e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 1.0998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1020
+    },
+    {
+      "epoch": 6.86,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.133333333333334e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7744,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1040
+    },
+    {
+      "epoch": 6.93,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2254.820068359375,
+      "eval_logps/rejected": -1904.1441650390625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.53334045410156,
+      "eval_rewards/margins": -30.936431884765625,
+      "eval_rewards/rejected": -166.59690856933594,
+      "eval_runtime": 169.9328,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 1050
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.852631578947369e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7891,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1060
+    },
+    {
+      "epoch": 7.13,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.571929824561404e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7203,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1080
+    },
+    {
+      "epoch": 7.26,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.291228070175439e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9077,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1100
+    },
+    {
+      "epoch": 7.26,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2254.888427734375,
+      "eval_logps/rejected": -1904.1827392578125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.54017639160156,
+      "eval_rewards/margins": -30.939420700073242,
+      "eval_rewards/rejected": -166.6007537841797,
+      "eval_runtime": 169.9818,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1100
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 1470,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 100,
+  "total_flos": 0.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae5309801a19049c58de3649400afbb558334e14e33dff69ca022789cf2400ea
+size 5688

checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-1200/adapter_config.json

@@ -0,0 +1,36 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj",
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "lm_head",
+    "embed_tokens"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a34892bba9479a33faeb36ceec4290aeba59d3a82abf645874a81f3fd670f9b
+size 1134834064

checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99903a5b8db99d3795bacf076d4b9cc93979ba0ebaf6f5895453ca3aa999dfec
+size 172772766

checkpoint-1200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27a79239f98d586c6d293becfe4724cb48ad892f743d1e770886cde54b3333d6
+size 14244

checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3d496c0c5cba2ece3b6fbeaa4eb9daaa3402ba1e3d2b995451385e27b0416f5
+size 1064

checkpoint-1200/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1200/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-1200/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-1200/trainer_state.json

@@ -0,0 +1,1137 @@
+{
+  "best_metric": 0.5,
+  "best_model_checkpoint": "./zephyr/10-04-24-Weni-WeniGPT-Agents-Zephyr-1.0.25-KTO_Experiment with a new tokenizer configuration for chat template of zephyr-2_max_steps-1470_batch_16_2024-04-10_ppid_9/checkpoint-300",
+  "epoch": 7.920792079207921,
+  "eval_steps": 50,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.13,
+      "grad_norm": 57.293792724609375,
+      "kl": 0.03853478282690048,
+      "learning_rate": 6.222222222222222e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7078,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 20
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 112.50944519042969,
+      "kl": 3.2648494243621826,
+      "learning_rate": 0.00014666666666666666,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6966,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 40
+    },
+    {
+      "epoch": 0.33,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -413.6161193847656,
+      "eval_logps/rejected": -362.2559509277344,
+      "eval_loss": 0.5063381791114807,
+      "eval_rewards/chosen": -13.412939071655273,
+      "eval_rewards/margins": -1.0048810243606567,
+      "eval_rewards/rejected": -12.408059120178223,
+      "eval_runtime": 170.1826,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 50
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 19.94582748413086,
+      "kl": 0.45922356843948364,
+      "learning_rate": 0.00019887719298245616,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5743,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 60
+    },
+    {
+      "epoch": 0.53,
+      "grad_norm": 79.92957305908203,
+      "kl": 0.0,
+      "learning_rate": 0.0001960701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6108,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 80
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 0.06103940308094025,
+      "kl": 0.0,
+      "learning_rate": 0.00019326315789473686,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.754,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 100
+    },
+    {
+      "epoch": 0.66,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2027.0018310546875,
+      "eval_logps/rejected": -1697.82177734375,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -174.75149536132812,
+      "eval_rewards/margins": -28.786863327026367,
+      "eval_rewards/rejected": -145.96463012695312,
+      "eval_runtime": 170.0562,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 100
+    },
+    {
+      "epoch": 0.79,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001904561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.95,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 120
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018764912280701756,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6274,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 140
+    },
+    {
+      "epoch": 0.99,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.81494140625,
+      "eval_logps/rejected": -1889.774169921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83285522460938,
+      "eval_rewards/margins": -30.672954559326172,
+      "eval_rewards/rejected": -165.15989685058594,
+      "eval_runtime": 169.8795,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 150
+    },
+    {
+      "epoch": 1.06,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001848421052631579,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6387,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 160
+    },
+    {
+      "epoch": 1.19,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00018203508771929826,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8327,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 180
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017922807017543862,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.642,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 200
+    },
+    {
+      "epoch": 1.32,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.916259765625,
+      "eval_logps/rejected": -1884.9520263671875,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14297485351562,
+      "eval_rewards/margins": -30.465293884277344,
+      "eval_rewards/rejected": -164.67767333984375,
+      "eval_runtime": 170.1489,
+      "eval_samples_per_second": 2.057,
+      "eval_steps_per_second": 0.517,
+      "step": 200
+    },
+    {
+      "epoch": 1.45,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017642105263157896,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7493,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 220
+    },
+    {
+      "epoch": 1.58,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001736140350877193,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6241,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 240
+    },
+    {
+      "epoch": 1.65,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2230.957275390625,
+      "eval_logps/rejected": -1885.0225830078125,
+      "eval_loss": 0.5000000596046448,
+      "eval_rewards/chosen": -195.14706420898438,
+      "eval_rewards/margins": -30.462318420410156,
+      "eval_rewards/rejected": -164.68475341796875,
+      "eval_runtime": 170.1092,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 250
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00017080701754385965,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9621,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 260
+    },
+    {
+      "epoch": 1.85,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.000168,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7279,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 280
+    },
+    {
+      "epoch": 1.98,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016519298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7477,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 300
+    },
+    {
+      "epoch": 1.98,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.164306640625,
+      "eval_logps/rejected": -1890.7996826171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.86773681640625,
+      "eval_rewards/margins": -30.605329513549805,
+      "eval_rewards/rejected": -165.26242065429688,
+      "eval_runtime": 170.0647,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 300
+    },
+    {
+      "epoch": 2.11,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00016238596491228072,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7111,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 320
+    },
+    {
+      "epoch": 2.24,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015957894736842105,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8685,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 340
+    },
+    {
+      "epoch": 2.31,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2238.054931640625,
+      "eval_logps/rejected": -1890.694580078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.85682678222656,
+      "eval_rewards/margins": -30.604921340942383,
+      "eval_rewards/rejected": -165.2519073486328,
+      "eval_runtime": 170.0528,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 350
+    },
+    {
+      "epoch": 2.38,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015677192982456142,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6905,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 360
+    },
+    {
+      "epoch": 2.51,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015396491228070175,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.736,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 380
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00015115789473684211,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.693,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 400
+    },
+    {
+      "epoch": 2.64,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.827392578125,
+      "eval_logps/rejected": -1890.5028076171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.83407592773438,
+      "eval_rewards/margins": -30.601318359375,
+      "eval_rewards/rejected": -165.23275756835938,
+      "eval_runtime": 170.2445,
+      "eval_samples_per_second": 2.056,
+      "eval_steps_per_second": 0.517,
+      "step": 400
+    },
+    {
+      "epoch": 2.77,
+      "grad_norm": 8.788210266175156e-07,
+      "kl": 0.0,
+      "learning_rate": 0.00014835087719298245,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8652,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 420
+    },
+    {
+      "epoch": 2.9,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001455438596491228,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.686,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 440
+    },
+    {
+      "epoch": 2.97,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.722412109375,
+      "eval_logps/rejected": -1890.4027099609375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.82354736328125,
+      "eval_rewards/margins": -30.600812911987305,
+      "eval_rewards/rejected": -165.22274780273438,
+      "eval_runtime": 170.3429,
+      "eval_samples_per_second": 2.055,
+      "eval_steps_per_second": 0.517,
+      "step": 450
+    },
+    {
+      "epoch": 3.04,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00014273684210526318,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6858,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 460
+    },
+    {
+      "epoch": 3.17,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001399298245614035,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8479,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 480
+    },
+    {
+      "epoch": 3.3,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013712280701754388,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6119,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 500
+    },
+    {
+      "epoch": 3.3,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.6083984375,
+      "eval_logps/rejected": -1890.3140869140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.81216430664062,
+      "eval_rewards/margins": -30.598268508911133,
+      "eval_rewards/rejected": -165.21388244628906,
+      "eval_runtime": 169.9488,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 500
+    },
+    {
+      "epoch": 3.43,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001343157894736842,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7107,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 520
+    },
+    {
+      "epoch": 3.56,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00013150877192982455,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5902,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 540
+    },
+    {
+      "epoch": 3.63,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2237.56494140625,
+      "eval_logps/rejected": -1890.3043212890625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -195.80784606933594,
+      "eval_rewards/margins": -30.59491539001465,
+      "eval_rewards/rejected": -165.21290588378906,
+      "eval_runtime": 169.9756,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 550
+    },
+    {
+      "epoch": 3.7,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.0001287017543859649,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9042,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 560
+    },
+    {
+      "epoch": 3.83,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012589473684210527,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7268,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 580
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012308771929824564,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7106,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 600
+    },
+    {
+      "epoch": 3.96,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.97509765625,
+      "eval_logps/rejected": -1893.87646484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24884033203125,
+      "eval_rewards/margins": -30.67871856689453,
+      "eval_rewards/rejected": -165.57012939453125,
+      "eval_runtime": 169.9427,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 600
+    },
+    {
+      "epoch": 4.09,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00012028070175438597,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6829,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 620
+    },
+    {
+      "epoch": 4.22,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011747368421052631,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8232,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 640
+    },
+    {
+      "epoch": 4.29,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2241.91552734375,
+      "eval_logps/rejected": -1893.757080078125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -196.24290466308594,
+      "eval_rewards/margins": -30.684709548950195,
+      "eval_rewards/rejected": -165.55816650390625,
+      "eval_runtime": 169.9605,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 650
+    },
+    {
+      "epoch": 4.36,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011466666666666667,
+      "logps/chosen": -2123.240234375,
+      "logps/rejected": NaN,
+      "loss": 0.6315,
+      "rewards/chosen": -188.09486389160156,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 660
+    },
+    {
+      "epoch": 4.49,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00011185964912280702,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 680
+    },
+    {
+      "epoch": 4.62,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010905263157894738,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.5881,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 700
+    },
+    {
+      "epoch": 4.62,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134033203125,
+      "eval_logps/rejected": -1901.2047119140625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1647491455078,
+      "eval_rewards/margins": -30.8618221282959,
+      "eval_rewards/rejected": -166.30291748046875,
+      "eval_runtime": 169.9704,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 700
+    },
+    {
+      "epoch": 4.75,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010624561403508772,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8756,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 720
+    },
+    {
+      "epoch": 4.88,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010343859649122807,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6156,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 740
+    },
+    {
+      "epoch": 4.95,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.90234375,
+      "eval_logps/rejected": -1901.0179443359375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.1415557861328,
+      "eval_rewards/margins": -30.857322692871094,
+      "eval_rewards/rejected": -166.28424072265625,
+      "eval_runtime": 169.9616,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 750
+    },
+    {
+      "epoch": 5.02,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 0.00010063157894736843,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7376,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 760
+    },
+    {
+      "epoch": 5.15,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.782456140350877e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 780
+    },
+    {
+      "epoch": 5.28,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.501754385964913e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6291,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 800
+    },
+    {
+      "epoch": 5.28,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2250.995849609375,
+      "eval_logps/rejected": -1901.1036376953125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.15087890625,
+      "eval_rewards/margins": -30.858049392700195,
+      "eval_rewards/rejected": -166.29283142089844,
+      "eval_runtime": 169.941,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 800
+    },
+    {
+      "epoch": 5.41,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.221052631578948e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7167,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 820
+    },
+    {
+      "epoch": 5.54,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.940350877192983e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6285,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 840
+    },
+    {
+      "epoch": 5.61,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.08837890625,
+      "eval_logps/rejected": -1901.1571044921875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16017150878906,
+      "eval_rewards/margins": -30.86201286315918,
+      "eval_rewards/rejected": -166.2981719970703,
+      "eval_runtime": 169.9583,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 850
+    },
+    {
+      "epoch": 5.68,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.659649122807018e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7898,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 860
+    },
+    {
+      "epoch": 5.81,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.378947368421053e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8174,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 880
+    },
+    {
+      "epoch": 5.94,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.098245614035088e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6918,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 900
+    },
+    {
+      "epoch": 5.94,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.1103515625,
+      "eval_logps/rejected": -1901.1773681640625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16233825683594,
+      "eval_rewards/margins": -30.86213493347168,
+      "eval_rewards/rejected": -166.30018615722656,
+      "eval_runtime": 170.0642,
+      "eval_samples_per_second": 2.058,
+      "eval_steps_per_second": 0.517,
+      "step": 900
+    },
+    {
+      "epoch": 6.07,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.817543859649124e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6965,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 920
+    },
+    {
+      "epoch": 6.2,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.536842105263158e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7869,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 940
+    },
+    {
+      "epoch": 6.27,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.116943359375,
+      "eval_logps/rejected": -1901.21484375,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16302490234375,
+      "eval_rewards/margins": -30.85906982421875,
+      "eval_rewards/rejected": -166.303955078125,
+      "eval_runtime": 169.9373,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 950
+    },
+    {
+      "epoch": 6.34,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.256140350877193e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6402,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 960
+    },
+    {
+      "epoch": 6.47,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.975438596491229e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.8122,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 980
+    },
+    {
+      "epoch": 6.6,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.694736842105264e-05,
+      "logps/chosen": -2150.89111328125,
+      "logps/rejected": NaN,
+      "loss": 0.5483,
+      "rewards/chosen": -190.25399780273438,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1000
+    },
+    {
+      "epoch": 6.6,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.134521484375,
+      "eval_logps/rejected": -1901.1729736328125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.16481018066406,
+      "eval_rewards/margins": -30.865028381347656,
+      "eval_rewards/rejected": -166.29977416992188,
+      "eval_runtime": 169.9607,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1000
+    },
+    {
+      "epoch": 6.73,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.414035087719299e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 1.0998,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1020
+    },
+    {
+      "epoch": 6.86,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.133333333333334e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7744,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1040
+    },
+    {
+      "epoch": 6.93,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2254.820068359375,
+      "eval_logps/rejected": -1904.1441650390625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.53334045410156,
+      "eval_rewards/margins": -30.936431884765625,
+      "eval_rewards/rejected": -166.59690856933594,
+      "eval_runtime": 169.9328,
+      "eval_samples_per_second": 2.06,
+      "eval_steps_per_second": 0.518,
+      "step": 1050
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.852631578947369e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7891,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1060
+    },
+    {
+      "epoch": 7.13,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.571929824561404e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7203,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1080
+    },
+    {
+      "epoch": 7.26,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.291228070175439e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9077,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1100
+    },
+    {
+      "epoch": 7.26,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2254.888427734375,
+      "eval_logps/rejected": -1904.1827392578125,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.54017639160156,
+      "eval_rewards/margins": -30.939420700073242,
+      "eval_rewards/rejected": -166.6007537841797,
+      "eval_runtime": 169.9818,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1100
+    },
+    {
+      "epoch": 7.39,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.010526315789474e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6196,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1120
+    },
+    {
+      "epoch": 7.52,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.729824561403509e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.664,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1140
+    },
+    {
+      "epoch": 7.59,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2252.107421875,
+      "eval_logps/rejected": -1901.9637451171875,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.26206970214844,
+      "eval_rewards/margins": -30.88323974609375,
+      "eval_rewards/rejected": -166.37884521484375,
+      "eval_runtime": 169.9835,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1150
+    },
+    {
+      "epoch": 7.66,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.449122807017544e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.7551,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1160
+    },
+    {
+      "epoch": 7.79,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.168421052631579e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.9193,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1180
+    },
+    {
+      "epoch": 7.92,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.887719298245614e-05,
+      "logps/chosen": NaN,
+      "logps/rejected": NaN,
+      "loss": 0.6126,
+      "rewards/chosen": NaN,
+      "rewards/margins": NaN,
+      "rewards/rejected": NaN,
+      "step": 1200
+    },
+    {
+      "epoch": 7.92,
+      "eval_kl": 0.0,
+      "eval_logps/chosen": -2251.9697265625,
+      "eval_logps/rejected": -1901.8804931640625,
+      "eval_loss": 0.5,
+      "eval_rewards/chosen": -197.24830627441406,
+      "eval_rewards/margins": -30.877805709838867,
+      "eval_rewards/rejected": -166.37051391601562,
+      "eval_runtime": 169.9515,
+      "eval_samples_per_second": 2.059,
+      "eval_steps_per_second": 0.518,
+      "step": 1200
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 1470,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 100,
+  "total_flos": 0.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae5309801a19049c58de3649400afbb558334e14e33dff69ca022789cf2400ea
+size 5688

checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0