Training in progress, step 90, checkpoint

checkpoint-90/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: Weni/WeniGPT-Agents-Mistral-1.0.0-SFT-merged
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-90/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Weni/WeniGPT-Agents-Mistral-1.0.0-SFT-merged",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-90/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c647bb3e381ad61c5d4a92e798a70ce72b8c621003e68040892036a29d0a2a0e
+size 167832240

checkpoint-90/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53b5e32d4e47f59c466e2e956356a532d6733d01dd7f9f83c6e71fa42c8ebf57
+size 335922386

checkpoint-90/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:418a5f105ae834c3075024076916b2a9475918fe034c12d0dd5b6d91f1aba467
+size 15024

checkpoint-90/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e07ace389d24bc1307b74f42a1e7b8f0117b0db853e2df64ff3f15cb92916a2
+size 15024

checkpoint-90/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da6a990f346d7014dffb28fa2bc7d3b890bd3c53712503fce3656da48d3d6e50
+size 15024

checkpoint-90/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e95f356ca38179b05993f55daece0223e96fa10b9a1b9ea2102a739211333f63
+size 15024

checkpoint-90/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e83e76d56bab031e020dbb8db0e99d785019f52c86d705dd2275cde1e9302a8d
+size 1064

checkpoint-90/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-90/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-90/tokenizer_config.json

@@ -0,0 +1,49 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'system' %}{{ '<<SYS>>\\n' + message['content'] + '\\n<</SYS>>\\n\\n' }}{% elif message['role'] == 'assistant' %}{{ ' '  + message['content'] + ' ' + eos_token }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "max_length": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-90/trainer_state.json

@@ -0,0 +1,204 @@
+{
+  "best_metric": 0.057673219591379166,
+  "best_model_checkpoint": "./mistral/21-04-24-Weni-WeniGPT-Agents-Mistral-1.0.0-SFT-1.0.17-DPO_Experiment on DPO with other hyperparameters and best SFT model of WeniGPT-2_max_steps-180_batch_8_2024-04-21_ppid_9/checkpoint-90",
+  "epoch": 2.903225806451613,
+  "eval_steps": 30,
+  "global_step": 90,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 69.4647445678711,
+      "learning_rate": 4.166666666666667e-06,
+      "logits/chosen": -1.8231691122055054,
+      "logits/rejected": -1.828621506690979,
+      "logps/chosen": -158.80148315429688,
+      "logps/rejected": -171.9478302001953,
+      "loss": 0.6797,
+      "rewards/accuracies": 0.25,
+      "rewards/chosen": 0.01820509135723114,
+      "rewards/margins": 0.018905125558376312,
+      "rewards/rejected": -0.0007000354235060513,
+      "step": 10
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 55.0553092956543,
+      "learning_rate": 4.741379310344828e-06,
+      "logits/chosen": -1.8079488277435303,
+      "logits/rejected": -1.8343168497085571,
+      "logps/chosen": -183.0499725341797,
+      "logps/rejected": -252.92544555664062,
+      "loss": 0.4325,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.6964864134788513,
+      "rewards/margins": 0.7227998375892639,
+      "rewards/rejected": -0.026313429698348045,
+      "step": 20
+    },
+    {
+      "epoch": 0.967741935483871,
+      "grad_norm": 12.128168106079102,
+      "learning_rate": 4.454022988505747e-06,
+      "logits/chosen": -1.8517696857452393,
+      "logits/rejected": -1.885725736618042,
+      "logps/chosen": -192.10862731933594,
+      "logps/rejected": -237.21206665039062,
+      "loss": 0.2533,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 1.7528797388076782,
+      "rewards/margins": 2.4615116119384766,
+      "rewards/rejected": -0.7086319327354431,
+      "step": 30
+    },
+    {
+      "epoch": 0.967741935483871,
+      "eval_logits/chosen": -1.8228849172592163,
+      "eval_logits/rejected": -1.8303192853927612,
+      "eval_logps/chosen": -143.72242736816406,
+      "eval_logps/rejected": -179.08694458007812,
+      "eval_loss": 0.18642649054527283,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 2.223684072494507,
+      "eval_rewards/margins": 2.5237960815429688,
+      "eval_rewards/rejected": -0.3001118302345276,
+      "eval_runtime": 12.9786,
+      "eval_samples_per_second": 2.157,
+      "eval_steps_per_second": 0.539,
+      "step": 30
+    },
+    {
+      "epoch": 1.2903225806451613,
+      "grad_norm": 14.146989822387695,
+      "learning_rate": 4.166666666666667e-06,
+      "logits/chosen": -1.8532211780548096,
+      "logits/rejected": -1.8761682510375977,
+      "logps/chosen": -176.05001831054688,
+      "logps/rejected": -235.13314819335938,
+      "loss": 0.1445,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.285470485687256,
+      "rewards/margins": 3.9319252967834473,
+      "rewards/rejected": -1.64645516872406,
+      "step": 40
+    },
+    {
+      "epoch": 1.6129032258064515,
+      "grad_norm": 11.002118110656738,
+      "learning_rate": 3.8793103448275865e-06,
+      "logits/chosen": -1.770261526107788,
+      "logits/rejected": -1.7761493921279907,
+      "logps/chosen": -149.0377197265625,
+      "logps/rejected": -169.15806579589844,
+      "loss": 0.1134,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 3.4034476280212402,
+      "rewards/margins": 4.229793071746826,
+      "rewards/rejected": -0.8263454437255859,
+      "step": 50
+    },
+    {
+      "epoch": 1.935483870967742,
+      "grad_norm": 16.15618896484375,
+      "learning_rate": 3.5919540229885056e-06,
+      "logits/chosen": -1.8206300735473633,
+      "logits/rejected": -1.845362663269043,
+      "logps/chosen": -148.51126098632812,
+      "logps/rejected": -217.98989868164062,
+      "loss": 0.088,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.102309226989746,
+      "rewards/margins": 4.121461391448975,
+      "rewards/rejected": -1.0191524028778076,
+      "step": 60
+    },
+    {
+      "epoch": 1.935483870967742,
+      "eval_logits/chosen": -1.8270312547683716,
+      "eval_logits/rejected": -1.8343855142593384,
+      "eval_logps/chosen": -139.5704803466797,
+      "eval_logps/rejected": -181.44911193847656,
+      "eval_loss": 0.0837571769952774,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 3.4692695140838623,
+      "eval_rewards/margins": 4.478025436401367,
+      "eval_rewards/rejected": -1.0087559223175049,
+      "eval_runtime": 12.9762,
+      "eval_samples_per_second": 2.158,
+      "eval_steps_per_second": 0.539,
+      "step": 60
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 3.0225539207458496,
+      "learning_rate": 3.3045977011494256e-06,
+      "logits/chosen": -1.8196799755096436,
+      "logits/rejected": -1.8384376764297485,
+      "logps/chosen": -155.3164825439453,
+      "logps/rejected": -224.11764526367188,
+      "loss": 0.0376,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.732748508453369,
+      "rewards/margins": 5.434271335601807,
+      "rewards/rejected": -1.7015235424041748,
+      "step": 70
+    },
+    {
+      "epoch": 2.5806451612903225,
+      "grad_norm": 0.4993923008441925,
+      "learning_rate": 3.017241379310345e-06,
+      "logits/chosen": -1.7981538772583008,
+      "logits/rejected": -1.8249304294586182,
+      "logps/chosen": -103.8951187133789,
+      "logps/rejected": -169.70159912109375,
+      "loss": 0.0468,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.9167656898498535,
+      "rewards/margins": 5.2651543617248535,
+      "rewards/rejected": -1.3483887910842896,
+      "step": 80
+    },
+    {
+      "epoch": 2.903225806451613,
+      "grad_norm": 1.028433084487915,
+      "learning_rate": 2.729885057471265e-06,
+      "logits/chosen": -1.7703297138214111,
+      "logits/rejected": -1.7897396087646484,
+      "logps/chosen": -105.6802749633789,
+      "logps/rejected": -199.07763671875,
+      "loss": 0.0138,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.359316825866699,
+      "rewards/margins": 6.259133338928223,
+      "rewards/rejected": -1.8998162746429443,
+      "step": 90
+    },
+    {
+      "epoch": 2.903225806451613,
+      "eval_logits/chosen": -1.8372517824172974,
+      "eval_logits/rejected": -1.8450374603271484,
+      "eval_logps/chosen": -136.87937927246094,
+      "eval_logps/rejected": -183.79039001464844,
+      "eval_loss": 0.057673219591379166,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 4.276595115661621,
+      "eval_rewards/margins": 5.987736225128174,
+      "eval_rewards/rejected": -1.7111417055130005,
+      "eval_runtime": 12.9682,
+      "eval_samples_per_second": 2.159,
+      "eval_steps_per_second": 0.54,
+      "step": 90
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 180,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 6,
+  "save_steps": 90,
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-90/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab27bfcf57472b92cbfa35939bbe709f52ec9360413f3f51657ae7dacd9c411d
+size 5432