End of training

README.md

@@ -1,12 +1,16 @@
 ---
 base_model: sanchit-gandhi/Mistral-7B-v0.1-6-layer
 tags:
+- alignment-handbook
+- trl
+- sft
+- generated_from_trainer
 - trl
 - sft
 - alignment-handbook
 - generated_from_trainer
 datasets:
-- generator
+- stingning/ultrachat
 model-index:
 - name: sanchit-gandhi/Mistral-7B-v0.1-6-layer
   results: []
@@ -17,7 +21,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # sanchit-gandhi/Mistral-7B-v0.1-6-layer
 
-This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the generator dataset.
+This model is a fine-tuned version of [sanchit-gandhi/Mistral-7B-v0.1-6-layer](https://huggingface.co/sanchit-gandhi/Mistral-7B-v0.1-6-layer) on the stingning/ultrachat dataset.
 It achieves the following results on the evaluation set:
 - Loss: 1.0042
 

config.json

@@ -21,6 +21,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.40.1",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 32000
 }

wandb/debug-internal.log

@@ -87,3 +87,9 @@
 2024-04-25 23:50:58,539 DEBUG   SenderThread:213806 [sender.py:send_request():406] send_request: stop_status
 2024-04-25 23:50:58,539 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: internal_messages
 2024-04-25 23:50:59,774 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:01,776 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:02,336 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:51:03,778 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:04,779 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:07,341 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:51:07,782 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log

wandb/run-20240425_235040-eo45cto5/files/output.log

@@ -37,3 +37,20 @@ Training completed. Do not forget to share your model on huggingface.co/models =
 [INFO|tokenization_utils_base.py:2488] 2024-04-25 23:50:57,950 >> tokenizer config file saved in ./tokenizer_config.json
 [INFO|tokenization_utils_base.py:2497] 2024-04-25 23:50:57,951 >> Special tokens file saved in ./special_tokens_map.json
 [INFO|modelcard.py:450] 2024-04-25 23:50:57,994 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'generator', 'type': 'generator', 'config': 'default', 'split': 'train', 'args': 'default'}}
+events.out.tfevents.1714089039.ip-26-0-167-177.213140.0: 100%|██████████| 5.07k/5.07k [00:00<00:00, 33.9kB/s]
+training_args.bin: 100%|██████████| 4.98k/4.98k [00:00<00:00, 30.1kB/s] | 0.00/5.07k [00:00<?, ?B/s]
+events.out.tfevents.1714089047.ip-26-0-167-177.213140.1: 100%|██████████| 364/364 [00:00<00:00, 2.09kB/s]
+Upload 3 LFS files: 100%|██████████| 3/3 [00:00<00:00,  7.88it/s]███████| 364/364 [00:00<00:00, 3.36kB/s]
+[INFO|modelcard.py:450] 2024-04-25 23:51:02,356 >> Dropping the following result as it does not have all the necessary fields:
+{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}, 'dataset': {'name': 'stingning/ultrachat', 'type': 'stingning/ultrachat', 'config': 'default', 'split': 'train', 'args': 'default'}}
+[INFO|configuration_utils.py:471] 2024-04-25 23:51:02,360 >> Configuration saved in ./config.json
+[INFO|trainer.py:3305] 2024-04-25 23:51:02,361 >> Saving model checkpoint to ./
+[INFO|configuration_utils.py:471] 2024-04-25 23:51:02,362 >> Configuration saved in ./config.json
+[INFO|configuration_utils.py:697] 2024-04-25 23:51:02,364 >> Configuration saved in ./generation_config.json
+2024-04-25 23:51:02 - INFO - __main__ - Model saved to ./
+2024-04-25 23:51:02 - INFO - __main__ - Pushing to hub...
+[INFO|modeling_utils.py:2590] 2024-04-25 23:51:07,339 >> Model weights saved in ./model.safetensors
+[INFO|tokenization_utils_base.py:2488] 2024-04-25 23:51:07,342 >> tokenizer config file saved in ./tokenizer_config.json
+[INFO|tokenization_utils_base.py:2497] 2024-04-25 23:51:07,344 >> Special tokens file saved in ./special_tokens_map.json
+[INFO|modelcard.py:450] 2024-04-25 23:51:07,388 >> Dropping the following result as it does not have all the necessary fields:

wandb/run-20240425_235040-eo45cto5/logs/debug-internal.log

@@ -87,3 +87,9 @@
 2024-04-25 23:50:58,539 DEBUG   SenderThread:213806 [sender.py:send_request():406] send_request: stop_status
 2024-04-25 23:50:58,539 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: internal_messages
 2024-04-25 23:50:59,774 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:01,776 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:02,336 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:51:03,778 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:04,779 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log
+2024-04-25 23:51:07,341 DEBUG   HandlerThread:213806 [handler.py:handle_request():146] handle_request: status_report
+2024-04-25 23:51:07,782 INFO    Thread-12 :213806 [dir_watcher.py:_on_file_modified():288] file/dir modified: /fsx/sanchit/distil-zephyr-1.5b-ssft-ultrachat/wandb/run-20240425_235040-eo45cto5/files/output.log