Hugging Face's logo

HachiML
/
Mists-7B-v01-single-turn

Feature Extraction
Transformers
Safetensors
mists
trl
sft
Generated from Trainer
custom_code
Model card Files Community
Mists-7B-v01-single-turn / README.md
HachiML's picture
HachiML
Upload processor
df38566 verified
preview code
|
raw
history blame contribute delete
2.76 kB
metadata
base_model: HachiML/Mists-7B-v01-projector-trained
license: apache-2.0
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: Mists-7B-v01-single-turn
    results: []

Visualize in Weights & Biases

Mists-7B-v01-single-turn

This model is a fine-tuned version of HachiML/Mists-7B-v01-projector-trained on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4228

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.6859 0.0420 400 1.1048
0.7572 0.0841 800 0.8318
0.664 0.1261 1200 0.7295
0.6135 0.1682 1600 0.6526
0.5707 0.2102 2000 0.6007
0.5506 0.2523 2400 0.5653
0.5255 0.2943 2800 0.5434
0.5106 0.3363 3200 0.5219
0.4909 0.3784 3600 0.5045
0.4773 0.4204 4000 0.4874
0.4664 0.4625 4400 0.4762
0.4555 0.5045 4800 0.4663
0.4516 0.5466 5200 0.4560
0.4466 0.5886 5600 0.4490
0.4403 0.6306 6000 0.4433
0.4323 0.6727 6400 0.4383
0.4337 0.7147 6800 0.4324
0.4214 0.7568 7200 0.4297
0.4153 0.7988 7600 0.4269
0.414 0.8409 8000 0.4250
0.4187 0.8829 8400 0.4238
0.418 0.9250 8800 0.4230
0.4126 0.9670 9200 0.4228

Framework versions

  • Transformers 4.42.3
  • Pytorch 2.0.1
  • Datasets 2.20.0
  • Tokenizers 0.19.1