Hugging Face's logo

HachiML
/
Mists-7B-v01-projector-trained

Feature Extraction
Transformers
Safetensors
mists
trl
sft
Generated from Trainer
custom_code
Model card Files Community
Mists-7B-v01-projector-trained / README.md
HachiML's picture
HachiML
End of training
f8dac98 verified
preview code
|
raw
history blame
2.57 kB
metadata
license: apache-2.0
base_model: HachiML/Mists-7B-v01-not-trained
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: Mists-7B-v01-projector-trained
    results: []

Visualize in Weights & Biases

Mists-7B-v01-projector-trained

This model is a fine-tuned version of HachiML/Mists-7B-v01-not-trained on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5989

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.002
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.8704 0.0526 2000 0.7797
0.7321 0.1051 4000 0.7205
0.7002 0.1577 6000 0.6992
0.6735 0.2102 8000 0.6755
0.6644 0.2628 10000 0.6625
0.6562 0.3153 12000 0.6539
0.6512 0.3679 14000 0.6473
0.638 0.4205 16000 0.6394
0.6266 0.4730 18000 0.6325
0.627 0.5256 20000 0.6278
0.6215 0.5781 22000 0.6241
0.6192 0.6307 24000 0.6199
0.6127 0.6832 26000 0.6163
0.6121 0.7358 28000 0.6131
0.6078 0.7884 30000 0.6089
0.5986 0.8409 32000 0.6058
0.6011 0.8935 34000 0.6027
0.5934 0.9460 36000 0.6005
0.589 0.9986 38000 0.5989

Framework versions

  • Transformers 4.42.3
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1