PaceKW/24PDInsight-Summarization

results

This model is a fine-tuned version of cahya/bert2bert-indonesian-summarization on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5550

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	5	1.3144
No log	2.0	10	0.9515
No log	3.0	15	0.6830
No log	4.0	20	0.5632
No log	5.0	25	0.5328
No log	6.0	30	0.5182
No log	7.0	35	0.5330
No log	8.0	40	0.5365
No log	9.0	45	0.5613
No log	10.0	50	0.5588
No log	11.0	55	0.5654
No log	12.0	60	0.5587
No log	13.0	65	0.5568
No log	14.0	70	0.5563
No log	15.0	75	0.5550

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1