gbert-base

This model is a fine-tuned version of deepset/gbert-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6361
• F1: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.6805	1.0	189	0.6439	0.0
0.6838	2.0	378	0.6409	0.0
0.6668	3.0	567	0.6376	0.0
0.6666	4.0	756	0.6388	0.0
0.684	5.0	945	0.6372	0.0
0.673	6.0	1134	0.6419	0.0
0.7006	7.0	1323	0.6381	0.0
0.6819	8.0	1512	0.6404	0.0
0.6937	9.0	1701	0.6387	0.0
0.6809	10.0	1890	0.6375	0.0
0.6753	11.0	2079	0.6386	0.0
0.6688	12.0	2268	0.6449	0.0
0.6898	13.0	2457	0.6407	0.0
0.6682	14.0	2646	0.6458	0.0
0.6923	15.0	2835	0.6498	0.0
0.6961	16.0	3024	0.6482	0.0
0.6934	17.0	3213	0.6432	0.0
0.6853	18.0	3402	0.6457	0.0
0.6747	19.0	3591	0.6489	0.0
0.6939	20.0	3780	0.6465	0.0
0.6838	21.0	3969	0.6425	0.0
0.6725	22.0	4158	0.6401	0.0
0.6736	23.0	4347	0.6435	0.0
0.6705	24.0	4536	0.6425	0.0
0.6838	25.0	4725	0.6408	0.0
0.6742	26.0	4914	0.6417	0.0
0.6658	27.0	5103	0.6405	0.0
0.6672	28.0	5292	0.6445	0.0
0.6845	29.0	5481	0.6403	0.0
0.661	30.0	5670	0.6408	0.0
0.6775	31.0	5859	0.6394	0.0
0.6556	32.0	6048	0.6420	0.0
0.6708	33.0	6237	0.6387	0.0
0.6633	34.0	6426	0.6384	0.0
0.6536	35.0	6615	0.6401	0.0
0.6681	36.0	6804	0.6383	0.0
0.6573	37.0	6993	0.6381	0.0
0.6489	38.0	7182	0.6381	0.0
0.6806	39.0	7371	0.6347	0.0
0.6267	40.0	7560	0.6373	0.0
0.6577	41.0	7749	0.6343	0.0
0.6464	42.0	7938	0.6347	0.0
0.6325	43.0	8127	0.6361	0.0
0.6583	44.0	8316	0.6363	0.0
0.6634	45.0	8505	0.6355	0.0
0.6504	46.0	8694	0.6347	0.0
0.6457	47.0	8883	0.6356	0.0
0.632	48.0	9072	0.6362	0.0
0.651	49.0	9261	0.6362	0.0
0.6538	50.0	9450	0.6361	0.0

Framework versions

• Transformers 4.32.1
• Pytorch 2.1.2
• Datasets 2.12.0
• Tokenizers 0.13.3