node-py/my_awesome_eli5_clm-model

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 2.2501
• Validation Loss: 4.2690
• Epoch: 35

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
7.3929	6.0819	0
5.9912	5.8303	1
5.7553	5.5854	2
5.5082	5.3645	3
5.2836	5.1815	4
5.0867	5.0252	5
4.9075	4.8834	6
4.7424	4.7747	7
4.5947	4.6684	8
4.4570	4.5836	9
4.3290	4.5194	10
4.2123	4.4408	11
4.1037	4.3965	12
3.9979	4.3630	13
3.8983	4.3101	14
3.8011	4.2792	15
3.7097	4.2592	16
3.6182	4.2285	17
3.5337	4.2061	18
3.4483	4.1943	19
3.3589	4.1787	20
3.2776	4.1684	21
3.1959	4.1764	22
3.1161	4.1673	23
3.0389	4.1643	24
2.9631	4.1647	25
2.8859	4.1639	26
2.8110	4.1737	27
2.7362	4.1677	28
2.6631	4.1952	29
2.5915	4.2045	30
2.5227	4.1999	31
2.4573	4.2159	32
2.3862	4.2372	33
2.3205	4.2358	34
2.2501	4.2690	35

Framework versions

• Transformers 4.44.0
• TensorFlow 2.16.1
• Datasets 2.21.0
• Tokenizers 0.19.1