xlm-roberta-base-azsci-topics

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on azsci_topics dataset. It achieves the following results on the evaluation set:

• Loss: 0.5075
• Precision: 0.8624
• Recall: 0.8707
• F1: 0.8645
• Accuracy: 0.8707

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	288	0.8674	0.6762	0.7422	0.6848	0.7422
1.3706	2.0	576	0.6110	0.8177	0.8264	0.8104	0.8264
1.3706	3.0	864	0.5391	0.8407	0.8490	0.8415	0.8490
0.5184	4.0	1152	0.5059	0.8505	0.8559	0.8493	0.8559
0.5184	5.0	1440	0.5075	0.8624	0.8707	0.8645	0.8707

Evaluation results

Topic	Precision	Recall	F1	Support
Aqrar elmlər	0.703704	0.703704	0.703704	27
Astronomiya	0	0	0	2
Biologiya elmləri	0.886598	0.819048	0.851485	105
Coğrafiya	0.75	0.705882	0.727273	17
Filologiya elmləri	0.91954	0.914286	0.916905	175
Fizika	0.710526	0.794118	0.75	34
Fəlsəfə	0.7	0.5	0.583333	14
Hüquq elmləri	1	1	1	29
Kimya	0.75	0.934426	0.832117	61
Memarlıq	1	0.4	0.571429	5
Mexanika	0	0	0	4
Pedaqogika	0.854545	1	0.921569	47
Psixologiya	0.823529	0.777778	0.8	18
Riyaziyyat	0.891892	0.846154	0.868421	39
Siyasi elmlər	0.785714	0.88	0.830189	25
Sosiologiya	0	0	0	4
Sənətşünaslıq	0.843137	0.914894	0.877551	47
Tarix	0.857143	0.846154	0.851613	78
Texnika elmləri	0.803922	0.788462	0.796117	104
Tibb elmləri	0.929936	0.986486	0.957377	148
Yer elmləri	0.692308	0.692308	0.692308	13
İqtisad elmləri	0.972603	0.934211	0.95302	152
Əczaçılıq elmləri	0	0	0	4
macro avg	0.690222	0.671213	0.673235	1152
weighted avg	0.862363	0.87066	0.864467	1152

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2