metadata

license: apache-2.0
tags:
  - generated_from_trainer
metrics:
  - f1
  - accuracy
model-index:
  - name: distilBERT-finetuned-resumes-sections
    results: []

distilBERT-finetuned-resumes-sections

This model is a fine-tuned version of Geotrend/distilbert-base-en-fr-cased on a private resume sections dataset. It achieves the following results on the evaluation set:

Loss: 0.0487
F1: 0.9512
Roc Auc: 0.9729
Accuracy: 0.9482

Model description

This model classifies a resume section into 12 classes.

Possible classes for a resume section

awards, certificates, contact/name/title, education, interests, languages, para, professional_experiences, projects, skills, soft_skills, summary.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
0.058	1.0	1083	0.0457	0.9186	0.9494	0.9020
0.0277	2.0	2166	0.0393	0.9327	0.9614	0.9251
0.0154	3.0	3249	0.0333	0.9425	0.9671	0.9367
0.0104	4.0	4332	0.0408	0.9357	0.9645	0.9293
0.0084	5.0	5415	0.0405	0.9376	0.9643	0.9298
0.0065	6.0	6498	0.0419	0.9439	0.9699	0.9385
0.0051	7.0	7581	0.0450	0.9412	0.9674	0.9376
0.0034	8.0	8664	0.0406	0.9433	0.9684	0.9372
0.0035	9.0	9747	0.0441	0.9403	0.9664	0.9358
0.0024	10.0	10830	0.0492	0.9419	0.9678	0.9367
0.0026	11.0	11913	0.0470	0.9468	0.9708	0.9436
0.0022	12.0	12996	0.0514	0.9424	0.9679	0.9395
0.0013	13.0	14079	0.0458	0.9478	0.9715	0.9441
0.0019	14.0	15162	0.0494	0.9477	0.9711	0.9450
0.0007	15.0	16245	0.0492	0.9496	0.9719	0.9464
0.0009	16.0	17328	0.0487	0.9512	0.9729	0.9482
0.001	17.0	18411	0.0510	0.9480	0.9711	0.9441
0.0006	18.0	19494	0.0532	0.9477	0.9709	0.9441
0.0007	19.0	20577	0.0511	0.9487	0.9720	0.9445
0.0005	20.0	21660	0.0522	0.9471	0.9710	0.9436

Framework versions

Transformers 4.20.1
Pytorch 1.12.0+cu113
Datasets 2.3.2
Tokenizers 0.12.1