Model description

entity-recognition-general-sota-v1-finetuned-ner

This model is a fine-tuned version of numind/entity-recognition-general-sota-v1 on Babelscape/MultiNerd dataset.

It achieves the following results on the validation set:

• Loss: 0.0396
• Precision: 0.9138
• Recall: 0.9146
• F1: 0.9142
• Accuracy: 0.9857

Training and evaluation data

The dataset if filtered on english language and sampled first 1M on train and 100k on validation. further filtered with data containing atleast one tag from labels2ids mentioned below. Training data - 131280 items Validation data - 16410 items

Trained on all tags from the MultiNERD dataset.

labels2ids = { "O": 0, "B-PER": 1, "I-PER": 2, "B-ORG": 3, "I-ORG": 4, "B-LOC": 5, "I-LOC": 6, "B-ANIM": 7, "I-ANIM": 8, "B-BIO": 9, "I-BIO": 10, "B-CEL": 11, "I-CEL": 12, "B-DIS": 13, "I-DIS": 14, "B-EVE": 15, "I-EVE": 16, "B-FOOD": 17, "I-FOOD": 18, "B-INST": 19, "I-INST": 20, "B-MEDIA": 21, "I-MEDIA": 22, "B-MYTH": 23, "I-MYTH": 24, "B-PLANT": 25, "I-PLANT": 26, "B-TIME": 27, "I-TIME": 28, "B-VEHI": 29, "I-VEHI": 30, }

Training procedure

HF Trainer module

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 20
• eval_batch_size: 20
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training & Test set evaluation results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.0323	1.0	6564	0.0396	0.9138	0.9146	0.9142	0.9857

Evaluation on test set: {'eval_loss': 0.02707073651254177, 'eval_precision': 0.9378337879893957, 'eval_recall': 0.9518034704620784, 'eval_f1': 0.9447669917943954, 'eval_accuracy': 0.9901678553418342, 'eval_runtime': 133.0665, 'eval_samples_per_second': 247.305, 'eval_steps_per_second': 30.917}

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0