esm2_t33_650M_UR50D-finetuned

This model is a fine-tuned version of facebook/esm2_t33_650M_UR50D on a task of predicting toxicity of protein sequences whether some protein is toxic (1) or non-toxic (0). It achieves the following results on the evaluation set:

• Loss: 0.4409
• Tp: 539
• Tn: 617
• Fp: 47
• Fn: 93
• Accuracy: 0.8920
• Precision: 0.9198
• Recall: 0.8528
• F1-score: 0.8851
• Auc: 0.8910
• Mcc: 0.7854

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Tp	Tn	Fp	Fn	Accuracy	Precision	Recall	F1-score	Auc	Mcc
0.393	1.0	1296	0.3616	507	615	49	125	0.8657	0.9119	0.8022	0.8535	0.8642	0.7356
0.3052	2.0	2592	0.3159	536	608	56	96	0.8827	0.9054	0.8481	0.8758	0.8819	0.7664
0.166	3.0	3888	0.4409	539	617	47	93	0.8920	0.9198	0.8528	0.8851	0.8910	0.7854

Framework versions

• Transformers 4.45.2
• Pytorch 1.13.1+cu117
• Datasets 3.0.1
• Tokenizers 0.20.1