---
license: mit
base_model: facebook/esm2_t12_35M_UR50D
tags:
- generated_from_trainer
metrics:
- spearmanr
model-index:
- name: esm2_t12_35M_UR50D-finetuned-rep7868aav2-v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# esm2_t12_35M_UR50D-finetuned-rep7868aav2-v1

This model is a fine-tuned version of [facebook/esm2_t12_35M_UR50D](https://huggingface.co/facebook/esm2_t12_35M_UR50D) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0505
- Spearmanr: 0.7430

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Spearmanr |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|
| 0.1227        | 1.0   | 590   | 0.1132          | 0.3426    |
| 0.117         | 2.0   | 1180  | 0.1105          | 0.3945    |
| 0.1177        | 3.0   | 1770  | 0.1150          | 0.2857    |
| 0.1143        | 4.0   | 2360  | 0.1107          | 0.3518    |
| 0.1176        | 5.0   | 2950  | 0.1133          | 0.3774    |
| 0.117         | 6.0   | 3540  | 0.1115          | 0.3230    |
| 0.1178        | 7.0   | 4130  | 0.1107          | 0.4432    |
| 0.1146        | 8.0   | 4720  | 0.1105          | 0.4608    |
| 0.1182        | 9.0   | 5310  | 0.1105          | 0.4004    |
| 0.114         | 10.0  | 5900  | 0.1122          | 0.5124    |
| 0.115         | 11.0  | 6490  | 0.1108          | 0.5551    |
| 0.1152        | 12.0  | 7080  | 0.1091          | 0.5391    |
| 0.1109        | 13.0  | 7670  | 0.0705          | 0.6234    |
| 0.0888        | 14.0  | 8260  | 0.0633          | 0.6810    |
| 0.0832        | 15.0  | 8850  | 0.0584          | 0.6952    |
| 0.0705        | 16.0  | 9440  | 0.0585          | 0.7102    |
| 0.0635        | 17.0  | 10030 | 0.0581          | 0.7167    |
| 0.0606        | 18.0  | 10620 | 0.0525          | 0.7326    |
| 0.0562        | 19.0  | 11210 | 0.0555          | 0.7403    |
| 0.055         | 20.0  | 11800 | 0.0505          | 0.7430    |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.17.1
- Tokenizers 0.15.2