metadata
license: apache-2.0
base_model: facebook/bart-large
tags:
- generated_from_trainer
datasets:
- clupubhealth
metrics:
- rouge
model-index:
- name: bart-pubhealth-expanded
results:
- task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: clupubhealth
type: clupubhealth
config: expanded
split: test
args: expanded
metrics:
- name: Rouge1
type: rouge
value: 29.8528
bart-pubhealth-expanded
This model is a fine-tuned version of facebook/bart-large on the clupubhealth dataset. It achieves the following results on the evaluation set:
- Loss: 2.3926
- Rouge1: 29.8528
- Rouge2: 10.8495
- Rougel: 23.3682
- Rougelsum: 23.7565
- Gen Len: 19.85
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 2.7469 | 0.26 | 500 | 2.0845 | 30.9611 | 10.7145 | 23.9719 | 24.1042 | 19.905 |
| 2.5524 | 0.51 | 1000 | 2.0628 | 32.0352 | 11.8898 | 24.9032 | 25.1368 | 19.895 |
| 2.429 | 0.77 | 1500 | 2.0787 | 32.2632 | 12.0353 | 25.1245 | 25.3728 | 19.895 |
| 2.2234 | 1.03 | 2000 | 2.1178 | 30.6437 | 11.5713 | 24.9071 | 25.1126 | 19.955 |
| 2.1249 | 1.29 | 2500 | 2.1183 | 31.6095 | 11.6573 | 25.0593 | 25.2063 | 19.87 |
| 2.0302 | 1.54 | 3000 | 2.1319 | 30.7417 | 11.4924 | 24.6388 | 24.8722 | 19.895 |
| 1.9761 | 1.8 | 3500 | 2.1850 | 31.6709 | 11.3036 | 24.4853 | 24.7571 | 19.87 |
| 1.8279 | 2.06 | 4000 | 2.2092 | 31.5778 | 11.59 | 24.7599 | 24.9956 | 19.825 |
| 1.8083 | 2.32 | 4500 | 2.1781 | 31.0441 | 10.7513 | 24.0656 | 24.3112 | 19.89 |
| 1.7527 | 2.57 | 5000 | 2.2155 | 31.1191 | 11.4519 | 24.4673 | 24.7157 | 19.81 |
| 1.723 | 2.83 | 5500 | 2.2024 | 31.9787 | 12.3158 | 24.9863 | 25.2597 | 19.94 |
| 1.5975 | 3.09 | 6000 | 2.2567 | 31.236 | 10.9733 | 24.1302 | 24.3433 | 19.9 |
| 1.5933 | 3.35 | 6500 | 2.2425 | 31.022 | 11.0249 | 24.1257 | 24.3555 | 19.92 |
| 1.5792 | 3.6 | 7000 | 2.2428 | 29.8844 | 10.3622 | 23.0802 | 23.4003 | 19.96 |
| 1.5718 | 3.86 | 7500 | 2.2367 | 31.2369 | 11.3854 | 24.8528 | 25.1287 | 19.815 |
| 1.4467 | 4.12 | 8000 | 2.2988 | 30.4903 | 10.4057 | 23.9914 | 24.239 | 19.715 |
| 1.4458 | 4.37 | 8500 | 2.2738 | 31.4345 | 11.2989 | 24.4239 | 24.6047 | 19.75 |
| 1.4342 | 4.63 | 9000 | 2.3092 | 28.8421 | 10.5744 | 23.0084 | 23.1741 | 19.855 |
| 1.4416 | 4.89 | 9500 | 2.2747 | 31.7111 | 11.5903 | 24.3422 | 24.6867 | 19.945 |
| 1.3437 | 5.15 | 10000 | 2.3203 | 31.11 | 11.0 | 24.6098 | 24.7362 | 19.81 |
| 1.3525 | 5.4 | 10500 | 2.3085 | 29.414 | 10.3412 | 23.3134 | 23.6552 | 19.935 |
| 1.3533 | 5.66 | 11000 | 2.3123 | 31.321 | 11.2686 | 23.9922 | 24.336 | 19.77 |
| 1.3248 | 5.92 | 11500 | 2.2916 | 30.8841 | 10.779 | 23.9407 | 24.0865 | 19.845 |
| 1.2617 | 6.18 | 12000 | 2.3530 | 29.7167 | 10.3162 | 23.4805 | 23.724 | 19.93 |
| 1.2846 | 6.43 | 12500 | 2.3712 | 28.3334 | 9.8425 | 22.1151 | 22.2951 | 19.92 |
| 1.2472 | 6.69 | 13000 | 2.3378 | 29.563 | 10.0033 | 23.1863 | 23.5065 | 19.865 |
| 1.2934 | 6.95 | 13500 | 2.3262 | 29.137 | 10.1232 | 22.9234 | 23.3799 | 19.855 |
| 1.2136 | 7.21 | 14000 | 2.3640 | 29.753 | 10.4865 | 23.4892 | 23.8778 | 19.885 |
| 1.2096 | 7.46 | 14500 | 2.3654 | 29.512 | 10.3891 | 23.0427 | 23.3684 | 19.88 |
| 1.211 | 7.72 | 15000 | 2.3491 | 30.9014 | 10.9117 | 24.127 | 24.3518 | 19.785 |
| 1.1954 | 7.98 | 15500 | 2.3626 | 29.0622 | 10.5405 | 22.7407 | 22.9454 | 19.84 |
| 1.1756 | 8.23 | 16000 | 2.3759 | 29.5277 | 10.2961 | 22.7888 | 23.1239 | 19.88 |
| 1.1516 | 8.49 | 16500 | 2.3772 | 29.3161 | 10.1894 | 23.0404 | 23.486 | 19.885 |
| 1.1604 | 8.75 | 17000 | 2.3710 | 29.6161 | 10.3543 | 22.8748 | 23.1849 | 19.905 |
| 1.1639 | 9.01 | 17500 | 2.3889 | 30.2817 | 10.8654 | 23.6438 | 23.8639 | 19.895 |
| 1.12 | 9.26 | 18000 | 2.3968 | 28.8747 | 9.8686 | 22.2775 | 22.6541 | 19.895 |
| 1.1136 | 9.52 | 18500 | 2.3950 | 30.1197 | 10.8992 | 23.2575 | 23.5732 | 19.86 |
| 1.1437 | 9.78 | 19000 | 2.3926 | 29.8528 | 10.8495 | 23.3682 | 23.7565 | 19.85 |
Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu117
- Datasets 2.7.1
- Tokenizers 0.13.2