bcai001/sumcse-sts-roberta-large

This is a reproduction of SumCSE. I don't know the author of the original paper.

SumCSE

5401.reimple-robert-large

epoch = 3.0
eval_CR = 92.18
eval_MPQA = 90.08
eval_MR = 87.44
eval_MRPC = 77.01
eval_SST2 = 92.43
eval_SUBJ = 93.46
eval_TREC = 81.49
eval_avg_sts = 0.8624794115602034
eval_avg_transfer = 87.72714285714285
eval_sickr_spearman = 0.8479586922026686
eval_stsb_spearman = 0.8770001309177381

------ test ------
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness |  Avg. |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
| 79.55 | 87.14 | 81.30 | 87.50 | 85.13 |    85.21     |      82.10      | 83.99 |
+-------+-------+-------+-------+-------+--------------+-----------------+-------+
+------+------+------+------+------+------+------+------+
|  MR  |  CR  | SUBJ | MPQA | SST2 | TREC | MRPC | Avg. |
+------+------+------+------+------+------+------+------+
| 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
+------+------+------+------+------+------+------+------+