|
--- |
|
language: |
|
- en |
|
license: mit |
|
model-index: |
|
- name: Zeta-Alpha-E5-Mistral |
|
results: |
|
- dataset: |
|
config: en |
|
name: MTEB AmazonCounterfactualClassification (en) |
|
revision: e8379541af4e31359cca9fbcf4b00f2671dba205 |
|
split: test |
|
type: mteb/amazon_counterfactual |
|
metrics: |
|
- type: accuracy |
|
value: 77.76119402985074 |
|
- type: ap |
|
value: 39.97673988468886 |
|
- type: ap_weighted |
|
value: 39.97673988468886 |
|
- type: f1 |
|
value: 71.23171737695898 |
|
- type: f1_weighted |
|
value: 79.55230307558237 |
|
- type: main_score |
|
value: 77.76119402985074 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB AmazonPolarityClassification |
|
revision: e2d317d38cd51312af73b3d32a06d1a08b442046 |
|
split: test |
|
type: mteb/amazon_polarity |
|
metrics: |
|
- type: accuracy |
|
value: 96.61810000000001 |
|
- type: ap |
|
value: 94.99559013902017 |
|
- type: ap_weighted |
|
value: 94.99559013902017 |
|
- type: f1 |
|
value: 96.61758649480731 |
|
- type: f1_weighted |
|
value: 96.61758649480731 |
|
- type: main_score |
|
value: 96.61810000000001 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: en |
|
name: MTEB AmazonReviewsClassification (en) |
|
revision: 1399c76144fd37290681b995c656ef9b2e06e26d |
|
split: test |
|
type: mteb/amazon_reviews_multi |
|
metrics: |
|
- type: accuracy |
|
value: 59.26199999999999 |
|
- type: f1 |
|
value: 56.32963321217333 |
|
- type: f1_weighted |
|
value: 56.32963321217333 |
|
- type: main_score |
|
value: 59.26199999999999 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB ArguAna |
|
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a |
|
split: test |
|
type: mteb/arguana |
|
metrics: |
|
- type: main_score |
|
value: 65.623 |
|
- type: map_at_1 |
|
value: 41.536 |
|
- type: map_at_10 |
|
value: 57.485 |
|
- type: map_at_100 |
|
value: 58.013000000000005 |
|
- type: map_at_1000 |
|
value: 58.013000000000005 |
|
- type: map_at_20 |
|
value: 57.957 |
|
- type: map_at_3 |
|
value: 53.284 |
|
- type: map_at_5 |
|
value: 55.837 |
|
- type: mrr_at_1 |
|
value: 42.17638691322902 |
|
- type: mrr_at_10 |
|
value: 57.7096175122492 |
|
- type: mrr_at_100 |
|
value: 58.23610809196743 |
|
- type: mrr_at_1000 |
|
value: 58.23673750573145 |
|
- type: mrr_at_20 |
|
value: 58.180348622747324 |
|
- type: mrr_at_3 |
|
value: 53.44950213371275 |
|
- type: mrr_at_5 |
|
value: 56.07396870554779 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 14.098091070036958 |
|
- type: nauc_map_at_1000_max |
|
value: -16.568377380844108 |
|
- type: nauc_map_at_1000_std |
|
value: -22.105696179585834 |
|
- type: nauc_map_at_100_diff1 |
|
value: 14.096542453201625 |
|
- type: nauc_map_at_100_max |
|
value: -16.57054590195526 |
|
- type: nauc_map_at_100_std |
|
value: -22.1090324366772 |
|
- type: nauc_map_at_10_diff1 |
|
value: 13.840246695558884 |
|
- type: nauc_map_at_10_max |
|
value: -16.52098795923224 |
|
- type: nauc_map_at_10_std |
|
value: -22.074328710004032 |
|
- type: nauc_map_at_1_diff1 |
|
value: 17.117727049808984 |
|
- type: nauc_map_at_1_max |
|
value: -18.587242049712614 |
|
- type: nauc_map_at_1_std |
|
value: -22.454707653498595 |
|
- type: nauc_map_at_20_diff1 |
|
value: 14.068130846454585 |
|
- type: nauc_map_at_20_max |
|
value: -16.53942858114966 |
|
- type: nauc_map_at_20_std |
|
value: -22.10921004077996 |
|
- type: nauc_map_at_3_diff1 |
|
value: 14.596579595737097 |
|
- type: nauc_map_at_3_max |
|
value: -15.62887067464894 |
|
- type: nauc_map_at_3_std |
|
value: -22.09058102549274 |
|
- type: nauc_map_at_5_diff1 |
|
value: 13.798507062284514 |
|
- type: nauc_map_at_5_max |
|
value: -16.36834850771522 |
|
- type: nauc_map_at_5_std |
|
value: -21.984206595455134 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 12.144909427474602 |
|
- type: nauc_mrr_at_1000_max |
|
value: -17.048787138426324 |
|
- type: nauc_mrr_at_1000_std |
|
value: -21.961966140898564 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 12.143403633612827 |
|
- type: nauc_mrr_at_100_max |
|
value: -17.050945262411012 |
|
- type: nauc_mrr_at_100_std |
|
value: -21.965305811191673 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 11.88548720648553 |
|
- type: nauc_mrr_at_10_max |
|
value: -16.996705857584736 |
|
- type: nauc_mrr_at_10_std |
|
value: -21.883645748542396 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 15.37682964765565 |
|
- type: nauc_mrr_at_1_max |
|
value: -17.989361001169087 |
|
- type: nauc_mrr_at_1_std |
|
value: -21.697830490637955 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 12.119044499779363 |
|
- type: nauc_mrr_at_20_max |
|
value: -17.018675761117027 |
|
- type: nauc_mrr_at_20_std |
|
value: -21.965554459307565 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 12.535001807278187 |
|
- type: nauc_mrr_at_3_max |
|
value: -16.38816957172248 |
|
- type: nauc_mrr_at_3_std |
|
value: -22.081293367465896 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 11.892111947679496 |
|
- type: nauc_mrr_at_5_max |
|
value: -16.79709351116846 |
|
- type: nauc_mrr_at_5_std |
|
value: -21.79512696140714 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 13.67006999549869 |
|
- type: nauc_ndcg_at_1000_max |
|
value: -16.236125687432107 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -21.810131960233065 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 13.637478389163462 |
|
- type: nauc_ndcg_at_100_max |
|
value: -16.28219720987127 |
|
- type: nauc_ndcg_at_100_std |
|
value: -21.880912370176876 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 12.558591199280556 |
|
- type: nauc_ndcg_at_10_max |
|
value: -15.952826009827106 |
|
- type: nauc_ndcg_at_10_std |
|
value: -21.818643731025382 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 17.117727049808984 |
|
- type: nauc_ndcg_at_1_max |
|
value: -18.587242049712614 |
|
- type: nauc_ndcg_at_1_std |
|
value: -22.454707653498595 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 13.402986057386181 |
|
- type: nauc_ndcg_at_20_max |
|
value: -16.072631062968746 |
|
- type: nauc_ndcg_at_20_std |
|
value: -21.98468803430586 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 14.059904782033348 |
|
- type: nauc_ndcg_at_3_max |
|
value: -14.433190101994514 |
|
- type: nauc_ndcg_at_3_std |
|
value: -21.990025270634135 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 12.434165121057134 |
|
- type: nauc_ndcg_at_5_max |
|
value: -15.650774158031522 |
|
- type: nauc_ndcg_at_5_std |
|
value: -21.636716447934305 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: 1.7151819945276745 |
|
- type: nauc_precision_at_1000_max |
|
value: 20.85546049013785 |
|
- type: nauc_precision_at_1000_std |
|
value: 77.3551884133584 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -7.961881099019577 |
|
- type: nauc_precision_at_100_max |
|
value: -1.8225484680865736 |
|
- type: nauc_precision_at_100_std |
|
value: 35.484449109425384 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 0.46638305609538855 |
|
- type: nauc_precision_at_10_max |
|
value: -11.023993018739485 |
|
- type: nauc_precision_at_10_std |
|
value: -19.111584616037852 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 17.117727049808984 |
|
- type: nauc_precision_at_1_max |
|
value: -18.587242049712614 |
|
- type: nauc_precision_at_1_std |
|
value: -22.454707653498595 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -1.0298881487766305 |
|
- type: nauc_precision_at_20_max |
|
value: -4.548017977674335 |
|
- type: nauc_precision_at_20_std |
|
value: -18.901496352112133 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 12.350178962124566 |
|
- type: nauc_precision_at_3_max |
|
value: -10.271126387937858 |
|
- type: nauc_precision_at_3_std |
|
value: -21.655307623793433 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 6.011571432832696 |
|
- type: nauc_precision_at_5_max |
|
value: -12.478026665421389 |
|
- type: nauc_precision_at_5_std |
|
value: -19.845124181363882 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 1.7151819945236155 |
|
- type: nauc_recall_at_1000_max |
|
value: 20.855460490135933 |
|
- type: nauc_recall_at_1000_std |
|
value: 77.35518841335626 |
|
- type: nauc_recall_at_100_diff1 |
|
value: -7.961881099020542 |
|
- type: nauc_recall_at_100_max |
|
value: -1.8225484680932273 |
|
- type: nauc_recall_at_100_std |
|
value: 35.48444910942399 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 0.46638305609538805 |
|
- type: nauc_recall_at_10_max |
|
value: -11.023993018739322 |
|
- type: nauc_recall_at_10_std |
|
value: -19.11158461603798 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 17.117727049808984 |
|
- type: nauc_recall_at_1_max |
|
value: -18.587242049712614 |
|
- type: nauc_recall_at_1_std |
|
value: -22.454707653498595 |
|
- type: nauc_recall_at_20_diff1 |
|
value: -1.029888148776229 |
|
- type: nauc_recall_at_20_max |
|
value: -4.548017977673906 |
|
- type: nauc_recall_at_20_std |
|
value: -18.901496352110804 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 12.350178962124682 |
|
- type: nauc_recall_at_3_max |
|
value: -10.271126387937805 |
|
- type: nauc_recall_at_3_std |
|
value: -21.65530762379337 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 6.0115714328326435 |
|
- type: nauc_recall_at_5_max |
|
value: -12.478026665421405 |
|
- type: nauc_recall_at_5_std |
|
value: -19.845124181363875 |
|
- type: ndcg_at_1 |
|
value: 41.536 |
|
- type: ndcg_at_10 |
|
value: 65.623 |
|
- type: ndcg_at_100 |
|
value: 67.63 |
|
- type: ndcg_at_1000 |
|
value: 67.64099999999999 |
|
- type: ndcg_at_20 |
|
value: 67.241 |
|
- type: ndcg_at_3 |
|
value: 57.048 |
|
- type: ndcg_at_5 |
|
value: 61.678999999999995 |
|
- type: precision_at_1 |
|
value: 41.536 |
|
- type: precision_at_10 |
|
value: 9.132 |
|
- type: precision_at_100 |
|
value: 0.996 |
|
- type: precision_at_1000 |
|
value: 0.1 |
|
- type: precision_at_20 |
|
value: 4.8759999999999994 |
|
- type: precision_at_3 |
|
value: 22.641 |
|
- type: precision_at_5 |
|
value: 15.845999999999998 |
|
- type: recall_at_1 |
|
value: 41.536 |
|
- type: recall_at_10 |
|
value: 91.323 |
|
- type: recall_at_100 |
|
value: 99.57300000000001 |
|
- type: recall_at_1000 |
|
value: 99.644 |
|
- type: recall_at_20 |
|
value: 97.51100000000001 |
|
- type: recall_at_3 |
|
value: 67.923 |
|
- type: recall_at_5 |
|
value: 79.232 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB ArxivClusteringP2P |
|
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d |
|
split: test |
|
type: mteb/arxiv-clustering-p2p |
|
metrics: |
|
- type: main_score |
|
value: 52.026635060007244 |
|
- type: v_measure |
|
value: 52.026635060007244 |
|
- type: v_measure_std |
|
value: 14.357137408692006 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB ArxivClusteringS2S |
|
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 |
|
split: test |
|
type: mteb/arxiv-clustering-s2s |
|
metrics: |
|
- type: main_score |
|
value: 47.834914950269855 |
|
- type: v_measure |
|
value: 47.834914950269855 |
|
- type: v_measure_std |
|
value: 14.487028918517247 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB AskUbuntuDupQuestions |
|
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 |
|
split: test |
|
type: mteb/askubuntudupquestions-reranking |
|
metrics: |
|
- type: main_score |
|
value: 64.5808745066313 |
|
- type: map |
|
value: 64.5808745066313 |
|
- type: mrr |
|
value: 77.56540473991997 |
|
- type: nAUC_map_diff1 |
|
value: 23.168800779252464 |
|
- type: nAUC_map_max |
|
value: 30.342203769599735 |
|
- type: nAUC_map_std |
|
value: 22.562701982176833 |
|
- type: nAUC_mrr_diff1 |
|
value: 27.79261544540621 |
|
- type: nAUC_mrr_max |
|
value: 43.302228243606045 |
|
- type: nAUC_mrr_std |
|
value: 26.432985515912673 |
|
task: |
|
type: Reranking |
|
- dataset: |
|
config: default |
|
name: MTEB BIOSSES |
|
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a |
|
split: test |
|
type: mteb/biosses-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 87.97778619889539 |
|
- type: cosine_spearman |
|
value: 86.44233109293758 |
|
- type: euclidean_pearson |
|
value: 86.6664224630525 |
|
- type: euclidean_spearman |
|
value: 86.44233109293758 |
|
- type: main_score |
|
value: 86.44233109293758 |
|
- type: manhattan_pearson |
|
value: 86.75174487553707 |
|
- type: manhattan_spearman |
|
value: 86.61402175201368 |
|
- type: pearson |
|
value: 87.97778619889539 |
|
- type: spearman |
|
value: 86.44233109293758 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB Banking77Classification |
|
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 |
|
split: test |
|
type: mteb/banking77 |
|
metrics: |
|
- type: accuracy |
|
value: 83.1103896103896 |
|
- type: f1 |
|
value: 82.2932953112279 |
|
- type: f1_weighted |
|
value: 82.2932953112279 |
|
- type: main_score |
|
value: 83.1103896103896 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB BiorxivClusteringP2P |
|
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 |
|
split: test |
|
type: mteb/biorxiv-clustering-p2p |
|
metrics: |
|
- type: main_score |
|
value: 43.73746639290943 |
|
- type: v_measure |
|
value: 43.73746639290943 |
|
- type: v_measure_std |
|
value: 0.8808902310879784 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB BiorxivClusteringS2S |
|
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 |
|
split: test |
|
type: mteb/biorxiv-clustering-s2s |
|
metrics: |
|
- type: main_score |
|
value: 40.73737737926463 |
|
- type: v_measure |
|
value: 40.73737737926463 |
|
- type: v_measure_std |
|
value: 0.6059982328960863 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB CQADupstackRetrieval |
|
revision: 46989137a86843e03a6195de44b09deda022eec7 |
|
split: test |
|
type: CQADupstackRetrieval_is_a_combined_dataset |
|
metrics: |
|
- type: main_score |
|
value: 48.852333333333334 |
|
- type: ndcg_at_10 |
|
value: 48.852333333333334 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB ClimateFEVER |
|
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380 |
|
split: test |
|
type: mteb/climate-fever |
|
metrics: |
|
- type: main_score |
|
value: 42.047000000000004 |
|
- type: map_at_1 |
|
value: 18.269 |
|
- type: map_at_10 |
|
value: 31.691999999999997 |
|
- type: map_at_100 |
|
value: 33.841 |
|
- type: map_at_1000 |
|
value: 34.009 |
|
- type: map_at_20 |
|
value: 32.885999999999996 |
|
- type: map_at_3 |
|
value: 26.558999999999997 |
|
- type: map_at_5 |
|
value: 29.119 |
|
- type: mrr_at_1 |
|
value: 41.69381107491857 |
|
- type: mrr_at_10 |
|
value: 54.293185460937984 |
|
- type: mrr_at_100 |
|
value: 54.87161862744807 |
|
- type: mrr_at_1000 |
|
value: 54.88722882443645 |
|
- type: mrr_at_20 |
|
value: 54.685754380853844 |
|
- type: mrr_at_3 |
|
value: 51.172638436482146 |
|
- type: mrr_at_5 |
|
value: 53.12052117263853 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 27.03807365621228 |
|
- type: nauc_map_at_1000_max |
|
value: 40.31079671403445 |
|
- type: nauc_map_at_1000_std |
|
value: 26.092423798773883 |
|
- type: nauc_map_at_100_diff1 |
|
value: 27.015360474734436 |
|
- type: nauc_map_at_100_max |
|
value: 40.28408194597505 |
|
- type: nauc_map_at_100_std |
|
value: 26.086029014261968 |
|
- type: nauc_map_at_10_diff1 |
|
value: 27.222731203652856 |
|
- type: nauc_map_at_10_max |
|
value: 40.01781109904128 |
|
- type: nauc_map_at_10_std |
|
value: 24.73681887890272 |
|
- type: nauc_map_at_1_diff1 |
|
value: 35.75107300356484 |
|
- type: nauc_map_at_1_max |
|
value: 40.2201153742901 |
|
- type: nauc_map_at_1_std |
|
value: 18.766249947929374 |
|
- type: nauc_map_at_20_diff1 |
|
value: 26.931804653893042 |
|
- type: nauc_map_at_20_max |
|
value: 40.21587995014608 |
|
- type: nauc_map_at_20_std |
|
value: 25.75452108695598 |
|
- type: nauc_map_at_3_diff1 |
|
value: 28.310387788680696 |
|
- type: nauc_map_at_3_max |
|
value: 39.285866053656385 |
|
- type: nauc_map_at_3_std |
|
value: 21.394962842915703 |
|
- type: nauc_map_at_5_diff1 |
|
value: 27.300839773785274 |
|
- type: nauc_map_at_5_max |
|
value: 39.3888708340898 |
|
- type: nauc_map_at_5_std |
|
value: 22.78299335246201 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 26.569029993582287 |
|
- type: nauc_mrr_at_1000_max |
|
value: 38.05698386072128 |
|
- type: nauc_mrr_at_1000_std |
|
value: 27.12877875031529 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 26.56693868451222 |
|
- type: nauc_mrr_at_100_max |
|
value: 38.06321319344823 |
|
- type: nauc_mrr_at_100_std |
|
value: 27.14409997788537 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 26.52694223161396 |
|
- type: nauc_mrr_at_10_max |
|
value: 38.120563154705 |
|
- type: nauc_mrr_at_10_std |
|
value: 27.11337497751667 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 29.371725407886277 |
|
- type: nauc_mrr_at_1_max |
|
value: 35.7850341702808 |
|
- type: nauc_mrr_at_1_std |
|
value: 22.69810863765783 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 26.567897033309247 |
|
- type: nauc_mrr_at_20_max |
|
value: 38.17484491649562 |
|
- type: nauc_mrr_at_20_std |
|
value: 27.218678564296972 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 26.582727973322427 |
|
- type: nauc_mrr_at_3_max |
|
value: 37.8745721692282 |
|
- type: nauc_mrr_at_3_std |
|
value: 26.567749469034307 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 26.404958533442898 |
|
- type: nauc_mrr_at_5_max |
|
value: 37.86090955141593 |
|
- type: nauc_mrr_at_5_std |
|
value: 26.65816459603454 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 25.7228323702742 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 41.024272689913296 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 31.373617783353815 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 25.467806967471812 |
|
- type: nauc_ndcg_at_100_max |
|
value: 40.68595692225817 |
|
- type: nauc_ndcg_at_100_std |
|
value: 31.327255356351774 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 25.65771458118311 |
|
- type: nauc_ndcg_at_10_max |
|
value: 40.2959313004829 |
|
- type: nauc_ndcg_at_10_std |
|
value: 28.21103387387833 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 29.371725407886277 |
|
- type: nauc_ndcg_at_1_max |
|
value: 35.7850341702808 |
|
- type: nauc_ndcg_at_1_std |
|
value: 22.69810863765783 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 25.008107221444327 |
|
- type: nauc_ndcg_at_20_max |
|
value: 40.613619354979626 |
|
- type: nauc_ndcg_at_20_std |
|
value: 30.216191744111416 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 25.85227194113396 |
|
- type: nauc_ndcg_at_3_max |
|
value: 38.32492256264965 |
|
- type: nauc_ndcg_at_3_std |
|
value: 23.735358525961033 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 25.747409532466243 |
|
- type: nauc_ndcg_at_5_max |
|
value: 39.4993084566524 |
|
- type: nauc_ndcg_at_5_std |
|
value: 25.19771375383721 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -8.149028290279253 |
|
- type: nauc_precision_at_1000_max |
|
value: -3.196086649201077 |
|
- type: nauc_precision_at_1000_std |
|
value: 13.643701012139948 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -1.892485292157653 |
|
- type: nauc_precision_at_100_max |
|
value: 7.7434454354621245 |
|
- type: nauc_precision_at_100_std |
|
value: 22.988854451791806 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 6.150550804828545 |
|
- type: nauc_precision_at_10_max |
|
value: 22.501131175285906 |
|
- type: nauc_precision_at_10_std |
|
value: 27.39677272392596 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 29.371725407886277 |
|
- type: nauc_precision_at_1_max |
|
value: 35.7850341702808 |
|
- type: nauc_precision_at_1_std |
|
value: 22.69810863765783 |
|
- type: nauc_precision_at_20_diff1 |
|
value: 2.283445965946842 |
|
- type: nauc_precision_at_20_max |
|
value: 18.59466543059599 |
|
- type: nauc_precision_at_20_std |
|
value: 29.0738299597803 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 12.963867454979258 |
|
- type: nauc_precision_at_3_max |
|
value: 30.449562657056333 |
|
- type: nauc_precision_at_3_std |
|
value: 25.581976194336352 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 8.512947940252289 |
|
- type: nauc_precision_at_5_max |
|
value: 26.12425424420038 |
|
- type: nauc_precision_at_5_std |
|
value: 24.877415885322808 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 17.151717317242028 |
|
- type: nauc_recall_at_1000_max |
|
value: 40.67913325938115 |
|
- type: nauc_recall_at_1000_std |
|
value: 49.54837910314142 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 16.83432440063162 |
|
- type: nauc_recall_at_100_max |
|
value: 34.46952489534257 |
|
- type: nauc_recall_at_100_std |
|
value: 38.26853671426454 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 19.50239551179883 |
|
- type: nauc_recall_at_10_max |
|
value: 35.74261290262663 |
|
- type: nauc_recall_at_10_std |
|
value: 28.630457514118163 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 35.75107300356484 |
|
- type: nauc_recall_at_1_max |
|
value: 40.2201153742901 |
|
- type: nauc_recall_at_1_std |
|
value: 18.766249947929374 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 16.723000685755707 |
|
- type: nauc_recall_at_20_max |
|
value: 35.272383093342405 |
|
- type: nauc_recall_at_20_std |
|
value: 32.934757635631335 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 24.024160029526794 |
|
- type: nauc_recall_at_3_max |
|
value: 38.07599046764463 |
|
- type: nauc_recall_at_3_std |
|
value: 22.648443171847685 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 21.588763686113676 |
|
- type: nauc_recall_at_5_max |
|
value: 37.16237158404055 |
|
- type: nauc_recall_at_5_std |
|
value: 24.45061830715902 |
|
- type: ndcg_at_1 |
|
value: 41.693999999999996 |
|
- type: ndcg_at_10 |
|
value: 42.047000000000004 |
|
- type: ndcg_at_100 |
|
value: 49.309 |
|
- type: ndcg_at_1000 |
|
value: 51.861999999999995 |
|
- type: ndcg_at_20 |
|
value: 44.982 |
|
- type: ndcg_at_3 |
|
value: 35.510000000000005 |
|
- type: ndcg_at_5 |
|
value: 37.529 |
|
- type: precision_at_1 |
|
value: 41.693999999999996 |
|
- type: precision_at_10 |
|
value: 13.114 |
|
- type: precision_at_100 |
|
value: 2.1069999999999998 |
|
- type: precision_at_1000 |
|
value: 0.259 |
|
- type: precision_at_20 |
|
value: 7.824000000000001 |
|
- type: precision_at_3 |
|
value: 26.796999999999997 |
|
- type: precision_at_5 |
|
value: 20.169 |
|
- type: recall_at_1 |
|
value: 18.269 |
|
- type: recall_at_10 |
|
value: 48.44 |
|
- type: recall_at_100 |
|
value: 72.909 |
|
- type: recall_at_1000 |
|
value: 86.79400000000001 |
|
- type: recall_at_20 |
|
value: 56.714 |
|
- type: recall_at_3 |
|
value: 31.85 |
|
- type: recall_at_5 |
|
value: 38.488 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB DBPedia |
|
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 |
|
split: test |
|
type: mteb/dbpedia |
|
metrics: |
|
- type: main_score |
|
value: 51.202000000000005 |
|
- type: map_at_1 |
|
value: 10.789 |
|
- type: map_at_10 |
|
value: 24.804000000000002 |
|
- type: map_at_100 |
|
value: 35.908 |
|
- type: map_at_1000 |
|
value: 37.97 |
|
- type: map_at_20 |
|
value: 29.236 |
|
- type: map_at_3 |
|
value: 17.055 |
|
- type: map_at_5 |
|
value: 20.342 |
|
- type: mrr_at_1 |
|
value: 78.0 |
|
- type: mrr_at_10 |
|
value: 83.82251984126987 |
|
- type: mrr_at_100 |
|
value: 84.00706659508124 |
|
- type: mrr_at_1000 |
|
value: 84.01172077534015 |
|
- type: mrr_at_20 |
|
value: 83.93946561479457 |
|
- type: mrr_at_3 |
|
value: 82.58333333333334 |
|
- type: mrr_at_5 |
|
value: 83.38333333333335 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 21.975683577384412 |
|
- type: nauc_map_at_1000_max |
|
value: 33.104767603973286 |
|
- type: nauc_map_at_1000_std |
|
value: 19.507936661697688 |
|
- type: nauc_map_at_100_diff1 |
|
value: 23.19428288878281 |
|
- type: nauc_map_at_100_max |
|
value: 32.47043490749479 |
|
- type: nauc_map_at_100_std |
|
value: 16.611980248500473 |
|
- type: nauc_map_at_10_diff1 |
|
value: 23.314074413061576 |
|
- type: nauc_map_at_10_max |
|
value: 18.52506648908812 |
|
- type: nauc_map_at_10_std |
|
value: -9.718219448424597 |
|
- type: nauc_map_at_1_diff1 |
|
value: 27.402329635171146 |
|
- type: nauc_map_at_1_max |
|
value: 5.898746976402726 |
|
- type: nauc_map_at_1_std |
|
value: -26.703327281110212 |
|
- type: nauc_map_at_20_diff1 |
|
value: 23.613670514044472 |
|
- type: nauc_map_at_20_max |
|
value: 25.008187375084763 |
|
- type: nauc_map_at_20_std |
|
value: -0.05206367166066498 |
|
- type: nauc_map_at_3_diff1 |
|
value: 25.673374223753598 |
|
- type: nauc_map_at_3_max |
|
value: 12.527419567406877 |
|
- type: nauc_map_at_3_std |
|
value: -20.06963757181341 |
|
- type: nauc_map_at_5_diff1 |
|
value: 24.74002578400672 |
|
- type: nauc_map_at_5_max |
|
value: 14.23437788867648 |
|
- type: nauc_map_at_5_std |
|
value: -16.317803876665256 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 53.26868100398232 |
|
- type: nauc_mrr_at_1000_max |
|
value: 67.65740877772801 |
|
- type: nauc_mrr_at_1000_std |
|
value: 39.43464159369656 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 53.25615896192901 |
|
- type: nauc_mrr_at_100_max |
|
value: 67.64777514366169 |
|
- type: nauc_mrr_at_100_std |
|
value: 39.410662043086006 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 52.94295111677663 |
|
- type: nauc_mrr_at_10_max |
|
value: 67.5393005296393 |
|
- type: nauc_mrr_at_10_std |
|
value: 39.31715440936177 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 57.148073445541826 |
|
- type: nauc_mrr_at_1_max |
|
value: 65.78742986970832 |
|
- type: nauc_mrr_at_1_std |
|
value: 34.198659989799246 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 53.223501273361265 |
|
- type: nauc_mrr_at_20_max |
|
value: 67.59762197314753 |
|
- type: nauc_mrr_at_20_std |
|
value: 39.359614957729356 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 53.619283112717184 |
|
- type: nauc_mrr_at_3_max |
|
value: 68.72067268448458 |
|
- type: nauc_mrr_at_3_std |
|
value: 40.53052904925793 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 52.86133436375577 |
|
- type: nauc_mrr_at_5_max |
|
value: 67.94415973414303 |
|
- type: nauc_mrr_at_5_std |
|
value: 40.09087346298919 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 31.008961737330505 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 49.39127418414386 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 37.509639671229806 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 32.50484024525448 |
|
- type: nauc_ndcg_at_100_max |
|
value: 46.300662423725605 |
|
- type: nauc_ndcg_at_100_std |
|
value: 28.488771981297162 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 27.911614286994414 |
|
- type: nauc_ndcg_at_10_max |
|
value: 44.70909339082426 |
|
- type: nauc_ndcg_at_10_std |
|
value: 25.644980583529154 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 51.27342509891256 |
|
- type: nauc_ndcg_at_1_max |
|
value: 54.75803307782269 |
|
- type: nauc_ndcg_at_1_std |
|
value: 27.4853058050954 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 30.29885920192407 |
|
- type: nauc_ndcg_at_20_max |
|
value: 43.45207612356715 |
|
- type: nauc_ndcg_at_20_std |
|
value: 21.59751863312113 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 31.251071625533843 |
|
- type: nauc_ndcg_at_3_max |
|
value: 48.45180697571009 |
|
- type: nauc_ndcg_at_3_std |
|
value: 32.70662167853583 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 26.175090671223877 |
|
- type: nauc_ndcg_at_5_max |
|
value: 45.2723355712432 |
|
- type: nauc_ndcg_at_5_std |
|
value: 31.461916393793 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -23.926082132378777 |
|
- type: nauc_precision_at_1000_max |
|
value: -9.350346667573811 |
|
- type: nauc_precision_at_1000_std |
|
value: 11.578726421051043 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -7.468660956171794 |
|
- type: nauc_precision_at_100_max |
|
value: 19.470414434634723 |
|
- type: nauc_precision_at_100_std |
|
value: 43.86244545951367 |
|
- type: nauc_precision_at_10_diff1 |
|
value: -2.090265656696684 |
|
- type: nauc_precision_at_10_max |
|
value: 30.778228684745386 |
|
- type: nauc_precision_at_10_std |
|
value: 44.882546930240984 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 57.148073445541826 |
|
- type: nauc_precision_at_1_max |
|
value: 65.78742986970832 |
|
- type: nauc_precision_at_1_std |
|
value: 34.198659989799246 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -3.075798118380347 |
|
- type: nauc_precision_at_20_max |
|
value: 29.52951501638172 |
|
- type: nauc_precision_at_20_std |
|
value: 47.266521222769676 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 11.892419680356198 |
|
- type: nauc_precision_at_3_max |
|
value: 43.146413741651415 |
|
- type: nauc_precision_at_3_std |
|
value: 45.2312022756118 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 0.5765950918056327 |
|
- type: nauc_precision_at_5_max |
|
value: 34.22132902314228 |
|
- type: nauc_precision_at_5_std |
|
value: 44.78272426908718 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 24.99872069707702 |
|
- type: nauc_recall_at_1000_max |
|
value: 42.17319464089324 |
|
- type: nauc_recall_at_1000_std |
|
value: 47.42376725148043 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 24.62929408109356 |
|
- type: nauc_recall_at_100_max |
|
value: 32.373805304406844 |
|
- type: nauc_recall_at_100_std |
|
value: 21.48682342485071 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 20.62337020665992 |
|
- type: nauc_recall_at_10_max |
|
value: 14.125308316827395 |
|
- type: nauc_recall_at_10_std |
|
value: -13.565096294162865 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 27.402329635171146 |
|
- type: nauc_recall_at_1_max |
|
value: 5.898746976402726 |
|
- type: nauc_recall_at_1_std |
|
value: -26.703327281110212 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 22.169766882731277 |
|
- type: nauc_recall_at_20_max |
|
value: 20.588762488556828 |
|
- type: nauc_recall_at_20_std |
|
value: -4.530608772737279 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 22.48622374174161 |
|
- type: nauc_recall_at_3_max |
|
value: 10.470407080375304 |
|
- type: nauc_recall_at_3_std |
|
value: -20.777479868757286 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 21.28438252298866 |
|
- type: nauc_recall_at_5_max |
|
value: 10.424120660451583 |
|
- type: nauc_recall_at_5_std |
|
value: -17.912853638432384 |
|
- type: ndcg_at_1 |
|
value: 65.375 |
|
- type: ndcg_at_10 |
|
value: 51.202000000000005 |
|
- type: ndcg_at_100 |
|
value: 56.12200000000001 |
|
- type: ndcg_at_1000 |
|
value: 63.306 |
|
- type: ndcg_at_20 |
|
value: 50.442 |
|
- type: ndcg_at_3 |
|
value: 56.437000000000005 |
|
- type: ndcg_at_5 |
|
value: 53.861000000000004 |
|
- type: precision_at_1 |
|
value: 78.0 |
|
- type: precision_at_10 |
|
value: 41.075 |
|
- type: precision_at_100 |
|
value: 13.032 |
|
- type: precision_at_1000 |
|
value: 2.516 |
|
- type: precision_at_20 |
|
value: 31.4 |
|
- type: precision_at_3 |
|
value: 59.833000000000006 |
|
- type: precision_at_5 |
|
value: 51.9 |
|
- type: recall_at_1 |
|
value: 10.789 |
|
- type: recall_at_10 |
|
value: 30.059 |
|
- type: recall_at_100 |
|
value: 61.817 |
|
- type: recall_at_1000 |
|
value: 84.672 |
|
- type: recall_at_20 |
|
value: 39.135 |
|
- type: recall_at_3 |
|
value: 18.017 |
|
- type: recall_at_5 |
|
value: 22.492 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB EmotionClassification |
|
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 |
|
split: test |
|
type: mteb/emotion |
|
metrics: |
|
- type: accuracy |
|
value: 57.715 |
|
- type: f1 |
|
value: 51.85468544437296 |
|
- type: f1_weighted |
|
value: 58.73946069844862 |
|
- type: main_score |
|
value: 57.715 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB FEVER |
|
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 |
|
split: test |
|
type: mteb/fever |
|
metrics: |
|
- type: main_score |
|
value: 92.438 |
|
- type: map_at_1 |
|
value: 82.678 |
|
- type: map_at_10 |
|
value: 89.90899999999999 |
|
- type: map_at_100 |
|
value: 90.09899999999999 |
|
- type: map_at_1000 |
|
value: 90.11 |
|
- type: map_at_20 |
|
value: 90.026 |
|
- type: map_at_3 |
|
value: 89.034 |
|
- type: map_at_5 |
|
value: 89.619 |
|
- type: mrr_at_1 |
|
value: 89.04890489048904 |
|
- type: mrr_at_10 |
|
value: 93.53417484605598 |
|
- type: mrr_at_100 |
|
value: 93.56969053658798 |
|
- type: mrr_at_1000 |
|
value: 93.56979354808294 |
|
- type: mrr_at_20 |
|
value: 93.56100677804474 |
|
- type: mrr_at_3 |
|
value: 93.25682568256818 |
|
- type: mrr_at_5 |
|
value: 93.46909690969086 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 50.19087206783256 |
|
- type: nauc_map_at_1000_max |
|
value: 26.223996443425424 |
|
- type: nauc_map_at_1000_std |
|
value: -8.531486546405336 |
|
- type: nauc_map_at_100_diff1 |
|
value: 50.12601833237827 |
|
- type: nauc_map_at_100_max |
|
value: 26.205753684531942 |
|
- type: nauc_map_at_100_std |
|
value: -8.502300882475792 |
|
- type: nauc_map_at_10_diff1 |
|
value: 49.48962186883297 |
|
- type: nauc_map_at_10_max |
|
value: 25.849578028607546 |
|
- type: nauc_map_at_10_std |
|
value: -8.58622126027856 |
|
- type: nauc_map_at_1_diff1 |
|
value: 56.88016472114475 |
|
- type: nauc_map_at_1_max |
|
value: 24.671479435457048 |
|
- type: nauc_map_at_1_std |
|
value: -11.980878470985619 |
|
- type: nauc_map_at_20_diff1 |
|
value: 49.813384246326905 |
|
- type: nauc_map_at_20_max |
|
value: 25.96508257517373 |
|
- type: nauc_map_at_20_std |
|
value: -8.568670117647939 |
|
- type: nauc_map_at_3_diff1 |
|
value: 49.087764097890165 |
|
- type: nauc_map_at_3_max |
|
value: 25.65938258554376 |
|
- type: nauc_map_at_3_std |
|
value: -8.859093431924775 |
|
- type: nauc_map_at_5_diff1 |
|
value: 49.08166208415013 |
|
- type: nauc_map_at_5_max |
|
value: 25.696246071825684 |
|
- type: nauc_map_at_5_std |
|
value: -8.431713254517472 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 73.35484368612293 |
|
- type: nauc_mrr_at_1000_max |
|
value: 35.657386688053336 |
|
- type: nauc_mrr_at_1000_std |
|
value: -18.09172713569766 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 73.35508125874483 |
|
- type: nauc_mrr_at_100_max |
|
value: 35.65842743437027 |
|
- type: nauc_mrr_at_100_std |
|
value: -18.08981699366641 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 73.29004337552368 |
|
- type: nauc_mrr_at_10_max |
|
value: 35.882001444609216 |
|
- type: nauc_mrr_at_10_std |
|
value: -18.05339396879553 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 74.48742882702338 |
|
- type: nauc_mrr_at_1_max |
|
value: 31.49138530538466 |
|
- type: nauc_mrr_at_1_std |
|
value: -19.510294856397955 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 73.3388656330962 |
|
- type: nauc_mrr_at_20_max |
|
value: 35.706948273788505 |
|
- type: nauc_mrr_at_20_std |
|
value: -18.140154123750992 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 73.22698350499 |
|
- type: nauc_mrr_at_3_max |
|
value: 36.4855373316516 |
|
- type: nauc_mrr_at_3_std |
|
value: -17.719256990311198 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 73.24460108538948 |
|
- type: nauc_mrr_at_5_max |
|
value: 36.322370705490634 |
|
- type: nauc_mrr_at_5_std |
|
value: -17.636279233457984 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 53.674109881592756 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 28.767387846727487 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -8.858681782014946 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 52.33608078847966 |
|
- type: nauc_ndcg_at_100_max |
|
value: 28.511414384159877 |
|
- type: nauc_ndcg_at_100_std |
|
value: -8.085385430073922 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 49.712295545440774 |
|
- type: nauc_ndcg_at_10_max |
|
value: 27.5674225152019 |
|
- type: nauc_ndcg_at_10_std |
|
value: -8.244677630275376 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 74.48742882702338 |
|
- type: nauc_ndcg_at_1_max |
|
value: 31.49138530538466 |
|
- type: nauc_ndcg_at_1_std |
|
value: -19.510294856397955 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 50.61628846813059 |
|
- type: nauc_ndcg_at_20_max |
|
value: 27.53989784238201 |
|
- type: nauc_ndcg_at_20_std |
|
value: -8.373695482986479 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 51.295860863016884 |
|
- type: nauc_ndcg_at_3_max |
|
value: 28.99776689198307 |
|
- type: nauc_ndcg_at_3_std |
|
value: -8.878181909861983 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 49.619081645734504 |
|
- type: nauc_ndcg_at_5_max |
|
value: 28.11109235395876 |
|
- type: nauc_ndcg_at_5_std |
|
value: -7.722157727171728 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -9.298540465937485 |
|
- type: nauc_precision_at_1000_max |
|
value: -1.3157308795912563 |
|
- type: nauc_precision_at_1000_std |
|
value: 1.897355386264135 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -12.246672190804334 |
|
- type: nauc_precision_at_100_max |
|
value: -0.9687067276412682 |
|
- type: nauc_precision_at_100_std |
|
value: 4.56074518564851 |
|
- type: nauc_precision_at_10_diff1 |
|
value: -15.533411370200923 |
|
- type: nauc_precision_at_10_max |
|
value: -2.191843047666222 |
|
- type: nauc_precision_at_10_std |
|
value: 3.6723841478730748 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 74.48742882702338 |
|
- type: nauc_precision_at_1_max |
|
value: 31.49138530538466 |
|
- type: nauc_precision_at_1_std |
|
value: -19.510294856397955 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -15.290364061922347 |
|
- type: nauc_precision_at_20_max |
|
value: -2.921722171191804 |
|
- type: nauc_precision_at_20_std |
|
value: 4.08482465973661 |
|
- type: nauc_precision_at_3_diff1 |
|
value: -8.208906597107383 |
|
- type: nauc_precision_at_3_max |
|
value: 2.9796478961627284 |
|
- type: nauc_precision_at_3_std |
|
value: 0.34366033602604895 |
|
- type: nauc_precision_at_5_diff1 |
|
value: -14.42241522747573 |
|
- type: nauc_precision_at_5_max |
|
value: -0.5633890785935999 |
|
- type: nauc_precision_at_5_std |
|
value: 3.7064496791809836 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: -0.5673198466803553 |
|
- type: nauc_recall_at_1000_max |
|
value: 21.92110385096128 |
|
- type: nauc_recall_at_1000_std |
|
value: 54.421987386115475 |
|
- type: nauc_recall_at_100_diff1 |
|
value: -0.6512704079314391 |
|
- type: nauc_recall_at_100_max |
|
value: 22.38252665262688 |
|
- type: nauc_recall_at_100_std |
|
value: 36.50750378730013 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 11.308848658347774 |
|
- type: nauc_recall_at_10_max |
|
value: 21.077700181656738 |
|
- type: nauc_recall_at_10_std |
|
value: 8.321338697504787 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 56.88016472114475 |
|
- type: nauc_recall_at_1_max |
|
value: 24.671479435457048 |
|
- type: nauc_recall_at_1_std |
|
value: -11.980878470985619 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 5.8415071379210755 |
|
- type: nauc_recall_at_20_max |
|
value: 16.97886837481554 |
|
- type: nauc_recall_at_20_std |
|
value: 12.529145693495494 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 27.396913234035086 |
|
- type: nauc_recall_at_3_max |
|
value: 24.897648442357994 |
|
- type: nauc_recall_at_3_std |
|
value: 0.8528297027573939 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 18.295838017557397 |
|
- type: nauc_recall_at_5_max |
|
value: 24.077879268127823 |
|
- type: nauc_recall_at_5_std |
|
value: 7.099403908855888 |
|
- type: ndcg_at_1 |
|
value: 89.049 |
|
- type: ndcg_at_10 |
|
value: 92.438 |
|
- type: ndcg_at_100 |
|
value: 93.016 |
|
- type: ndcg_at_1000 |
|
value: 93.17699999999999 |
|
- type: ndcg_at_20 |
|
value: 92.713 |
|
- type: ndcg_at_3 |
|
value: 91.40599999999999 |
|
- type: ndcg_at_5 |
|
value: 92.026 |
|
- type: precision_at_1 |
|
value: 89.049 |
|
- type: precision_at_10 |
|
value: 10.917 |
|
- type: precision_at_100 |
|
value: 1.146 |
|
- type: precision_at_1000 |
|
value: 0.117 |
|
- type: precision_at_20 |
|
value: 5.56 |
|
- type: precision_at_3 |
|
value: 34.598 |
|
- type: precision_at_5 |
|
value: 21.323 |
|
- type: recall_at_1 |
|
value: 82.678 |
|
- type: recall_at_10 |
|
value: 96.465 |
|
- type: recall_at_100 |
|
value: 98.571 |
|
- type: recall_at_1000 |
|
value: 99.496 |
|
- type: recall_at_20 |
|
value: 97.342 |
|
- type: recall_at_3 |
|
value: 93.696 |
|
- type: recall_at_5 |
|
value: 95.324 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB FiQA2018 |
|
revision: 27a168819829fe9bcd655c2df245fb19452e8e06 |
|
split: test |
|
type: mteb/fiqa |
|
metrics: |
|
- type: main_score |
|
value: 58.781000000000006 |
|
- type: map_at_1 |
|
value: 31.107000000000003 |
|
- type: map_at_10 |
|
value: 50.955 |
|
- type: map_at_100 |
|
value: 53.177 |
|
- type: map_at_1000 |
|
value: 53.291 |
|
- type: map_at_20 |
|
value: 52.271 |
|
- type: map_at_3 |
|
value: 44.762 |
|
- type: map_at_5 |
|
value: 48.379 |
|
- type: mrr_at_1 |
|
value: 58.48765432098766 |
|
- type: mrr_at_10 |
|
value: 66.11429551244368 |
|
- type: mrr_at_100 |
|
value: 66.68929754431386 |
|
- type: mrr_at_1000 |
|
value: 66.71304995006113 |
|
- type: mrr_at_20 |
|
value: 66.46821550477237 |
|
- type: mrr_at_3 |
|
value: 64.22325102880657 |
|
- type: mrr_at_5 |
|
value: 65.41152263374484 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 45.490146083064445 |
|
- type: nauc_map_at_1000_max |
|
value: 33.573139354617126 |
|
- type: nauc_map_at_1000_std |
|
value: -14.07140489937541 |
|
- type: nauc_map_at_100_diff1 |
|
value: 45.48828357408913 |
|
- type: nauc_map_at_100_max |
|
value: 33.51907944260763 |
|
- type: nauc_map_at_100_std |
|
value: -14.059609883903152 |
|
- type: nauc_map_at_10_diff1 |
|
value: 45.70748844526757 |
|
- type: nauc_map_at_10_max |
|
value: 31.667587503334587 |
|
- type: nauc_map_at_10_std |
|
value: -15.076948336390855 |
|
- type: nauc_map_at_1_diff1 |
|
value: 51.42775649850064 |
|
- type: nauc_map_at_1_max |
|
value: 16.56862308325116 |
|
- type: nauc_map_at_1_std |
|
value: -14.684731980257675 |
|
- type: nauc_map_at_20_diff1 |
|
value: 45.754998522906284 |
|
- type: nauc_map_at_20_max |
|
value: 33.03759060247343 |
|
- type: nauc_map_at_20_std |
|
value: -14.750787459968736 |
|
- type: nauc_map_at_3_diff1 |
|
value: 46.45241223088609 |
|
- type: nauc_map_at_3_max |
|
value: 26.607789112226467 |
|
- type: nauc_map_at_3_std |
|
value: -14.997049792585598 |
|
- type: nauc_map_at_5_diff1 |
|
value: 45.87702900983919 |
|
- type: nauc_map_at_5_max |
|
value: 30.076255479914348 |
|
- type: nauc_map_at_5_std |
|
value: -15.062787509367553 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 55.64889336097758 |
|
- type: nauc_mrr_at_1000_max |
|
value: 48.57022261913911 |
|
- type: nauc_mrr_at_1000_std |
|
value: -12.428435474800143 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 55.62957328562593 |
|
- type: nauc_mrr_at_100_max |
|
value: 48.56575267775789 |
|
- type: nauc_mrr_at_100_std |
|
value: -12.415226616847987 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 55.5931002027865 |
|
- type: nauc_mrr_at_10_max |
|
value: 48.428200063552374 |
|
- type: nauc_mrr_at_10_std |
|
value: -12.590361961152267 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 59.470635489729105 |
|
- type: nauc_mrr_at_1_max |
|
value: 49.66866699872627 |
|
- type: nauc_mrr_at_1_std |
|
value: -13.590112604913607 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 55.60145155716686 |
|
- type: nauc_mrr_at_20_max |
|
value: 48.58677663675733 |
|
- type: nauc_mrr_at_20_std |
|
value: -12.454093344399036 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 55.76657118158415 |
|
- type: nauc_mrr_at_3_max |
|
value: 48.88547787372198 |
|
- type: nauc_mrr_at_3_std |
|
value: -13.299744066289124 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 55.55217612731964 |
|
- type: nauc_mrr_at_5_max |
|
value: 48.56957852769844 |
|
- type: nauc_mrr_at_5_std |
|
value: -12.876904435466624 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 47.2645656074121 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 39.95808937564202 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -11.366829207572232 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 46.89043419464991 |
|
- type: nauc_ndcg_at_100_max |
|
value: 39.00034359981605 |
|
- type: nauc_ndcg_at_100_std |
|
value: -10.697277437129921 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 47.07625032910763 |
|
- type: nauc_ndcg_at_10_max |
|
value: 35.51275239983428 |
|
- type: nauc_ndcg_at_10_std |
|
value: -13.965305287946128 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 59.470635489729105 |
|
- type: nauc_ndcg_at_1_max |
|
value: 49.66866699872627 |
|
- type: nauc_ndcg_at_1_std |
|
value: -13.590112604913607 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 47.44262917418296 |
|
- type: nauc_ndcg_at_20_max |
|
value: 37.6804112715633 |
|
- type: nauc_ndcg_at_20_std |
|
value: -13.174880813005297 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 44.56982475937759 |
|
- type: nauc_ndcg_at_3_max |
|
value: 37.96424549723314 |
|
- type: nauc_ndcg_at_3_std |
|
value: -13.657607148249964 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 45.427291740214024 |
|
- type: nauc_ndcg_at_5_max |
|
value: 35.42232275517991 |
|
- type: nauc_ndcg_at_5_std |
|
value: -14.510048307634808 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -16.58479747595096 |
|
- type: nauc_precision_at_1000_max |
|
value: 27.22386867486023 |
|
- type: nauc_precision_at_1000_std |
|
value: 9.41210384044254 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -11.640382840009572 |
|
- type: nauc_precision_at_100_max |
|
value: 30.20752947841474 |
|
- type: nauc_precision_at_100_std |
|
value: 10.72773947232612 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 3.2540578244055594 |
|
- type: nauc_precision_at_10_max |
|
value: 35.80515547017638 |
|
- type: nauc_precision_at_10_std |
|
value: 0.299517152086918 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 59.470635489729105 |
|
- type: nauc_precision_at_1_max |
|
value: 49.66866699872627 |
|
- type: nauc_precision_at_1_std |
|
value: -13.590112604913607 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -1.8627219860435185 |
|
- type: nauc_precision_at_20_max |
|
value: 35.9181314633325 |
|
- type: nauc_precision_at_20_std |
|
value: 4.491869749000042 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 17.94168903901189 |
|
- type: nauc_precision_at_3_max |
|
value: 41.67388438464254 |
|
- type: nauc_precision_at_3_std |
|
value: -5.38615084998387 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 9.312012525324068 |
|
- type: nauc_precision_at_5_max |
|
value: 39.52463080415461 |
|
- type: nauc_precision_at_5_std |
|
value: -2.615286156278468 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 37.0960616996064 |
|
- type: nauc_recall_at_1000_max |
|
value: 46.91967503624078 |
|
- type: nauc_recall_at_1000_std |
|
value: 36.70723581015844 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 32.54497560045993 |
|
- type: nauc_recall_at_100_max |
|
value: 26.846226776082734 |
|
- type: nauc_recall_at_100_std |
|
value: 9.257918182671672 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 40.05619869408745 |
|
- type: nauc_recall_at_10_max |
|
value: 25.504319960057014 |
|
- type: nauc_recall_at_10_std |
|
value: -12.57012842016253 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 51.42775649850064 |
|
- type: nauc_recall_at_1_max |
|
value: 16.56862308325116 |
|
- type: nauc_recall_at_1_std |
|
value: -14.684731980257675 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 39.34128607816815 |
|
- type: nauc_recall_at_20_max |
|
value: 28.31147877410395 |
|
- type: nauc_recall_at_20_std |
|
value: -10.295180225906224 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 41.31333745355922 |
|
- type: nauc_recall_at_3_max |
|
value: 22.642649370921276 |
|
- type: nauc_recall_at_3_std |
|
value: -14.44811859378254 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 39.91795256714951 |
|
- type: nauc_recall_at_5_max |
|
value: 24.396817798634245 |
|
- type: nauc_recall_at_5_std |
|
value: -13.696077909471175 |
|
- type: ndcg_at_1 |
|
value: 58.48799999999999 |
|
- type: ndcg_at_10 |
|
value: 58.781000000000006 |
|
- type: ndcg_at_100 |
|
value: 65.212 |
|
- type: ndcg_at_1000 |
|
value: 66.85900000000001 |
|
- type: ndcg_at_20 |
|
value: 61.529999999999994 |
|
- type: ndcg_at_3 |
|
value: 54.864000000000004 |
|
- type: ndcg_at_5 |
|
value: 56.223 |
|
- type: precision_at_1 |
|
value: 58.48799999999999 |
|
- type: precision_at_10 |
|
value: 16.111 |
|
- type: precision_at_100 |
|
value: 2.298 |
|
- type: precision_at_1000 |
|
value: 0.257 |
|
- type: precision_at_20 |
|
value: 9.306000000000001 |
|
- type: precision_at_3 |
|
value: 36.317 |
|
- type: precision_at_5 |
|
value: 26.759 |
|
- type: recall_at_1 |
|
value: 31.107000000000003 |
|
- type: recall_at_10 |
|
value: 65.08500000000001 |
|
- type: recall_at_100 |
|
value: 87.91 |
|
- type: recall_at_1000 |
|
value: 97.817 |
|
- type: recall_at_20 |
|
value: 73.282 |
|
- type: recall_at_3 |
|
value: 49.317 |
|
- type: recall_at_5 |
|
value: 56.617 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB HotpotQA |
|
revision: ab518f4d6fcca38d87c25209f94beba119d02014 |
|
split: test |
|
type: mteb/hotpotqa |
|
metrics: |
|
- type: main_score |
|
value: 79.449 |
|
- type: map_at_1 |
|
value: 41.384 |
|
- type: map_at_10 |
|
value: 72.844 |
|
- type: map_at_100 |
|
value: 73.589 |
|
- type: map_at_1000 |
|
value: 73.624 |
|
- type: map_at_20 |
|
value: 73.317 |
|
- type: map_at_3 |
|
value: 69.427 |
|
- type: map_at_5 |
|
value: 71.68299999999999 |
|
- type: mrr_at_1 |
|
value: 82.76839972991222 |
|
- type: mrr_at_10 |
|
value: 87.92045807744634 |
|
- type: mrr_at_100 |
|
value: 88.05529589670978 |
|
- type: mrr_at_1000 |
|
value: 88.05935891074716 |
|
- type: mrr_at_20 |
|
value: 88.01402112970962 |
|
- type: mrr_at_3 |
|
value: 87.3216295295969 |
|
- type: mrr_at_5 |
|
value: 87.69705154175074 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 20.396629535039704 |
|
- type: nauc_map_at_1000_max |
|
value: 39.10949908339265 |
|
- type: nauc_map_at_1000_std |
|
value: 10.224729673688502 |
|
- type: nauc_map_at_100_diff1 |
|
value: 20.381077063965574 |
|
- type: nauc_map_at_100_max |
|
value: 39.12262980169527 |
|
- type: nauc_map_at_100_std |
|
value: 10.256952440972226 |
|
- type: nauc_map_at_10_diff1 |
|
value: 20.227214487416916 |
|
- type: nauc_map_at_10_max |
|
value: 39.065878364926085 |
|
- type: nauc_map_at_10_std |
|
value: 9.830819360569484 |
|
- type: nauc_map_at_1_diff1 |
|
value: 60.61929089121275 |
|
- type: nauc_map_at_1_max |
|
value: 49.53547409224507 |
|
- type: nauc_map_at_1_std |
|
value: 0.2722096857291782 |
|
- type: nauc_map_at_20_diff1 |
|
value: 20.183915365850165 |
|
- type: nauc_map_at_20_max |
|
value: 39.06905710390586 |
|
- type: nauc_map_at_20_std |
|
value: 10.244769286257812 |
|
- type: nauc_map_at_3_diff1 |
|
value: 18.953350220177363 |
|
- type: nauc_map_at_3_max |
|
value: 36.89647666189664 |
|
- type: nauc_map_at_3_std |
|
value: 6.856939205711613 |
|
- type: nauc_map_at_5_diff1 |
|
value: 19.74313508534105 |
|
- type: nauc_map_at_5_max |
|
value: 38.42860611762909 |
|
- type: nauc_map_at_5_std |
|
value: 8.620757357067802 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 60.544760748070416 |
|
- type: nauc_mrr_at_1000_max |
|
value: 53.536700750600176 |
|
- type: nauc_mrr_at_1000_std |
|
value: 4.356103341419562 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 60.543037850402825 |
|
- type: nauc_mrr_at_100_max |
|
value: 53.54473925679791 |
|
- type: nauc_mrr_at_100_std |
|
value: 4.3713759172294475 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 60.57585979923885 |
|
- type: nauc_mrr_at_10_max |
|
value: 53.65882404973961 |
|
- type: nauc_mrr_at_10_std |
|
value: 4.46866142907982 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 60.61929089121275 |
|
- type: nauc_mrr_at_1_max |
|
value: 49.53547409224507 |
|
- type: nauc_mrr_at_1_std |
|
value: 0.2722096857291782 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 60.541893232518674 |
|
- type: nauc_mrr_at_20_max |
|
value: 53.6135776399171 |
|
- type: nauc_mrr_at_20_std |
|
value: 4.443552945861195 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 60.46996364153697 |
|
- type: nauc_mrr_at_3_max |
|
value: 53.981024588336936 |
|
- type: nauc_mrr_at_3_std |
|
value: 4.300285863686253 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 60.562791070200426 |
|
- type: nauc_mrr_at_5_max |
|
value: 53.884058343579966 |
|
- type: nauc_mrr_at_5_std |
|
value: 4.35333313705802 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 26.909558826785485 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 43.2090252545764 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 13.24632397019833 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 26.4096138903785 |
|
- type: nauc_ndcg_at_100_max |
|
value: 43.50667894420325 |
|
- type: nauc_ndcg_at_100_std |
|
value: 14.272929786830657 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 25.261392560708607 |
|
- type: nauc_ndcg_at_10_max |
|
value: 43.02496845139645 |
|
- type: nauc_ndcg_at_10_std |
|
value: 12.753991213996402 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 60.61929089121275 |
|
- type: nauc_ndcg_at_1_max |
|
value: 49.53547409224507 |
|
- type: nauc_ndcg_at_1_std |
|
value: 0.2722096857291782 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 25.15730629354081 |
|
- type: nauc_ndcg_at_20_max |
|
value: 43.10358742768409 |
|
- type: nauc_ndcg_at_20_std |
|
value: 14.103247675055986 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 23.492158440363873 |
|
- type: nauc_ndcg_at_3_max |
|
value: 39.880317429264736 |
|
- type: nauc_ndcg_at_3_std |
|
value: 7.852278799949863 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 24.46471897598423 |
|
- type: nauc_ndcg_at_5_max |
|
value: 41.901821932685294 |
|
- type: nauc_ndcg_at_5_std |
|
value: 10.33482164145028 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: 14.556112531859444 |
|
- type: nauc_precision_at_1000_max |
|
value: 54.51236512101235 |
|
- type: nauc_precision_at_1000_std |
|
value: 68.89420216988455 |
|
- type: nauc_precision_at_100_diff1 |
|
value: 14.116319404924122 |
|
- type: nauc_precision_at_100_max |
|
value: 50.42943334977378 |
|
- type: nauc_precision_at_100_std |
|
value: 49.80016017936658 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 14.530495877243805 |
|
- type: nauc_precision_at_10_max |
|
value: 43.89651175033577 |
|
- type: nauc_precision_at_10_std |
|
value: 24.764789718434958 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 60.61929089121275 |
|
- type: nauc_precision_at_1_max |
|
value: 49.53547409224507 |
|
- type: nauc_precision_at_1_std |
|
value: 0.2722096857291782 |
|
- type: nauc_precision_at_20_diff1 |
|
value: 11.499635650364958 |
|
- type: nauc_precision_at_20_max |
|
value: 44.499499741252265 |
|
- type: nauc_precision_at_20_std |
|
value: 33.743842605352725 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 14.621019803797811 |
|
- type: nauc_precision_at_3_max |
|
value: 38.1391146398071 |
|
- type: nauc_precision_at_3_std |
|
value: 11.050680597126348 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 14.878056511475538 |
|
- type: nauc_precision_at_5_max |
|
value: 41.52854585813069 |
|
- type: nauc_precision_at_5_std |
|
value: 16.596884488946877 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 14.556112531860405 |
|
- type: nauc_recall_at_1000_max |
|
value: 54.512365121012444 |
|
- type: nauc_recall_at_1000_std |
|
value: 68.89420216988472 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 14.11631940492389 |
|
- type: nauc_recall_at_100_max |
|
value: 50.42943334977325 |
|
- type: nauc_recall_at_100_std |
|
value: 49.80016017936635 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 14.530495877243975 |
|
- type: nauc_recall_at_10_max |
|
value: 43.89651175033581 |
|
- type: nauc_recall_at_10_std |
|
value: 24.764789718434855 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 60.61929089121275 |
|
- type: nauc_recall_at_1_max |
|
value: 49.53547409224507 |
|
- type: nauc_recall_at_1_std |
|
value: 0.2722096857291782 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 11.499635650364953 |
|
- type: nauc_recall_at_20_max |
|
value: 44.499499741252166 |
|
- type: nauc_recall_at_20_std |
|
value: 33.74384260535269 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 14.621019803797758 |
|
- type: nauc_recall_at_3_max |
|
value: 38.139114639807104 |
|
- type: nauc_recall_at_3_std |
|
value: 11.050680597126208 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 14.87805651147543 |
|
- type: nauc_recall_at_5_max |
|
value: 41.52854585813069 |
|
- type: nauc_recall_at_5_std |
|
value: 16.59688448894684 |
|
- type: ndcg_at_1 |
|
value: 82.768 |
|
- type: ndcg_at_10 |
|
value: 79.449 |
|
- type: ndcg_at_100 |
|
value: 81.878 |
|
- type: ndcg_at_1000 |
|
value: 82.526 |
|
- type: ndcg_at_20 |
|
value: 80.601 |
|
- type: ndcg_at_3 |
|
value: 74.899 |
|
- type: ndcg_at_5 |
|
value: 77.586 |
|
- type: precision_at_1 |
|
value: 82.768 |
|
- type: precision_at_10 |
|
value: 16.804 |
|
- type: precision_at_100 |
|
value: 1.8659999999999999 |
|
- type: precision_at_1000 |
|
value: 0.19499999999999998 |
|
- type: precision_at_20 |
|
value: 8.770999999999999 |
|
- type: precision_at_3 |
|
value: 49.381 |
|
- type: precision_at_5 |
|
value: 31.746000000000002 |
|
- type: recall_at_1 |
|
value: 41.384 |
|
- type: recall_at_10 |
|
value: 84.018 |
|
- type: recall_at_100 |
|
value: 93.30199999999999 |
|
- type: recall_at_1000 |
|
value: 97.529 |
|
- type: recall_at_20 |
|
value: 87.711 |
|
- type: recall_at_3 |
|
value: 74.072 |
|
- type: recall_at_5 |
|
value: 79.365 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB ImdbClassification |
|
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 |
|
split: test |
|
type: mteb/imdb |
|
metrics: |
|
- type: accuracy |
|
value: 95.50119999999997 |
|
- type: ap |
|
value: 93.27855740989341 |
|
- type: ap_weighted |
|
value: 93.27855740989341 |
|
- type: f1 |
|
value: 95.49922732391366 |
|
- type: f1_weighted |
|
value: 95.49922732391366 |
|
- type: main_score |
|
value: 95.50119999999997 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB MSMARCO |
|
revision: c5a29a104738b98a9e76336939199e264163d4a0 |
|
split: dev |
|
type: mteb/msmarco |
|
metrics: |
|
- type: main_score |
|
value: 44.181 |
|
- type: map_at_1 |
|
value: 24.3 |
|
- type: map_at_10 |
|
value: 37.064 |
|
- type: map_at_100 |
|
value: 38.217 |
|
- type: map_at_1000 |
|
value: 38.261 |
|
- type: map_at_20 |
|
value: 37.797 |
|
- type: map_at_3 |
|
value: 33.03 |
|
- type: map_at_5 |
|
value: 35.382000000000005 |
|
- type: mrr_at_1 |
|
value: 25.014326647564474 |
|
- type: mrr_at_10 |
|
value: 37.67002092145362 |
|
- type: mrr_at_100 |
|
value: 38.76618716955713 |
|
- type: mrr_at_1000 |
|
value: 38.803895343578624 |
|
- type: mrr_at_20 |
|
value: 38.372875531879025 |
|
- type: mrr_at_3 |
|
value: 33.74164278892073 |
|
- type: mrr_at_5 |
|
value: 36.04250238777461 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 37.38914109165067 |
|
- type: nauc_map_at_1000_max |
|
value: 9.290439022090213 |
|
- type: nauc_map_at_1000_std |
|
value: -17.68507604596775 |
|
- type: nauc_map_at_100_diff1 |
|
value: 37.3858106261435 |
|
- type: nauc_map_at_100_max |
|
value: 9.292194370842791 |
|
- type: nauc_map_at_100_std |
|
value: -17.6461510679294 |
|
- type: nauc_map_at_10_diff1 |
|
value: 37.24355836056403 |
|
- type: nauc_map_at_10_max |
|
value: 9.19029394636661 |
|
- type: nauc_map_at_10_std |
|
value: -18.216369315567626 |
|
- type: nauc_map_at_1_diff1 |
|
value: 40.298938486026984 |
|
- type: nauc_map_at_1_max |
|
value: 8.149499405622326 |
|
- type: nauc_map_at_1_std |
|
value: -17.09168853307602 |
|
- type: nauc_map_at_20_diff1 |
|
value: 37.344123641575216 |
|
- type: nauc_map_at_20_max |
|
value: 9.24559383901809 |
|
- type: nauc_map_at_20_std |
|
value: -17.842740773642962 |
|
- type: nauc_map_at_3_diff1 |
|
value: 37.4023127968177 |
|
- type: nauc_map_at_3_max |
|
value: 8.930674077317596 |
|
- type: nauc_map_at_3_std |
|
value: -18.68520909934096 |
|
- type: nauc_map_at_5_diff1 |
|
value: 37.12600186091895 |
|
- type: nauc_map_at_5_max |
|
value: 9.173506919924861 |
|
- type: nauc_map_at_5_std |
|
value: -18.625677130615294 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 37.34256456294692 |
|
- type: nauc_mrr_at_1000_max |
|
value: 9.276741130450404 |
|
- type: nauc_mrr_at_1000_std |
|
value: -17.41693013754444 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 37.33775949993714 |
|
- type: nauc_mrr_at_100_max |
|
value: 9.28051163202218 |
|
- type: nauc_mrr_at_100_std |
|
value: -17.381741706111445 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 37.21506505847139 |
|
- type: nauc_mrr_at_10_max |
|
value: 9.200324529184542 |
|
- type: nauc_mrr_at_10_std |
|
value: -17.904321523440817 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 40.314678345050915 |
|
- type: nauc_mrr_at_1_max |
|
value: 8.193685362111243 |
|
- type: nauc_mrr_at_1_std |
|
value: -17.096535887474175 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 37.293746882874004 |
|
- type: nauc_mrr_at_20_max |
|
value: 9.256273923676206 |
|
- type: nauc_mrr_at_20_std |
|
value: -17.528338232043577 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 37.254812254578376 |
|
- type: nauc_mrr_at_3_max |
|
value: 8.903676300128614 |
|
- type: nauc_mrr_at_3_std |
|
value: -18.49940979312031 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 37.08969825523026 |
|
- type: nauc_mrr_at_5_max |
|
value: 9.194982897416688 |
|
- type: nauc_mrr_at_5_std |
|
value: -18.291840579141315 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 36.930810397557096 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 9.8356345032183 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -16.308145152943887 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 36.901149744427414 |
|
- type: nauc_ndcg_at_100_max |
|
value: 9.96065454342114 |
|
- type: nauc_ndcg_at_100_std |
|
value: -14.983815239399584 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 36.441571794416724 |
|
- type: nauc_ndcg_at_10_max |
|
value: 9.57337658776914 |
|
- type: nauc_ndcg_at_10_std |
|
value: -17.88037638294921 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 40.314678345050915 |
|
- type: nauc_ndcg_at_1_max |
|
value: 8.193685362111243 |
|
- type: nauc_ndcg_at_1_std |
|
value: -17.096535887474175 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 36.775334219857484 |
|
- type: nauc_ndcg_at_20_max |
|
value: 9.789544462660507 |
|
- type: nauc_ndcg_at_20_std |
|
value: -16.465733594062474 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 36.58838956901628 |
|
- type: nauc_ndcg_at_3_max |
|
value: 9.089768089567865 |
|
- type: nauc_ndcg_at_3_std |
|
value: -19.12823913473232 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 36.147729725463364 |
|
- type: nauc_ndcg_at_5_max |
|
value: 9.53707003144017 |
|
- type: nauc_ndcg_at_5_std |
|
value: -18.91372487441106 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -6.013504255890098 |
|
- type: nauc_precision_at_1000_max |
|
value: 6.319348588937731 |
|
- type: nauc_precision_at_1000_std |
|
value: 6.360339202992953 |
|
- type: nauc_precision_at_100_diff1 |
|
value: 14.846649240680357 |
|
- type: nauc_precision_at_100_max |
|
value: 11.751644343520605 |
|
- type: nauc_precision_at_100_std |
|
value: 16.881205928162444 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 30.328513184776966 |
|
- type: nauc_precision_at_10_max |
|
value: 9.988509735977631 |
|
- type: nauc_precision_at_10_std |
|
value: -15.609966599969837 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 40.314678345050915 |
|
- type: nauc_precision_at_1_max |
|
value: 8.193685362111243 |
|
- type: nauc_precision_at_1_std |
|
value: -17.096535887474175 |
|
- type: nauc_precision_at_20_diff1 |
|
value: 28.248245250811543 |
|
- type: nauc_precision_at_20_max |
|
value: 10.953279209883918 |
|
- type: nauc_precision_at_20_std |
|
value: -7.365540710727016 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 33.6150964111514 |
|
- type: nauc_precision_at_3_max |
|
value: 9.216455510763346 |
|
- type: nauc_precision_at_3_std |
|
value: -20.45932513010908 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 31.518755311864705 |
|
- type: nauc_precision_at_5_max |
|
value: 10.019710006442747 |
|
- type: nauc_precision_at_5_std |
|
value: -19.740528698385468 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 12.207155589507542 |
|
- type: nauc_recall_at_1000_max |
|
value: 39.3447783153665 |
|
- type: nauc_recall_at_1000_std |
|
value: 74.60352827999826 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 32.993666280768615 |
|
- type: nauc_recall_at_100_max |
|
value: 16.487188889720816 |
|
- type: nauc_recall_at_100_std |
|
value: 26.828206265371275 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 33.65453771237772 |
|
- type: nauc_recall_at_10_max |
|
value: 10.71869814574723 |
|
- type: nauc_recall_at_10_std |
|
value: -16.27859785753318 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 40.298938486026984 |
|
- type: nauc_recall_at_1_max |
|
value: 8.149499405622326 |
|
- type: nauc_recall_at_1_std |
|
value: -17.09168853307602 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 34.60034971417269 |
|
- type: nauc_recall_at_20_max |
|
value: 12.076871992384788 |
|
- type: nauc_recall_at_20_std |
|
value: -8.224571589978806 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 34.24661417034744 |
|
- type: nauc_recall_at_3_max |
|
value: 9.464103325281997 |
|
- type: nauc_recall_at_3_std |
|
value: -20.329748455626195 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 33.042225241281585 |
|
- type: nauc_recall_at_5_max |
|
value: 10.486814885646142 |
|
- type: nauc_recall_at_5_std |
|
value: -19.7259662900716 |
|
- type: ndcg_at_1 |
|
value: 25.013999999999996 |
|
- type: ndcg_at_10 |
|
value: 44.181 |
|
- type: ndcg_at_100 |
|
value: 49.673 |
|
- type: ndcg_at_1000 |
|
value: 50.705999999999996 |
|
- type: ndcg_at_20 |
|
value: 46.798 |
|
- type: ndcg_at_3 |
|
value: 36.037 |
|
- type: ndcg_at_5 |
|
value: 40.214 |
|
- type: precision_at_1 |
|
value: 25.013999999999996 |
|
- type: precision_at_10 |
|
value: 6.9110000000000005 |
|
- type: precision_at_100 |
|
value: 0.9650000000000001 |
|
- type: precision_at_1000 |
|
value: 0.105 |
|
- type: precision_at_20 |
|
value: 4.004 |
|
- type: precision_at_3 |
|
value: 15.238999999999999 |
|
- type: precision_at_5 |
|
value: 11.264000000000001 |
|
- type: recall_at_1 |
|
value: 24.3 |
|
- type: recall_at_10 |
|
value: 66.06400000000001 |
|
- type: recall_at_100 |
|
value: 91.291 |
|
- type: recall_at_1000 |
|
value: 99.054 |
|
- type: recall_at_20 |
|
value: 76.25699999999999 |
|
- type: recall_at_3 |
|
value: 44.039 |
|
- type: recall_at_5 |
|
value: 54.053 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: en |
|
name: MTEB MTOPDomainClassification (en) |
|
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf |
|
split: test |
|
type: mteb/mtop_domain |
|
metrics: |
|
- type: accuracy |
|
value: 96.35430916552667 |
|
- type: f1 |
|
value: 96.16669219074517 |
|
- type: f1_weighted |
|
value: 96.35506582065435 |
|
- type: main_score |
|
value: 96.35430916552667 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: en |
|
name: MTEB MTOPIntentClassification (en) |
|
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba |
|
split: test |
|
type: mteb/mtop_intent |
|
metrics: |
|
- type: accuracy |
|
value: 80.51527587779297 |
|
- type: f1 |
|
value: 59.350461259612345 |
|
- type: f1_weighted |
|
value: 81.51891267687044 |
|
- type: main_score |
|
value: 80.51527587779297 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: en |
|
name: MTEB MassiveIntentClassification (en) |
|
revision: 4672e20407010da34463acc759c162ca9734bca6 |
|
split: test |
|
type: mteb/amazon_massive_intent |
|
metrics: |
|
- type: accuracy |
|
value: 77.31338264963013 |
|
- type: f1 |
|
value: 75.29547524788576 |
|
- type: f1_weighted |
|
value: 76.26831259224058 |
|
- type: main_score |
|
value: 77.31338264963013 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: en |
|
name: MTEB MassiveScenarioClassification (en) |
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 |
|
split: test |
|
type: mteb/amazon_massive_scenario |
|
metrics: |
|
- type: accuracy |
|
value: 79.97982515131137 |
|
- type: f1 |
|
value: 79.34057805450769 |
|
- type: f1_weighted |
|
value: 79.73023446597212 |
|
- type: main_score |
|
value: 79.97982515131137 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB MedrxivClusteringP2P |
|
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 |
|
split: test |
|
type: mteb/medrxiv-clustering-p2p |
|
metrics: |
|
- type: main_score |
|
value: 38.37635785818304 |
|
- type: v_measure |
|
value: 38.37635785818304 |
|
- type: v_measure_std |
|
value: 1.6943794496059137 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB MedrxivClusteringS2S |
|
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 |
|
split: test |
|
type: mteb/medrxiv-clustering-s2s |
|
metrics: |
|
- type: main_score |
|
value: 37.6711034083755 |
|
- type: v_measure |
|
value: 37.6711034083755 |
|
- type: v_measure_std |
|
value: 1.1408887612104992 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB MindSmallReranking |
|
revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7 |
|
split: test |
|
type: mteb/mind_small |
|
metrics: |
|
- type: main_score |
|
value: 32.20170969306457 |
|
- type: map |
|
value: 32.20170969306457 |
|
- type: mrr |
|
value: 33.41738896071552 |
|
- type: nAUC_map_diff1 |
|
value: 12.077124363492512 |
|
- type: nAUC_map_max |
|
value: -20.336429990396454 |
|
- type: nAUC_map_std |
|
value: 0.10724031251638018 |
|
- type: nAUC_mrr_diff1 |
|
value: 11.405695518900744 |
|
- type: nAUC_mrr_max |
|
value: -15.0727490448132 |
|
- type: nAUC_mrr_std |
|
value: 1.8987958512727106 |
|
task: |
|
type: Reranking |
|
- dataset: |
|
config: default |
|
name: MTEB NFCorpus |
|
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814 |
|
split: test |
|
type: mteb/nfcorpus |
|
metrics: |
|
- type: main_score |
|
value: 40.463 |
|
- type: map_at_1 |
|
value: 6.4990000000000006 |
|
- type: map_at_10 |
|
value: 15.699 |
|
- type: map_at_100 |
|
value: 19.895 |
|
- type: map_at_1000 |
|
value: 21.537 |
|
- type: map_at_20 |
|
value: 17.429 |
|
- type: map_at_3 |
|
value: 11.48 |
|
- type: map_at_5 |
|
value: 13.383999999999999 |
|
- type: mrr_at_1 |
|
value: 52.63157894736842 |
|
- type: mrr_at_10 |
|
value: 61.60401002506265 |
|
- type: mrr_at_100 |
|
value: 62.04336653809741 |
|
- type: mrr_at_1000 |
|
value: 62.07610833363911 |
|
- type: mrr_at_20 |
|
value: 61.88033067968176 |
|
- type: mrr_at_3 |
|
value: 59.44272445820435 |
|
- type: mrr_at_5 |
|
value: 60.89783281733746 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 18.58585974547791 |
|
- type: nauc_map_at_1000_max |
|
value: 30.25465935470905 |
|
- type: nauc_map_at_1000_std |
|
value: 10.987080017051682 |
|
- type: nauc_map_at_100_diff1 |
|
value: 20.02651798573329 |
|
- type: nauc_map_at_100_max |
|
value: 30.108719787095467 |
|
- type: nauc_map_at_100_std |
|
value: 7.882019722247158 |
|
- type: nauc_map_at_10_diff1 |
|
value: 23.02800157136177 |
|
- type: nauc_map_at_10_max |
|
value: 22.8723397741279 |
|
- type: nauc_map_at_10_std |
|
value: -3.762893117006399 |
|
- type: nauc_map_at_1_diff1 |
|
value: 37.94611136294878 |
|
- type: nauc_map_at_1_max |
|
value: 7.297492349938244 |
|
- type: nauc_map_at_1_std |
|
value: -17.813930346562152 |
|
- type: nauc_map_at_20_diff1 |
|
value: 21.981440837881113 |
|
- type: nauc_map_at_20_max |
|
value: 26.759497880383837 |
|
- type: nauc_map_at_20_std |
|
value: 0.18040330674839283 |
|
- type: nauc_map_at_3_diff1 |
|
value: 27.066009968256555 |
|
- type: nauc_map_at_3_max |
|
value: 10.488797596450187 |
|
- type: nauc_map_at_3_std |
|
value: -14.013059830876845 |
|
- type: nauc_map_at_5_diff1 |
|
value: 25.493785001708446 |
|
- type: nauc_map_at_5_max |
|
value: 16.217756878539337 |
|
- type: nauc_map_at_5_std |
|
value: -10.714238788014212 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 28.488264933723528 |
|
- type: nauc_mrr_at_1000_max |
|
value: 45.94151165403325 |
|
- type: nauc_mrr_at_1000_std |
|
value: 25.20231778025588 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 28.4886630218298 |
|
- type: nauc_mrr_at_100_max |
|
value: 45.9702575916014 |
|
- type: nauc_mrr_at_100_std |
|
value: 25.22848732842774 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 28.535257017998294 |
|
- type: nauc_mrr_at_10_max |
|
value: 45.86005605851268 |
|
- type: nauc_mrr_at_10_std |
|
value: 24.81744203643852 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 29.824630548327285 |
|
- type: nauc_mrr_at_1_max |
|
value: 44.19891968145314 |
|
- type: nauc_mrr_at_1_std |
|
value: 23.21413139777098 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 28.54642005356483 |
|
- type: nauc_mrr_at_20_max |
|
value: 46.08926361963997 |
|
- type: nauc_mrr_at_20_std |
|
value: 25.39517294920476 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 28.230929109259407 |
|
- type: nauc_mrr_at_3_max |
|
value: 44.05364599618201 |
|
- type: nauc_mrr_at_3_std |
|
value: 23.828100697992724 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 29.669751924690758 |
|
- type: nauc_mrr_at_5_max |
|
value: 45.36862661497384 |
|
- type: nauc_mrr_at_5_std |
|
value: 23.787166807022505 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 18.515898773404377 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 44.57748675979855 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 29.205899131269604 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 15.88197701276405 |
|
- type: nauc_ndcg_at_100_max |
|
value: 39.62665883972109 |
|
- type: nauc_ndcg_at_100_std |
|
value: 25.186347352251754 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 16.220798038950925 |
|
- type: nauc_ndcg_at_10_max |
|
value: 39.67757337154769 |
|
- type: nauc_ndcg_at_10_std |
|
value: 25.634534917262403 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 31.448775879462932 |
|
- type: nauc_ndcg_at_1_max |
|
value: 44.4256421079556 |
|
- type: nauc_ndcg_at_1_std |
|
value: 23.093987850437355 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 15.417507391228035 |
|
- type: nauc_ndcg_at_20_max |
|
value: 37.52014353976055 |
|
- type: nauc_ndcg_at_20_std |
|
value: 23.880617920537915 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 18.01018470616153 |
|
- type: nauc_ndcg_at_3_max |
|
value: 39.135814950810804 |
|
- type: nauc_ndcg_at_3_std |
|
value: 21.40850285781106 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 18.502338826072368 |
|
- type: nauc_ndcg_at_5_max |
|
value: 40.2043937728194 |
|
- type: nauc_ndcg_at_5_std |
|
value: 22.242499743433424 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -13.648652068964681 |
|
- type: nauc_precision_at_1000_max |
|
value: 3.5821865423513426 |
|
- type: nauc_precision_at_1000_std |
|
value: 35.481456041211274 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -11.342790040792961 |
|
- type: nauc_precision_at_100_max |
|
value: 18.41811151847882 |
|
- type: nauc_precision_at_100_std |
|
value: 44.901842372597336 |
|
- type: nauc_precision_at_10_diff1 |
|
value: -1.9404654865248405 |
|
- type: nauc_precision_at_10_max |
|
value: 40.91955602631143 |
|
- type: nauc_precision_at_10_std |
|
value: 41.38128398646734 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 29.824630548327285 |
|
- type: nauc_precision_at_1_max |
|
value: 44.19891968145314 |
|
- type: nauc_precision_at_1_std |
|
value: 23.21413139777098 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -5.046696327994225 |
|
- type: nauc_precision_at_20_max |
|
value: 33.653422186725386 |
|
- type: nauc_precision_at_20_std |
|
value: 40.97689615511939 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 5.1767717826900785 |
|
- type: nauc_precision_at_3_max |
|
value: 38.01276130261592 |
|
- type: nauc_precision_at_3_std |
|
value: 25.71468883159735 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 3.847065262189492 |
|
- type: nauc_precision_at_5_max |
|
value: 41.00941977122254 |
|
- type: nauc_precision_at_5_std |
|
value: 31.044768384177246 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 7.975632504947066 |
|
- type: nauc_recall_at_1000_max |
|
value: 18.83264064904865 |
|
- type: nauc_recall_at_1000_std |
|
value: 15.023940189337717 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 10.354458867884487 |
|
- type: nauc_recall_at_100_max |
|
value: 27.16900376430975 |
|
- type: nauc_recall_at_100_std |
|
value: 14.160333284050214 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 18.04347857307359 |
|
- type: nauc_recall_at_10_max |
|
value: 19.082544744457774 |
|
- type: nauc_recall_at_10_std |
|
value: -5.107813434157397 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 37.94611136294878 |
|
- type: nauc_recall_at_1_max |
|
value: 7.297492349938244 |
|
- type: nauc_recall_at_1_std |
|
value: -17.813930346562152 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 16.658153504941193 |
|
- type: nauc_recall_at_20_max |
|
value: 23.214261213582382 |
|
- type: nauc_recall_at_20_std |
|
value: -0.6964816170313349 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 23.65600569767465 |
|
- type: nauc_recall_at_3_max |
|
value: 6.543906048065431 |
|
- type: nauc_recall_at_3_std |
|
value: -15.496093666790777 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 22.112315726267077 |
|
- type: nauc_recall_at_5_max |
|
value: 12.258969896916307 |
|
- type: nauc_recall_at_5_std |
|
value: -12.922832334587008 |
|
- type: ndcg_at_1 |
|
value: 50.929 |
|
- type: ndcg_at_10 |
|
value: 40.463 |
|
- type: ndcg_at_100 |
|
value: 36.909 |
|
- type: ndcg_at_1000 |
|
value: 45.617999999999995 |
|
- type: ndcg_at_20 |
|
value: 37.772 |
|
- type: ndcg_at_3 |
|
value: 46.315 |
|
- type: ndcg_at_5 |
|
value: 44.052 |
|
- type: precision_at_1 |
|
value: 52.632 |
|
- type: precision_at_10 |
|
value: 29.814 |
|
- type: precision_at_100 |
|
value: 9.325 |
|
- type: precision_at_1000 |
|
value: 2.236 |
|
- type: precision_at_20 |
|
value: 22.073999999999998 |
|
- type: precision_at_3 |
|
value: 42.931000000000004 |
|
- type: precision_at_5 |
|
value: 37.957 |
|
- type: recall_at_1 |
|
value: 6.4990000000000006 |
|
- type: recall_at_10 |
|
value: 20.232 |
|
- type: recall_at_100 |
|
value: 36.846000000000004 |
|
- type: recall_at_1000 |
|
value: 69.03 |
|
- type: recall_at_20 |
|
value: 24.448 |
|
- type: recall_at_3 |
|
value: 13.258000000000001 |
|
- type: recall_at_5 |
|
value: 16.255 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB NQ |
|
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31 |
|
split: test |
|
type: mteb/nq |
|
metrics: |
|
- type: main_score |
|
value: 70.518 |
|
- type: map_at_1 |
|
value: 46.233999999999995 |
|
- type: map_at_10 |
|
value: 63.519999999999996 |
|
- type: map_at_100 |
|
value: 64.14699999999999 |
|
- type: map_at_1000 |
|
value: 64.154 |
|
- type: map_at_20 |
|
value: 63.975 |
|
- type: map_at_3 |
|
value: 59.797 |
|
- type: map_at_5 |
|
value: 62.226000000000006 |
|
- type: mrr_at_1 |
|
value: 51.76709154113557 |
|
- type: mrr_at_10 |
|
value: 65.79852489470095 |
|
- type: mrr_at_100 |
|
value: 66.19480681115492 |
|
- type: mrr_at_1000 |
|
value: 66.19993656063721 |
|
- type: mrr_at_20 |
|
value: 66.0923632685851 |
|
- type: mrr_at_3 |
|
value: 63.185592893008746 |
|
- type: mrr_at_5 |
|
value: 64.93385477018151 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 43.35077155361084 |
|
- type: nauc_map_at_1000_max |
|
value: 37.282536180921085 |
|
- type: nauc_map_at_1000_std |
|
value: -4.64357984773174 |
|
- type: nauc_map_at_100_diff1 |
|
value: 43.35098576601616 |
|
- type: nauc_map_at_100_max |
|
value: 37.28998747522813 |
|
- type: nauc_map_at_100_std |
|
value: -4.638151362399621 |
|
- type: nauc_map_at_10_diff1 |
|
value: 43.131007214082594 |
|
- type: nauc_map_at_10_max |
|
value: 37.430076712266846 |
|
- type: nauc_map_at_10_std |
|
value: -4.90614475410035 |
|
- type: nauc_map_at_1_diff1 |
|
value: 45.843123692592485 |
|
- type: nauc_map_at_1_max |
|
value: 30.160164681399227 |
|
- type: nauc_map_at_1_std |
|
value: -6.110582951655118 |
|
- type: nauc_map_at_20_diff1 |
|
value: 43.30588135441681 |
|
- type: nauc_map_at_20_max |
|
value: 37.41321766111187 |
|
- type: nauc_map_at_20_std |
|
value: -4.628074353861448 |
|
- type: nauc_map_at_3_diff1 |
|
value: 42.690411835598695 |
|
- type: nauc_map_at_3_max |
|
value: 36.64069333510947 |
|
- type: nauc_map_at_3_std |
|
value: -6.2899609993355545 |
|
- type: nauc_map_at_5_diff1 |
|
value: 42.906814471744134 |
|
- type: nauc_map_at_5_max |
|
value: 37.27599132551781 |
|
- type: nauc_map_at_5_std |
|
value: -5.512203849661435 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 43.77989113830799 |
|
- type: nauc_mrr_at_1000_max |
|
value: 38.01009876981156 |
|
- type: nauc_mrr_at_1000_std |
|
value: -2.0250764367321654 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 43.78071481914773 |
|
- type: nauc_mrr_at_100_max |
|
value: 38.01603112272088 |
|
- type: nauc_mrr_at_100_std |
|
value: -2.019685020907906 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 43.582338882429156 |
|
- type: nauc_mrr_at_10_max |
|
value: 38.19577506300954 |
|
- type: nauc_mrr_at_10_std |
|
value: -2.011905402842086 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 46.544635554669576 |
|
- type: nauc_mrr_at_1_max |
|
value: 33.82720628969995 |
|
- type: nauc_mrr_at_1_std |
|
value: -2.924293824382781 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 43.713682995581614 |
|
- type: nauc_mrr_at_20_max |
|
value: 38.09918392374771 |
|
- type: nauc_mrr_at_20_std |
|
value: -1.9583477023239 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 43.35807398052401 |
|
- type: nauc_mrr_at_3_max |
|
value: 38.39129780935902 |
|
- type: nauc_mrr_at_3_std |
|
value: -2.287791352096624 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 43.4126448419642 |
|
- type: nauc_mrr_at_5_max |
|
value: 38.27294037073721 |
|
- type: nauc_mrr_at_5_std |
|
value: -2.166655666337289 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 43.26202839737687 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 38.493273787010615 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -2.9983001465713524 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 43.25688556190981 |
|
- type: nauc_ndcg_at_100_max |
|
value: 38.68155788574137 |
|
- type: nauc_ndcg_at_100_std |
|
value: -2.8355616191757487 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 42.37071983774907 |
|
- type: nauc_ndcg_at_10_max |
|
value: 39.60970139451164 |
|
- type: nauc_ndcg_at_10_std |
|
value: -3.5877671856177775 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 46.614780156517845 |
|
- type: nauc_ndcg_at_1_max |
|
value: 33.863655999315526 |
|
- type: nauc_ndcg_at_1_std |
|
value: -2.839239881422542 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 42.97845395193472 |
|
- type: nauc_ndcg_at_20_max |
|
value: 39.53053589334249 |
|
- type: nauc_ndcg_at_20_std |
|
value: -2.6507495263904515 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 41.65390869521735 |
|
- type: nauc_ndcg_at_3_max |
|
value: 38.4851846089685 |
|
- type: nauc_ndcg_at_3_std |
|
value: -5.6296606018146 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 41.89640848285409 |
|
- type: nauc_ndcg_at_5_max |
|
value: 39.293659812249615 |
|
- type: nauc_ndcg_at_5_std |
|
value: -4.754462409312945 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -10.848480634403051 |
|
- type: nauc_precision_at_1000_max |
|
value: 1.3436973699935175 |
|
- type: nauc_precision_at_1000_std |
|
value: 19.044141500097957 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -9.018095533261604 |
|
- type: nauc_precision_at_100_max |
|
value: 4.0402155161025695 |
|
- type: nauc_precision_at_100_std |
|
value: 19.492823636364996 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 3.947100636096294 |
|
- type: nauc_precision_at_10_max |
|
value: 20.598641503195907 |
|
- type: nauc_precision_at_10_std |
|
value: 13.522240087840858 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 46.614780156517845 |
|
- type: nauc_precision_at_1_max |
|
value: 33.863655999315526 |
|
- type: nauc_precision_at_1_std |
|
value: -2.839239881422542 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -2.1791072352475336 |
|
- type: nauc_precision_at_20_max |
|
value: 14.03887841842901 |
|
- type: nauc_precision_at_20_std |
|
value: 18.846129471001632 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 21.09092861833543 |
|
- type: nauc_precision_at_3_max |
|
value: 34.122841034361805 |
|
- type: nauc_precision_at_3_std |
|
value: 2.5513201020031064 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 12.181140062410874 |
|
- type: nauc_precision_at_5_max |
|
value: 27.903435474574234 |
|
- type: nauc_precision_at_5_std |
|
value: 7.6589638998570315 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 59.28482230176634 |
|
- type: nauc_recall_at_1000_max |
|
value: 85.47306385133284 |
|
- type: nauc_recall_at_1000_std |
|
value: 76.45740117805659 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 44.31190730138568 |
|
- type: nauc_recall_at_100_max |
|
value: 66.30976579719086 |
|
- type: nauc_recall_at_100_std |
|
value: 30.65274229759539 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 34.885747244334866 |
|
- type: nauc_recall_at_10_max |
|
value: 50.998198327439404 |
|
- type: nauc_recall_at_10_std |
|
value: -2.7025509359838193 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 45.843123692592485 |
|
- type: nauc_recall_at_1_max |
|
value: 30.160164681399227 |
|
- type: nauc_recall_at_1_std |
|
value: -6.110582951655118 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 37.873054394800825 |
|
- type: nauc_recall_at_20_max |
|
value: 59.21039923637266 |
|
- type: nauc_recall_at_20_std |
|
value: 9.352312696050557 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 35.703271085627776 |
|
- type: nauc_recall_at_3_max |
|
value: 41.19400688280121 |
|
- type: nauc_recall_at_3_std |
|
value: -7.9624895195139 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 34.831972383157925 |
|
- type: nauc_recall_at_5_max |
|
value: 44.82018386701478 |
|
- type: nauc_recall_at_5_std |
|
value: -7.046789506164082 |
|
- type: ndcg_at_1 |
|
value: 51.73799999999999 |
|
- type: ndcg_at_10 |
|
value: 70.518 |
|
- type: ndcg_at_100 |
|
value: 72.841 |
|
- type: ndcg_at_1000 |
|
value: 72.99799999999999 |
|
- type: ndcg_at_20 |
|
value: 71.895 |
|
- type: ndcg_at_3 |
|
value: 64.06500000000001 |
|
- type: ndcg_at_5 |
|
value: 67.86999999999999 |
|
- type: precision_at_1 |
|
value: 51.73799999999999 |
|
- type: precision_at_10 |
|
value: 10.698 |
|
- type: precision_at_100 |
|
value: 1.2 |
|
- type: precision_at_1000 |
|
value: 0.121 |
|
- type: precision_at_20 |
|
value: 5.691 |
|
- type: precision_at_3 |
|
value: 28.37 |
|
- type: precision_at_5 |
|
value: 19.363 |
|
- type: recall_at_1 |
|
value: 46.233999999999995 |
|
- type: recall_at_10 |
|
value: 89.062 |
|
- type: recall_at_100 |
|
value: 98.622 |
|
- type: recall_at_1000 |
|
value: 99.754 |
|
- type: recall_at_20 |
|
value: 94.052 |
|
- type: recall_at_3 |
|
value: 72.994 |
|
- type: recall_at_5 |
|
value: 81.525 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB QuoraRetrieval |
|
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259 |
|
split: test |
|
type: mteb/quora |
|
metrics: |
|
- type: main_score |
|
value: 89.885 |
|
- type: map_at_1 |
|
value: 72.379 |
|
- type: map_at_10 |
|
value: 86.455 |
|
- type: map_at_100 |
|
value: 87.087 |
|
- type: map_at_1000 |
|
value: 87.1 |
|
- type: map_at_20 |
|
value: 86.883 |
|
- type: map_at_3 |
|
value: 83.663 |
|
- type: map_at_5 |
|
value: 85.443 |
|
- type: mrr_at_1 |
|
value: 83.27 |
|
- type: mrr_at_10 |
|
value: 89.13586904761888 |
|
- type: mrr_at_100 |
|
value: 89.22177886254626 |
|
- type: mrr_at_1000 |
|
value: 89.22204575963424 |
|
- type: mrr_at_20 |
|
value: 89.20621913458041 |
|
- type: mrr_at_3 |
|
value: 88.37999999999981 |
|
- type: mrr_at_5 |
|
value: 88.89349999999978 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 78.17410401832315 |
|
- type: nauc_map_at_1000_max |
|
value: 33.114749237960986 |
|
- type: nauc_map_at_1000_std |
|
value: -49.79724283243796 |
|
- type: nauc_map_at_100_diff1 |
|
value: 78.17873434671671 |
|
- type: nauc_map_at_100_max |
|
value: 33.101626543573325 |
|
- type: nauc_map_at_100_std |
|
value: -49.82883017160494 |
|
- type: nauc_map_at_10_diff1 |
|
value: 78.28052682172311 |
|
- type: nauc_map_at_10_max |
|
value: 32.626693803188694 |
|
- type: nauc_map_at_10_std |
|
value: -51.941057676350034 |
|
- type: nauc_map_at_1_diff1 |
|
value: 81.06079816824507 |
|
- type: nauc_map_at_1_max |
|
value: 25.638093235123616 |
|
- type: nauc_map_at_1_std |
|
value: -43.230210939240344 |
|
- type: nauc_map_at_20_diff1 |
|
value: 78.22103944842512 |
|
- type: nauc_map_at_20_max |
|
value: 32.94488423505404 |
|
- type: nauc_map_at_20_std |
|
value: -50.69181407781227 |
|
- type: nauc_map_at_3_diff1 |
|
value: 78.75453877967588 |
|
- type: nauc_map_at_3_max |
|
value: 30.645950847243686 |
|
- type: nauc_map_at_3_std |
|
value: -52.983886453956266 |
|
- type: nauc_map_at_5_diff1 |
|
value: 78.44984884302167 |
|
- type: nauc_map_at_5_max |
|
value: 31.69697839442234 |
|
- type: nauc_map_at_5_std |
|
value: -53.21480554718401 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 78.90502271071976 |
|
- type: nauc_mrr_at_1000_max |
|
value: 35.902725888631075 |
|
- type: nauc_mrr_at_1000_std |
|
value: -45.82579843551156 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 78.90552803580407 |
|
- type: nauc_mrr_at_100_max |
|
value: 35.90392790964254 |
|
- type: nauc_mrr_at_100_std |
|
value: -45.82489205475015 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 78.89432223469271 |
|
- type: nauc_mrr_at_10_max |
|
value: 35.86669566861425 |
|
- type: nauc_mrr_at_10_std |
|
value: -46.0616841694464 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 79.53513360034344 |
|
- type: nauc_mrr_at_1_max |
|
value: 35.299514657188006 |
|
- type: nauc_mrr_at_1_std |
|
value: -43.17936948437256 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 78.90707352031835 |
|
- type: nauc_mrr_at_20_max |
|
value: 35.906499072241296 |
|
- type: nauc_mrr_at_20_std |
|
value: -45.8904084451193 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 78.70913062166218 |
|
- type: nauc_mrr_at_3_max |
|
value: 36.16709621132144 |
|
- type: nauc_mrr_at_3_std |
|
value: -46.00948004774822 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 78.91095031555673 |
|
- type: nauc_mrr_at_5_max |
|
value: 36.010878683954566 |
|
- type: nauc_mrr_at_5_std |
|
value: -46.31731368609175 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 78.19132492477127 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 34.5208358892501 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -47.938360906488974 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 78.24549799575261 |
|
- type: nauc_ndcg_at_100_max |
|
value: 34.48869025578818 |
|
- type: nauc_ndcg_at_100_std |
|
value: -48.02996375451253 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 78.15340584208084 |
|
- type: nauc_ndcg_at_10_max |
|
value: 33.5226981818058 |
|
- type: nauc_ndcg_at_10_std |
|
value: -51.690477519601494 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 79.55459365767561 |
|
- type: nauc_ndcg_at_1_max |
|
value: 35.25214101433387 |
|
- type: nauc_ndcg_at_1_std |
|
value: -43.10088819860409 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 78.27277286768546 |
|
- type: nauc_ndcg_at_20_max |
|
value: 33.997104745595564 |
|
- type: nauc_ndcg_at_20_std |
|
value: -50.10549601980995 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 77.68820501917479 |
|
- type: nauc_ndcg_at_3_max |
|
value: 33.00389630941839 |
|
- type: nauc_ndcg_at_3_std |
|
value: -51.00595251236665 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 78.08093149961476 |
|
- type: nauc_ndcg_at_5_max |
|
value: 33.03434664578743 |
|
- type: nauc_ndcg_at_5_std |
|
value: -52.37122386447497 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -44.49830608740945 |
|
- type: nauc_precision_at_1000_max |
|
value: -7.3283280714307395 |
|
- type: nauc_precision_at_1000_std |
|
value: 38.55076692876393 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -44.252675314263904 |
|
- type: nauc_precision_at_100_max |
|
value: -7.038454433556829 |
|
- type: nauc_precision_at_100_std |
|
value: 38.247323997481615 |
|
- type: nauc_precision_at_10_diff1 |
|
value: -40.192852013615216 |
|
- type: nauc_precision_at_10_max |
|
value: -3.7258976649568036 |
|
- type: nauc_precision_at_10_std |
|
value: 25.983458444206182 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 79.55459365767561 |
|
- type: nauc_precision_at_1_max |
|
value: 35.25214101433387 |
|
- type: nauc_precision_at_1_std |
|
value: -43.10088819860409 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -43.020749754821495 |
|
- type: nauc_precision_at_20_max |
|
value: -5.7062060443801075 |
|
- type: nauc_precision_at_20_std |
|
value: 32.8862431943092 |
|
- type: nauc_precision_at_3_diff1 |
|
value: -22.843593386293996 |
|
- type: nauc_precision_at_3_max |
|
value: 4.474275296763041 |
|
- type: nauc_precision_at_3_std |
|
value: 6.119920479600398 |
|
- type: nauc_precision_at_5_diff1 |
|
value: -33.598088334605045 |
|
- type: nauc_precision_at_5_max |
|
value: -0.41505757559350775 |
|
- type: nauc_precision_at_5_std |
|
value: 16.52526817965026 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 28.726073762912847 |
|
- type: nauc_recall_at_1000_max |
|
value: -57.390873015654066 |
|
- type: nauc_recall_at_1000_std |
|
value: 69.71288515421948 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 83.06070133460443 |
|
- type: nauc_recall_at_100_max |
|
value: 33.27991294763942 |
|
- type: nauc_recall_at_100_std |
|
value: -42.785112479889655 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 74.73877865072825 |
|
- type: nauc_recall_at_10_max |
|
value: 27.81410621945221 |
|
- type: nauc_recall_at_10_std |
|
value: -75.85371099008806 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 81.06079816824507 |
|
- type: nauc_recall_at_1_max |
|
value: 25.638093235123616 |
|
- type: nauc_recall_at_1_std |
|
value: -43.230210939240344 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 76.04615040930837 |
|
- type: nauc_recall_at_20_max |
|
value: 27.47173316749929 |
|
- type: nauc_recall_at_20_std |
|
value: -78.29029550423172 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 75.29987903678384 |
|
- type: nauc_recall_at_3_max |
|
value: 27.48543826795177 |
|
- type: nauc_recall_at_3_std |
|
value: -60.91023011356427 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 74.71682412813378 |
|
- type: nauc_recall_at_5_max |
|
value: 27.255092143441562 |
|
- type: nauc_recall_at_5_std |
|
value: -69.03177732393821 |
|
- type: ndcg_at_1 |
|
value: 83.26 |
|
- type: ndcg_at_10 |
|
value: 89.885 |
|
- type: ndcg_at_100 |
|
value: 90.968 |
|
- type: ndcg_at_1000 |
|
value: 91.02799999999999 |
|
- type: ndcg_at_20 |
|
value: 90.52900000000001 |
|
- type: ndcg_at_3 |
|
value: 87.443 |
|
- type: ndcg_at_5 |
|
value: 88.81 |
|
- type: precision_at_1 |
|
value: 83.26 |
|
- type: precision_at_10 |
|
value: 13.581999999999999 |
|
- type: precision_at_100 |
|
value: 1.541 |
|
- type: precision_at_1000 |
|
value: 0.157 |
|
- type: precision_at_20 |
|
value: 7.210999999999999 |
|
- type: precision_at_3 |
|
value: 38.323 |
|
- type: precision_at_5 |
|
value: 25.069999999999997 |
|
- type: recall_at_1 |
|
value: 72.379 |
|
- type: recall_at_10 |
|
value: 96.261 |
|
- type: recall_at_100 |
|
value: 99.779 |
|
- type: recall_at_1000 |
|
value: 99.996 |
|
- type: recall_at_20 |
|
value: 98.301 |
|
- type: recall_at_3 |
|
value: 89.101 |
|
- type: recall_at_5 |
|
value: 93.11500000000001 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB RedditClustering |
|
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb |
|
split: test |
|
type: mteb/reddit-clustering |
|
metrics: |
|
- type: main_score |
|
value: 61.87769077476204 |
|
- type: v_measure |
|
value: 61.87769077476204 |
|
- type: v_measure_std |
|
value: 5.290405218730049 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB RedditClusteringP2P |
|
revision: 385e3cb46b4cfa89021f56c4380204149d0efe33 |
|
split: test |
|
type: mteb/reddit-clustering-p2p |
|
metrics: |
|
- type: main_score |
|
value: 68.29553057563754 |
|
- type: v_measure |
|
value: 68.29553057563754 |
|
- type: v_measure_std |
|
value: 13.019229253711732 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB SCIDOCS |
|
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88 |
|
split: test |
|
type: mteb/scidocs |
|
metrics: |
|
- type: main_score |
|
value: 20.86 |
|
- type: map_at_1 |
|
value: 4.843 |
|
- type: map_at_10 |
|
value: 12.457 |
|
- type: map_at_100 |
|
value: 14.648 |
|
- type: map_at_1000 |
|
value: 14.965 |
|
- type: map_at_20 |
|
value: 13.596 |
|
- type: map_at_3 |
|
value: 8.776 |
|
- type: map_at_5 |
|
value: 10.528 |
|
- type: mrr_at_1 |
|
value: 23.799999999999997 |
|
- type: mrr_at_10 |
|
value: 34.93765873015872 |
|
- type: mrr_at_100 |
|
value: 36.054095036751825 |
|
- type: mrr_at_1000 |
|
value: 36.10871797082569 |
|
- type: mrr_at_20 |
|
value: 35.57880859465608 |
|
- type: mrr_at_3 |
|
value: 31.54999999999999 |
|
- type: mrr_at_5 |
|
value: 33.53999999999998 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 16.889540490911525 |
|
- type: nauc_map_at_1000_max |
|
value: 25.726340275186143 |
|
- type: nauc_map_at_1000_std |
|
value: 9.926911665196988 |
|
- type: nauc_map_at_100_diff1 |
|
value: 16.889355521248202 |
|
- type: nauc_map_at_100_max |
|
value: 25.628741550328126 |
|
- type: nauc_map_at_100_std |
|
value: 9.637062917997012 |
|
- type: nauc_map_at_10_diff1 |
|
value: 16.972521218507854 |
|
- type: nauc_map_at_10_max |
|
value: 24.810172126870363 |
|
- type: nauc_map_at_10_std |
|
value: 7.09295422867669 |
|
- type: nauc_map_at_1_diff1 |
|
value: 24.9292922418417 |
|
- type: nauc_map_at_1_max |
|
value: 15.49253311874767 |
|
- type: nauc_map_at_1_std |
|
value: -0.4754734108717385 |
|
- type: nauc_map_at_20_diff1 |
|
value: 16.945564955989113 |
|
- type: nauc_map_at_20_max |
|
value: 25.197327599885362 |
|
- type: nauc_map_at_20_std |
|
value: 7.972256233219635 |
|
- type: nauc_map_at_3_diff1 |
|
value: 19.503723922705067 |
|
- type: nauc_map_at_3_max |
|
value: 20.795879090480057 |
|
- type: nauc_map_at_3_std |
|
value: 1.5828913591118658 |
|
- type: nauc_map_at_5_diff1 |
|
value: 19.80474780705204 |
|
- type: nauc_map_at_5_max |
|
value: 24.040173591299723 |
|
- type: nauc_map_at_5_std |
|
value: 4.153642430396917 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 21.80300741603344 |
|
- type: nauc_mrr_at_1000_max |
|
value: 19.98123409846586 |
|
- type: nauc_mrr_at_1000_std |
|
value: 3.6325335777371377 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 21.804966803578946 |
|
- type: nauc_mrr_at_100_max |
|
value: 19.9965104601956 |
|
- type: nauc_mrr_at_100_std |
|
value: 3.6713772865070107 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 21.66109150475663 |
|
- type: nauc_mrr_at_10_max |
|
value: 19.873876575424404 |
|
- type: nauc_mrr_at_10_std |
|
value: 3.3387503298795584 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 24.868548821073084 |
|
- type: nauc_mrr_at_1_max |
|
value: 16.189915011439044 |
|
- type: nauc_mrr_at_1_std |
|
value: -0.17692171251799987 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 21.677427533247375 |
|
- type: nauc_mrr_at_20_max |
|
value: 19.967193157614872 |
|
- type: nauc_mrr_at_20_std |
|
value: 3.639825799332009 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 21.681117207511825 |
|
- type: nauc_mrr_at_3_max |
|
value: 19.132660363303295 |
|
- type: nauc_mrr_at_3_std |
|
value: 1.6613642176263752 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 21.833332207271884 |
|
- type: nauc_mrr_at_5_max |
|
value: 19.926480855266213 |
|
- type: nauc_mrr_at_5_std |
|
value: 2.901801717093585 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 16.92599483752314 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 27.126582080942814 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 16.638448489514683 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 16.96586885959473 |
|
- type: nauc_ndcg_at_100_max |
|
value: 26.675878724175046 |
|
- type: nauc_ndcg_at_100_std |
|
value: 15.369335585614245 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 16.59779893225997 |
|
- type: nauc_ndcg_at_10_max |
|
value: 24.865338966132818 |
|
- type: nauc_ndcg_at_10_std |
|
value: 8.934209252745864 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 24.868548821073084 |
|
- type: nauc_ndcg_at_1_max |
|
value: 16.189915011439044 |
|
- type: nauc_ndcg_at_1_std |
|
value: -0.17692171251799987 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 16.647406628819976 |
|
- type: nauc_ndcg_at_20_max |
|
value: 25.64488140369063 |
|
- type: nauc_ndcg_at_20_std |
|
value: 10.587157641309098 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 19.093302254257377 |
|
- type: nauc_ndcg_at_3_max |
|
value: 21.33725971448413 |
|
- type: nauc_ndcg_at_3_std |
|
value: 2.549021710462978 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 19.495189389728836 |
|
- type: nauc_ndcg_at_5_max |
|
value: 24.21965138651894 |
|
- type: nauc_ndcg_at_5_std |
|
value: 5.549408503444251 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: 7.4232833098081565 |
|
- type: nauc_precision_at_1000_max |
|
value: 25.24619675919913 |
|
- type: nauc_precision_at_1000_std |
|
value: 32.79744946411614 |
|
- type: nauc_precision_at_100_diff1 |
|
value: 10.550449529674747 |
|
- type: nauc_precision_at_100_max |
|
value: 25.652112631579726 |
|
- type: nauc_precision_at_100_std |
|
value: 26.65722909800614 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 11.195653785882708 |
|
- type: nauc_precision_at_10_max |
|
value: 26.469986306854977 |
|
- type: nauc_precision_at_10_std |
|
value: 14.05089697514966 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 24.868548821073084 |
|
- type: nauc_precision_at_1_max |
|
value: 16.189915011439044 |
|
- type: nauc_precision_at_1_std |
|
value: -0.17692171251799987 |
|
- type: nauc_precision_at_20_diff1 |
|
value: 11.16738184991032 |
|
- type: nauc_precision_at_20_max |
|
value: 26.53741675130711 |
|
- type: nauc_precision_at_20_std |
|
value: 16.250110771034542 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 16.917872510926284 |
|
- type: nauc_precision_at_3_max |
|
value: 23.22094310791854 |
|
- type: nauc_precision_at_3_std |
|
value: 3.9255078517383906 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 16.898056883587824 |
|
- type: nauc_precision_at_5_max |
|
value: 27.39457295203392 |
|
- type: nauc_precision_at_5_std |
|
value: 8.924759582566171 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 7.516072705946253 |
|
- type: nauc_recall_at_1000_max |
|
value: 25.001682297424594 |
|
- type: nauc_recall_at_1000_std |
|
value: 33.86296283879721 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 10.435705067998168 |
|
- type: nauc_recall_at_100_max |
|
value: 25.31622603650995 |
|
- type: nauc_recall_at_100_std |
|
value: 26.758897185352097 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 11.110953419292343 |
|
- type: nauc_recall_at_10_max |
|
value: 25.970593144433085 |
|
- type: nauc_recall_at_10_std |
|
value: 13.92252981022314 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 24.9292922418417 |
|
- type: nauc_recall_at_1_max |
|
value: 15.49253311874767 |
|
- type: nauc_recall_at_1_std |
|
value: -0.4754734108717385 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 11.050515317424548 |
|
- type: nauc_recall_at_20_max |
|
value: 26.068866115743134 |
|
- type: nauc_recall_at_20_std |
|
value: 16.13787216291987 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 17.013383740580203 |
|
- type: nauc_recall_at_3_max |
|
value: 22.49105285578937 |
|
- type: nauc_recall_at_3_std |
|
value: 3.5741090487413687 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 16.973540662242602 |
|
- type: nauc_recall_at_5_max |
|
value: 26.78087164318061 |
|
- type: nauc_recall_at_5_std |
|
value: 8.68040862354009 |
|
- type: ndcg_at_1 |
|
value: 23.799999999999997 |
|
- type: ndcg_at_10 |
|
value: 20.86 |
|
- type: ndcg_at_100 |
|
value: 29.145 |
|
- type: ndcg_at_1000 |
|
value: 34.518 |
|
- type: ndcg_at_20 |
|
value: 23.892 |
|
- type: ndcg_at_3 |
|
value: 19.541 |
|
- type: ndcg_at_5 |
|
value: 17.166999999999998 |
|
- type: precision_at_1 |
|
value: 23.799999999999997 |
|
- type: precision_at_10 |
|
value: 10.9 |
|
- type: precision_at_100 |
|
value: 2.281 |
|
- type: precision_at_1000 |
|
value: 0.357 |
|
- type: precision_at_20 |
|
value: 7.21 |
|
- type: precision_at_3 |
|
value: 18.3 |
|
- type: precision_at_5 |
|
value: 15.120000000000001 |
|
- type: recall_at_1 |
|
value: 4.843 |
|
- type: recall_at_10 |
|
value: 22.12 |
|
- type: recall_at_100 |
|
value: 46.257 |
|
- type: recall_at_1000 |
|
value: 72.382 |
|
- type: recall_at_20 |
|
value: 29.253 |
|
- type: recall_at_3 |
|
value: 11.158 |
|
- type: recall_at_5 |
|
value: 15.347 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB SICK-R |
|
revision: 20a6d6f312dd54037fe07a32d58e5e168867909d |
|
split: test |
|
type: mteb/sickr-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 85.65467419133347 |
|
- type: cosine_spearman |
|
value: 81.88046945336663 |
|
- type: euclidean_pearson |
|
value: 82.82887106181879 |
|
- type: euclidean_spearman |
|
value: 81.88047605481775 |
|
- type: main_score |
|
value: 81.88046945336663 |
|
- type: manhattan_pearson |
|
value: 82.7839019603756 |
|
- type: manhattan_spearman |
|
value: 81.83505450284663 |
|
- type: pearson |
|
value: 85.65467419133347 |
|
- type: spearman |
|
value: 81.88046945336663 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STS12 |
|
revision: a0d554a64d88156834ff5ae9920b964011b16384 |
|
split: test |
|
type: mteb/sts12-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 85.8979872663498 |
|
- type: cosine_spearman |
|
value: 78.63991285161867 |
|
- type: euclidean_pearson |
|
value: 81.20243176386163 |
|
- type: euclidean_spearman |
|
value: 78.64021127260493 |
|
- type: main_score |
|
value: 78.63991285161867 |
|
- type: manhattan_pearson |
|
value: 81.58673652635328 |
|
- type: manhattan_spearman |
|
value: 79.03930665482164 |
|
- type: pearson |
|
value: 85.8979872663498 |
|
- type: spearman |
|
value: 78.63991285161867 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STS13 |
|
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca |
|
split: test |
|
type: mteb/sts13-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 87.10598414063074 |
|
- type: cosine_spearman |
|
value: 87.12110799581852 |
|
- type: euclidean_pearson |
|
value: 86.52284239759508 |
|
- type: euclidean_spearman |
|
value: 87.12110799581852 |
|
- type: main_score |
|
value: 87.12110799581852 |
|
- type: manhattan_pearson |
|
value: 86.61105352996736 |
|
- type: manhattan_spearman |
|
value: 87.34100209521596 |
|
- type: pearson |
|
value: 87.10598414063074 |
|
- type: spearman |
|
value: 87.12110799581852 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STS14 |
|
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 |
|
split: test |
|
type: mteb/sts14-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 85.66540041627184 |
|
- type: cosine_spearman |
|
value: 83.55263671417923 |
|
- type: euclidean_pearson |
|
value: 84.2332532036626 |
|
- type: euclidean_spearman |
|
value: 83.55264421653584 |
|
- type: main_score |
|
value: 83.55263671417923 |
|
- type: manhattan_pearson |
|
value: 84.14418954784165 |
|
- type: manhattan_spearman |
|
value: 83.58193360267302 |
|
- type: pearson |
|
value: 85.66540041627184 |
|
- type: spearman |
|
value: 83.55263671417923 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STS15 |
|
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 |
|
split: test |
|
type: mteb/sts15-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 89.83404956912175 |
|
- type: cosine_spearman |
|
value: 90.09569633194636 |
|
- type: euclidean_pearson |
|
value: 89.31121256629982 |
|
- type: euclidean_spearman |
|
value: 90.09569632193572 |
|
- type: main_score |
|
value: 90.09569633194636 |
|
- type: manhattan_pearson |
|
value: 89.30064909066367 |
|
- type: manhattan_spearman |
|
value: 90.20232732019451 |
|
- type: pearson |
|
value: 89.83404956912175 |
|
- type: spearman |
|
value: 90.09569633194636 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STS16 |
|
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 |
|
split: test |
|
type: mteb/sts16-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 86.27894370732598 |
|
- type: cosine_spearman |
|
value: 87.22000226558832 |
|
- type: euclidean_pearson |
|
value: 85.92822715155758 |
|
- type: euclidean_spearman |
|
value: 87.22000226558832 |
|
- type: main_score |
|
value: 87.22000226558832 |
|
- type: manhattan_pearson |
|
value: 85.9498561399522 |
|
- type: manhattan_spearman |
|
value: 87.28837300894288 |
|
- type: pearson |
|
value: 86.27894370732598 |
|
- type: spearman |
|
value: 87.22000226558832 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: en-en |
|
name: MTEB STS17 (en-en) |
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c |
|
split: test |
|
type: mteb/sts17-crosslingual-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 91.60185356782324 |
|
- type: cosine_spearman |
|
value: 91.43471625912765 |
|
- type: euclidean_pearson |
|
value: 91.52529087606635 |
|
- type: euclidean_spearman |
|
value: 91.43471625912765 |
|
- type: main_score |
|
value: 91.43471625912765 |
|
- type: manhattan_pearson |
|
value: 91.34917173506308 |
|
- type: manhattan_spearman |
|
value: 91.2112665439884 |
|
- type: pearson |
|
value: 91.60185356782324 |
|
- type: spearman |
|
value: 91.43471625912765 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: en |
|
name: MTEB STS22 (en) |
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 |
|
split: test |
|
type: mteb/sts22-crosslingual-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 68.735098373629 |
|
- type: cosine_spearman |
|
value: 67.76156085991387 |
|
- type: euclidean_pearson |
|
value: 68.38053954511516 |
|
- type: euclidean_spearman |
|
value: 67.76156085991387 |
|
- type: main_score |
|
value: 67.76156085991387 |
|
- type: manhattan_pearson |
|
value: 68.4533080173714 |
|
- type: manhattan_spearman |
|
value: 67.76676959397871 |
|
- type: pearson |
|
value: 68.735098373629 |
|
- type: spearman |
|
value: 67.76156085991387 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB STSBenchmark |
|
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 |
|
split: test |
|
type: mteb/stsbenchmark-sts |
|
metrics: |
|
- type: cosine_pearson |
|
value: 87.63236624274985 |
|
- type: cosine_spearman |
|
value: 88.27561759951514 |
|
- type: euclidean_pearson |
|
value: 87.61137355553329 |
|
- type: euclidean_spearman |
|
value: 88.27561759951514 |
|
- type: main_score |
|
value: 88.27561759951514 |
|
- type: manhattan_pearson |
|
value: 87.63505381780153 |
|
- type: manhattan_spearman |
|
value: 88.41268943146845 |
|
- type: pearson |
|
value: 87.63236624274985 |
|
- type: spearman |
|
value: 88.27561759951514 |
|
task: |
|
type: STS |
|
- dataset: |
|
config: default |
|
name: MTEB SciDocsRR |
|
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab |
|
split: test |
|
type: mteb/scidocs-reranking |
|
metrics: |
|
- type: main_score |
|
value: 85.16412972900244 |
|
- type: map |
|
value: 85.16412972900244 |
|
- type: mrr |
|
value: 96.15786628041529 |
|
- type: nAUC_map_diff1 |
|
value: -1.5068306084088756 |
|
- type: nAUC_map_max |
|
value: 48.81296049442589 |
|
- type: nAUC_map_std |
|
value: 65.0187132933644 |
|
- type: nAUC_mrr_diff1 |
|
value: 44.22872564939586 |
|
- type: nAUC_mrr_max |
|
value: 85.19719227096536 |
|
- type: nAUC_mrr_std |
|
value: 79.62669870868876 |
|
task: |
|
type: Reranking |
|
- dataset: |
|
config: default |
|
name: MTEB SciFact |
|
revision: 0228b52cf27578f30900b9e5271d331663a030d7 |
|
split: test |
|
type: mteb/scifact |
|
metrics: |
|
- type: main_score |
|
value: 77.377 |
|
- type: map_at_1 |
|
value: 62.217 |
|
- type: map_at_10 |
|
value: 73.115 |
|
- type: map_at_100 |
|
value: 73.63499999999999 |
|
- type: map_at_1000 |
|
value: 73.644 |
|
- type: map_at_20 |
|
value: 73.528 |
|
- type: map_at_3 |
|
value: 70.62 |
|
- type: map_at_5 |
|
value: 72.16 |
|
- type: mrr_at_1 |
|
value: 65.0 |
|
- type: mrr_at_10 |
|
value: 74.05542328042326 |
|
- type: mrr_at_100 |
|
value: 74.46295785951277 |
|
- type: mrr_at_1000 |
|
value: 74.47168088874803 |
|
- type: mrr_at_20 |
|
value: 74.35632423132421 |
|
- type: mrr_at_3 |
|
value: 72.55555555555556 |
|
- type: mrr_at_5 |
|
value: 73.38888888888887 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 72.31754010838618 |
|
- type: nauc_map_at_1000_max |
|
value: 60.59518156728312 |
|
- type: nauc_map_at_1000_std |
|
value: -3.601504782295705 |
|
- type: nauc_map_at_100_diff1 |
|
value: 72.32057771059107 |
|
- type: nauc_map_at_100_max |
|
value: 60.60481879601873 |
|
- type: nauc_map_at_100_std |
|
value: -3.6030430073837167 |
|
- type: nauc_map_at_10_diff1 |
|
value: 72.15009895006031 |
|
- type: nauc_map_at_10_max |
|
value: 60.49958178006608 |
|
- type: nauc_map_at_10_std |
|
value: -4.305475753173601 |
|
- type: nauc_map_at_1_diff1 |
|
value: 76.32919417574946 |
|
- type: nauc_map_at_1_max |
|
value: 54.77358788281581 |
|
- type: nauc_map_at_1_std |
|
value: -9.773898055794557 |
|
- type: nauc_map_at_20_diff1 |
|
value: 72.15740734516393 |
|
- type: nauc_map_at_20_max |
|
value: 60.61318821265446 |
|
- type: nauc_map_at_20_std |
|
value: -3.6016854193910803 |
|
- type: nauc_map_at_3_diff1 |
|
value: 72.07435404889445 |
|
- type: nauc_map_at_3_max |
|
value: 56.93970890047747 |
|
- type: nauc_map_at_3_std |
|
value: -8.697324220121793 |
|
- type: nauc_map_at_5_diff1 |
|
value: 72.42599960854554 |
|
- type: nauc_map_at_5_max |
|
value: 60.12535137001906 |
|
- type: nauc_map_at_5_std |
|
value: -4.437892354037166 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 72.75103842052889 |
|
- type: nauc_mrr_at_1000_max |
|
value: 62.72341811793062 |
|
- type: nauc_mrr_at_1000_std |
|
value: -0.7759889099766357 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 72.75396801842608 |
|
- type: nauc_mrr_at_100_max |
|
value: 62.73241247525427 |
|
- type: nauc_mrr_at_100_std |
|
value: -0.7786866224468205 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 72.5942754009733 |
|
- type: nauc_mrr_at_10_max |
|
value: 62.895066542256664 |
|
- type: nauc_mrr_at_10_std |
|
value: -0.9018200301159104 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 77.63311362465076 |
|
- type: nauc_mrr_at_1_max |
|
value: 62.42059294219759 |
|
- type: nauc_mrr_at_1_std |
|
value: -1.3182520953698476 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 72.58522880943326 |
|
- type: nauc_mrr_at_20_max |
|
value: 62.73063935403417 |
|
- type: nauc_mrr_at_20_std |
|
value: -0.7910003366564456 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: 72.70751722757556 |
|
- type: nauc_mrr_at_3_max |
|
value: 62.38218933726893 |
|
- type: nauc_mrr_at_3_std |
|
value: -1.7126398606397155 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 72.57550761997256 |
|
- type: nauc_mrr_at_5_max |
|
value: 62.70945847818393 |
|
- type: nauc_mrr_at_5_std |
|
value: -0.30886077098332143 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 71.6036105202873 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 61.99911514670603 |
|
- type: nauc_ndcg_at_1000_std |
|
value: -2.050470755577302 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 71.70345074974581 |
|
- type: nauc_ndcg_at_100_max |
|
value: 62.374525611545714 |
|
- type: nauc_ndcg_at_100_std |
|
value: -1.922345118135967 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 70.40027928749286 |
|
- type: nauc_ndcg_at_10_max |
|
value: 62.36595526966657 |
|
- type: nauc_ndcg_at_10_std |
|
value: -3.862278807246422 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 77.63311362465076 |
|
- type: nauc_ndcg_at_1_max |
|
value: 62.42059294219759 |
|
- type: nauc_ndcg_at_1_std |
|
value: -1.3182520953698476 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 70.21719291674641 |
|
- type: nauc_ndcg_at_20_max |
|
value: 62.356711760569404 |
|
- type: nauc_ndcg_at_20_std |
|
value: -2.240360396463778 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 70.72483260039468 |
|
- type: nauc_ndcg_at_3_max |
|
value: 59.465348910073445 |
|
- type: nauc_ndcg_at_3_std |
|
value: -6.379991854598364 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 70.91296936044013 |
|
- type: nauc_ndcg_at_5_max |
|
value: 61.5986283773017 |
|
- type: nauc_ndcg_at_5_std |
|
value: -3.064893399445654 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -25.399544557043956 |
|
- type: nauc_precision_at_1000_max |
|
value: 17.838641101318792 |
|
- type: nauc_precision_at_1000_std |
|
value: 54.531382221213185 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -15.78139909072201 |
|
- type: nauc_precision_at_100_max |
|
value: 24.183801380755472 |
|
- type: nauc_precision_at_100_std |
|
value: 50.39320972640593 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 4.1199958514831 |
|
- type: nauc_precision_at_10_max |
|
value: 37.922630159717926 |
|
- type: nauc_precision_at_10_std |
|
value: 32.94959551960178 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 77.63311362465076 |
|
- type: nauc_precision_at_1_max |
|
value: 62.42059294219759 |
|
- type: nauc_precision_at_1_std |
|
value: -1.3182520953698476 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -8.926047159112303 |
|
- type: nauc_precision_at_20_max |
|
value: 29.369903951067172 |
|
- type: nauc_precision_at_20_std |
|
value: 41.793379234725904 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 36.51209832895358 |
|
- type: nauc_precision_at_3_max |
|
value: 51.07398992745159 |
|
- type: nauc_precision_at_3_std |
|
value: 13.831661495933623 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 19.526084047733807 |
|
- type: nauc_precision_at_5_max |
|
value: 46.67537950098273 |
|
- type: nauc_precision_at_5_std |
|
value: 31.06747779005178 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: .nan |
|
- type: nauc_recall_at_1000_max |
|
value: .nan |
|
- type: nauc_recall_at_1000_std |
|
value: .nan |
|
- type: nauc_recall_at_100_diff1 |
|
value: 77.45764972655711 |
|
- type: nauc_recall_at_100_max |
|
value: 85.69427771108462 |
|
- type: nauc_recall_at_100_std |
|
value: 10.277444311057575 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 59.14685653975806 |
|
- type: nauc_recall_at_10_max |
|
value: 67.75739956082005 |
|
- type: nauc_recall_at_10_std |
|
value: -12.22251646924215 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 76.32919417574946 |
|
- type: nauc_recall_at_1_max |
|
value: 54.77358788281581 |
|
- type: nauc_recall_at_1_std |
|
value: -9.773898055794557 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 49.90908644159423 |
|
- type: nauc_recall_at_20_max |
|
value: 70.55383556931541 |
|
- type: nauc_recall_at_20_std |
|
value: -3.7004275394368182 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 64.34183819693267 |
|
- type: nauc_recall_at_3_max |
|
value: 55.782789721196444 |
|
- type: nauc_recall_at_3_std |
|
value: -13.886583892174077 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 63.467364565196135 |
|
- type: nauc_recall_at_5_max |
|
value: 62.51562390716315 |
|
- type: nauc_recall_at_5_std |
|
value: -4.715416491952255 |
|
- type: ndcg_at_1 |
|
value: 65.0 |
|
- type: ndcg_at_10 |
|
value: 77.377 |
|
- type: ndcg_at_100 |
|
value: 79.36 |
|
- type: ndcg_at_1000 |
|
value: 79.644 |
|
- type: ndcg_at_20 |
|
value: 78.61200000000001 |
|
- type: ndcg_at_3 |
|
value: 73.624 |
|
- type: ndcg_at_5 |
|
value: 75.458 |
|
- type: precision_at_1 |
|
value: 65.0 |
|
- type: precision_at_10 |
|
value: 10.100000000000001 |
|
- type: precision_at_100 |
|
value: 1.107 |
|
- type: precision_at_1000 |
|
value: 0.11299999999999999 |
|
- type: precision_at_20 |
|
value: 5.333 |
|
- type: precision_at_3 |
|
value: 28.778 |
|
- type: precision_at_5 |
|
value: 18.8 |
|
- type: recall_at_1 |
|
value: 62.217 |
|
- type: recall_at_10 |
|
value: 89.156 |
|
- type: recall_at_100 |
|
value: 97.667 |
|
- type: recall_at_1000 |
|
value: 100.0 |
|
- type: recall_at_20 |
|
value: 93.667 |
|
- type: recall_at_3 |
|
value: 79.183 |
|
- type: recall_at_5 |
|
value: 83.672 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB SprintDuplicateQuestions |
|
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 |
|
split: test |
|
type: mteb/sprintduplicatequestions-pairclassification |
|
metrics: |
|
- type: cos_sim_accuracy |
|
value: 99.8019801980198 |
|
- type: cos_sim_ap |
|
value: 95.25139396923107 |
|
- type: dot_sim_accuracy |
|
value: 0.998019801980198 |
|
- type: dot_sim_ap |
|
value: 95.25139396923107 |
|
- type: max_accuracy |
|
value: 99.8019801980198 |
|
- type: max_ap |
|
value: 95.43878917155146 |
|
- type: max_f1 |
|
value: 90.0398406374502 |
|
- type: main_score |
|
value: 95.43878917155146 |
|
task: |
|
type: PairClassification |
|
- dataset: |
|
config: default |
|
name: MTEB StackExchangeClustering |
|
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 |
|
split: test |
|
type: mteb/stackexchange-clustering |
|
metrics: |
|
- type: main_score |
|
value: 75.91311883393888 |
|
- type: v_measure |
|
value: 75.91311883393888 |
|
- type: v_measure_std |
|
value: 3.286198100593212 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB StackExchangeClusteringP2P |
|
revision: 815ca46b2622cec33ccafc3735d572c266efdb44 |
|
split: test |
|
type: mteb/stackexchange-clustering-p2p |
|
metrics: |
|
- type: main_score |
|
value: 47.171049215275694 |
|
- type: v_measure |
|
value: 47.171049215275694 |
|
- type: v_measure_std |
|
value: 1.6586563477857534 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB StackOverflowDupQuestions |
|
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 |
|
split: test |
|
type: mteb/stackoverflowdupquestions-reranking |
|
metrics: |
|
- type: main_score |
|
value: 54.15041943470163 |
|
- type: map |
|
value: 54.15041943470163 |
|
- type: mrr |
|
value: 55.03112798149563 |
|
- type: nAUC_map_diff1 |
|
value: 39.50144777017669 |
|
- type: nAUC_map_max |
|
value: 14.024793174481395 |
|
- type: nAUC_map_std |
|
value: 6.533766502190137 |
|
- type: nAUC_mrr_diff1 |
|
value: 39.72560651870919 |
|
- type: nAUC_mrr_max |
|
value: 14.807887392821616 |
|
- type: nAUC_mrr_std |
|
value: 7.270272018791473 |
|
task: |
|
type: Reranking |
|
- dataset: |
|
config: default |
|
name: MTEB SummEval |
|
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c |
|
split: test |
|
type: mteb/summeval |
|
metrics: |
|
- type: cosine_pearson |
|
value: 31.57515534177576 |
|
- type: cosine_spearman |
|
value: 31.415247541636194 |
|
- type: dot_pearson |
|
value: 31.575170220667488 |
|
- type: dot_spearman |
|
value: 31.415247541636194 |
|
- type: main_score |
|
value: 31.415247541636194 |
|
- type: pearson |
|
value: 31.57515534177576 |
|
- type: spearman |
|
value: 31.415247541636194 |
|
task: |
|
type: Summarization |
|
- dataset: |
|
config: default |
|
name: MTEB TRECCOVID |
|
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e |
|
split: test |
|
type: mteb/trec-covid |
|
metrics: |
|
- type: main_score |
|
value: 83.67999999999999 |
|
- type: map_at_1 |
|
value: 0.243 |
|
- type: map_at_10 |
|
value: 2.167 |
|
- type: map_at_100 |
|
value: 13.750000000000002 |
|
- type: map_at_1000 |
|
value: 33.537 |
|
- type: map_at_20 |
|
value: 4.047 |
|
- type: map_at_3 |
|
value: 0.694 |
|
- type: map_at_5 |
|
value: 1.141 |
|
- type: mrr_at_1 |
|
value: 94.0 |
|
- type: mrr_at_10 |
|
value: 97.0 |
|
- type: mrr_at_100 |
|
value: 97.0 |
|
- type: mrr_at_1000 |
|
value: 97.0 |
|
- type: mrr_at_20 |
|
value: 97.0 |
|
- type: mrr_at_3 |
|
value: 97.0 |
|
- type: mrr_at_5 |
|
value: 97.0 |
|
- type: nauc_map_at_1000_diff1 |
|
value: 4.890354942949616 |
|
- type: nauc_map_at_1000_max |
|
value: 29.279958833328408 |
|
- type: nauc_map_at_1000_std |
|
value: 77.2405348865942 |
|
- type: nauc_map_at_100_diff1 |
|
value: 31.835069149380868 |
|
- type: nauc_map_at_100_max |
|
value: 14.523120708509271 |
|
- type: nauc_map_at_100_std |
|
value: 39.682149025882886 |
|
- type: nauc_map_at_10_diff1 |
|
value: 43.45574726953753 |
|
- type: nauc_map_at_10_max |
|
value: -2.9143965183484246 |
|
- type: nauc_map_at_10_std |
|
value: 2.8052301238653756 |
|
- type: nauc_map_at_1_diff1 |
|
value: 26.134637426782753 |
|
- type: nauc_map_at_1_max |
|
value: -3.108959317897608 |
|
- type: nauc_map_at_1_std |
|
value: -5.781123480253076 |
|
- type: nauc_map_at_20_diff1 |
|
value: 45.735224340099236 |
|
- type: nauc_map_at_20_max |
|
value: -1.099022132339708 |
|
- type: nauc_map_at_20_std |
|
value: 7.6378546013151905 |
|
- type: nauc_map_at_3_diff1 |
|
value: 35.70649469812688 |
|
- type: nauc_map_at_3_max |
|
value: -9.710213033638656 |
|
- type: nauc_map_at_3_std |
|
value: -3.6668161574691056 |
|
- type: nauc_map_at_5_diff1 |
|
value: 37.6110093992781 |
|
- type: nauc_map_at_5_max |
|
value: -8.6295080300384 |
|
- type: nauc_map_at_5_std |
|
value: -3.2709712613287145 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_1000_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_1000_std |
|
value: 69.88795518207282 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_100_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_100_std |
|
value: 69.88795518207282 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_10_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_10_std |
|
value: 69.88795518207282 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: -22.36227824463097 |
|
- type: nauc_mrr_at_1_max |
|
value: 63.741051976346206 |
|
- type: nauc_mrr_at_1_std |
|
value: 69.88795518207289 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_20_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_20_std |
|
value: 69.88795518207282 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_3_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_3_std |
|
value: 69.88795518207282 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: -22.362278244631675 |
|
- type: nauc_mrr_at_5_max |
|
value: 63.74105197634592 |
|
- type: nauc_mrr_at_5_std |
|
value: 69.88795518207282 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 11.950362559089744 |
|
- type: nauc_ndcg_at_1000_max |
|
value: 27.0707842379056 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 72.43903405163071 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: -3.597031398660954 |
|
- type: nauc_ndcg_at_100_max |
|
value: 24.415981061123944 |
|
- type: nauc_ndcg_at_100_std |
|
value: 74.01146007854192 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: 17.368676394860337 |
|
- type: nauc_ndcg_at_10_max |
|
value: 27.014276985741652 |
|
- type: nauc_ndcg_at_10_std |
|
value: 50.032884783457476 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 5.824544582933801 |
|
- type: nauc_ndcg_at_1_max |
|
value: 39.22818791946299 |
|
- type: nauc_ndcg_at_1_std |
|
value: 29.32406519654831 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: 17.816409720909615 |
|
- type: nauc_ndcg_at_20_max |
|
value: 25.056392180259827 |
|
- type: nauc_ndcg_at_20_std |
|
value: 58.05680238138826 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 15.010486876001556 |
|
- type: nauc_ndcg_at_3_max |
|
value: 4.023535837214374 |
|
- type: nauc_ndcg_at_3_std |
|
value: 22.55308565809234 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 12.73162605923733 |
|
- type: nauc_ndcg_at_5_max |
|
value: 15.425379568695105 |
|
- type: nauc_ndcg_at_5_std |
|
value: 34.4442400670659 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: -29.218427320110436 |
|
- type: nauc_precision_at_1000_max |
|
value: 29.90719259849769 |
|
- type: nauc_precision_at_1000_std |
|
value: 48.95093300052051 |
|
- type: nauc_precision_at_100_diff1 |
|
value: -6.881054858812464 |
|
- type: nauc_precision_at_100_max |
|
value: 30.388273677316956 |
|
- type: nauc_precision_at_100_std |
|
value: 76.1031398803066 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 24.298416597687574 |
|
- type: nauc_precision_at_10_max |
|
value: 44.38332754799598 |
|
- type: nauc_precision_at_10_std |
|
value: 61.64143369558439 |
|
- type: nauc_precision_at_1_diff1 |
|
value: -22.36227824463097 |
|
- type: nauc_precision_at_1_max |
|
value: 63.741051976346206 |
|
- type: nauc_precision_at_1_std |
|
value: 69.88795518207289 |
|
- type: nauc_precision_at_20_diff1 |
|
value: 21.823848783430545 |
|
- type: nauc_precision_at_20_max |
|
value: 32.815202091292875 |
|
- type: nauc_precision_at_20_std |
|
value: 61.4003619545546 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 7.264709295578332 |
|
- type: nauc_precision_at_3_max |
|
value: 18.088275115082432 |
|
- type: nauc_precision_at_3_std |
|
value: 46.315423001044266 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 19.4281378539196 |
|
- type: nauc_precision_at_5_max |
|
value: 30.042729922926426 |
|
- type: nauc_precision_at_5_std |
|
value: 48.803961503134936 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: 14.078781719704242 |
|
- type: nauc_recall_at_1000_max |
|
value: 24.205288710944746 |
|
- type: nauc_recall_at_1000_std |
|
value: 60.19521883992679 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 34.68620796161708 |
|
- type: nauc_recall_at_100_max |
|
value: 5.862669275470962 |
|
- type: nauc_recall_at_100_std |
|
value: 23.779387105339538 |
|
- type: nauc_recall_at_10_diff1 |
|
value: 41.60859491145645 |
|
- type: nauc_recall_at_10_max |
|
value: -6.060553984265031 |
|
- type: nauc_recall_at_10_std |
|
value: -3.0401474174665597 |
|
- type: nauc_recall_at_1_diff1 |
|
value: 26.134637426782753 |
|
- type: nauc_recall_at_1_max |
|
value: -3.108959317897608 |
|
- type: nauc_recall_at_1_std |
|
value: -5.781123480253076 |
|
- type: nauc_recall_at_20_diff1 |
|
value: 43.884440668985256 |
|
- type: nauc_recall_at_20_max |
|
value: -5.215456096089841 |
|
- type: nauc_recall_at_20_std |
|
value: 0.6346955652816175 |
|
- type: nauc_recall_at_3_diff1 |
|
value: 36.682959590903515 |
|
- type: nauc_recall_at_3_max |
|
value: -14.003318698999372 |
|
- type: nauc_recall_at_3_std |
|
value: -8.732791648435722 |
|
- type: nauc_recall_at_5_diff1 |
|
value: 37.55874033777468 |
|
- type: nauc_recall_at_5_max |
|
value: -11.475194910000303 |
|
- type: nauc_recall_at_5_std |
|
value: -8.24171387960509 |
|
- type: ndcg_at_1 |
|
value: 90.0 |
|
- type: ndcg_at_10 |
|
value: 83.67999999999999 |
|
- type: ndcg_at_100 |
|
value: 66.268 |
|
- type: ndcg_at_1000 |
|
value: 59.95700000000001 |
|
- type: ndcg_at_20 |
|
value: 80.41199999999999 |
|
- type: ndcg_at_3 |
|
value: 86.989 |
|
- type: ndcg_at_5 |
|
value: 85.60600000000001 |
|
- type: precision_at_1 |
|
value: 94.0 |
|
- type: precision_at_10 |
|
value: 87.0 |
|
- type: precision_at_100 |
|
value: 68.10000000000001 |
|
- type: precision_at_1000 |
|
value: 26.404 |
|
- type: precision_at_20 |
|
value: 83.7 |
|
- type: precision_at_3 |
|
value: 91.333 |
|
- type: precision_at_5 |
|
value: 89.60000000000001 |
|
- type: recall_at_1 |
|
value: 0.243 |
|
- type: recall_at_10 |
|
value: 2.307 |
|
- type: recall_at_100 |
|
value: 16.713 |
|
- type: recall_at_1000 |
|
value: 56.433 |
|
- type: recall_at_20 |
|
value: 4.3950000000000005 |
|
- type: recall_at_3 |
|
value: 0.721 |
|
- type: recall_at_5 |
|
value: 1.194 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB Touche2020 |
|
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f |
|
split: test |
|
type: mteb/touche2020 |
|
metrics: |
|
- type: main_score |
|
value: 27.095999999999997 |
|
- type: map_at_1 |
|
value: 2.708 |
|
- type: map_at_10 |
|
value: 10.926 |
|
- type: map_at_100 |
|
value: 17.023 |
|
- type: map_at_1000 |
|
value: 18.802 |
|
- type: map_at_20 |
|
value: 14.075 |
|
- type: map_at_3 |
|
value: 6.213 |
|
- type: map_at_5 |
|
value: 8.399 |
|
- type: mrr_at_1 |
|
value: 38.775510204081634 |
|
- type: mrr_at_10 |
|
value: 55.96938775510204 |
|
- type: mrr_at_100 |
|
value: 56.566806209663355 |
|
- type: mrr_at_1000 |
|
value: 56.586429443572314 |
|
- type: mrr_at_20 |
|
value: 56.566806209663355 |
|
- type: mrr_at_3 |
|
value: 53.06122448979592 |
|
- type: mrr_at_5 |
|
value: 54.48979591836734 |
|
- type: nauc_map_at_1000_diff1 |
|
value: -3.7447722422200505 |
|
- type: nauc_map_at_1000_max |
|
value: -21.154942580599432 |
|
- type: nauc_map_at_1000_std |
|
value: -3.3769353126366854 |
|
- type: nauc_map_at_100_diff1 |
|
value: -4.211734469956019 |
|
- type: nauc_map_at_100_max |
|
value: -20.97390043676955 |
|
- type: nauc_map_at_100_std |
|
value: -7.108253122379712 |
|
- type: nauc_map_at_10_diff1 |
|
value: -2.503617891657346 |
|
- type: nauc_map_at_10_max |
|
value: -19.76603379959943 |
|
- type: nauc_map_at_10_std |
|
value: -24.813694071646186 |
|
- type: nauc_map_at_1_diff1 |
|
value: -0.6946291628228135 |
|
- type: nauc_map_at_1_max |
|
value: -27.928780525228326 |
|
- type: nauc_map_at_1_std |
|
value: -26.644256007057386 |
|
- type: nauc_map_at_20_diff1 |
|
value: -8.140689983350077 |
|
- type: nauc_map_at_20_max |
|
value: -21.331762857202346 |
|
- type: nauc_map_at_20_std |
|
value: -18.46503512945984 |
|
- type: nauc_map_at_3_diff1 |
|
value: 1.4806459479634606 |
|
- type: nauc_map_at_3_max |
|
value: -20.57096686541149 |
|
- type: nauc_map_at_3_std |
|
value: -27.53855079505183 |
|
- type: nauc_map_at_5_diff1 |
|
value: -2.4911557022868833 |
|
- type: nauc_map_at_5_max |
|
value: -18.468614237544944 |
|
- type: nauc_map_at_5_std |
|
value: -27.422000270609885 |
|
- type: nauc_mrr_at_1000_diff1 |
|
value: 0.5901324153382446 |
|
- type: nauc_mrr_at_1000_max |
|
value: -29.43201840557888 |
|
- type: nauc_mrr_at_1000_std |
|
value: -22.113570283308878 |
|
- type: nauc_mrr_at_100_diff1 |
|
value: 0.6140852308779037 |
|
- type: nauc_mrr_at_100_max |
|
value: -29.423158073762274 |
|
- type: nauc_mrr_at_100_std |
|
value: -22.03830742373018 |
|
- type: nauc_mrr_at_10_diff1 |
|
value: 1.4017303142295798 |
|
- type: nauc_mrr_at_10_max |
|
value: -29.96128226635445 |
|
- type: nauc_mrr_at_10_std |
|
value: -21.182800337655188 |
|
- type: nauc_mrr_at_1_diff1 |
|
value: 2.9967188734445642 |
|
- type: nauc_mrr_at_1_max |
|
value: -28.076201809234135 |
|
- type: nauc_mrr_at_1_std |
|
value: -23.829475793931397 |
|
- type: nauc_mrr_at_20_diff1 |
|
value: 0.6140852308779037 |
|
- type: nauc_mrr_at_20_max |
|
value: -29.423158073762274 |
|
- type: nauc_mrr_at_20_std |
|
value: -22.03830742373018 |
|
- type: nauc_mrr_at_3_diff1 |
|
value: -1.7324100961545983 |
|
- type: nauc_mrr_at_3_max |
|
value: -31.25504536750873 |
|
- type: nauc_mrr_at_3_std |
|
value: -27.693245095141595 |
|
- type: nauc_mrr_at_5_diff1 |
|
value: 0.9366378266246876 |
|
- type: nauc_mrr_at_5_max |
|
value: -28.61911855691654 |
|
- type: nauc_mrr_at_5_std |
|
value: -23.51734198003236 |
|
- type: nauc_ndcg_at_1000_diff1 |
|
value: 5.589806586986813 |
|
- type: nauc_ndcg_at_1000_max |
|
value: -25.54091728191453 |
|
- type: nauc_ndcg_at_1000_std |
|
value: 18.867289766624364 |
|
- type: nauc_ndcg_at_100_diff1 |
|
value: 5.269555604924481 |
|
- type: nauc_ndcg_at_100_max |
|
value: -25.294068947248 |
|
- type: nauc_ndcg_at_100_std |
|
value: 12.57359579076201 |
|
- type: nauc_ndcg_at_10_diff1 |
|
value: -1.8036041625138828 |
|
- type: nauc_ndcg_at_10_max |
|
value: -23.89433650527811 |
|
- type: nauc_ndcg_at_10_std |
|
value: -18.669805340174104 |
|
- type: nauc_ndcg_at_1_diff1 |
|
value: 1.7320960153524356 |
|
- type: nauc_ndcg_at_1_max |
|
value: -30.98970297820504 |
|
- type: nauc_ndcg_at_1_std |
|
value: -22.039818727732 |
|
- type: nauc_ndcg_at_20_diff1 |
|
value: -7.71266194406333 |
|
- type: nauc_ndcg_at_20_max |
|
value: -28.764052281890564 |
|
- type: nauc_ndcg_at_20_std |
|
value: -14.058766573885803 |
|
- type: nauc_ndcg_at_3_diff1 |
|
value: 3.4222049394447023 |
|
- type: nauc_ndcg_at_3_max |
|
value: -23.010397388596147 |
|
- type: nauc_ndcg_at_3_std |
|
value: -23.917570461776442 |
|
- type: nauc_ndcg_at_5_diff1 |
|
value: 0.4359085390014115 |
|
- type: nauc_ndcg_at_5_max |
|
value: -18.328017574440583 |
|
- type: nauc_ndcg_at_5_std |
|
value: -22.301590122411703 |
|
- type: nauc_precision_at_1000_diff1 |
|
value: 5.705380133328601 |
|
- type: nauc_precision_at_1000_max |
|
value: 29.738757046781583 |
|
- type: nauc_precision_at_1000_std |
|
value: 37.25317043193516 |
|
- type: nauc_precision_at_100_diff1 |
|
value: 18.099479915822755 |
|
- type: nauc_precision_at_100_max |
|
value: 1.039647603335084 |
|
- type: nauc_precision_at_100_std |
|
value: 68.43506311503532 |
|
- type: nauc_precision_at_10_diff1 |
|
value: 1.6010906915801002 |
|
- type: nauc_precision_at_10_max |
|
value: -16.21198992516715 |
|
- type: nauc_precision_at_10_std |
|
value: -10.55666484527 |
|
- type: nauc_precision_at_1_diff1 |
|
value: 2.9967188734445642 |
|
- type: nauc_precision_at_1_max |
|
value: -28.076201809234135 |
|
- type: nauc_precision_at_1_std |
|
value: -23.829475793931397 |
|
- type: nauc_precision_at_20_diff1 |
|
value: -9.646266503089361 |
|
- type: nauc_precision_at_20_max |
|
value: -19.25399592456934 |
|
- type: nauc_precision_at_20_std |
|
value: 4.154373672246843 |
|
- type: nauc_precision_at_3_diff1 |
|
value: 6.468923962729313 |
|
- type: nauc_precision_at_3_max |
|
value: -16.75495139962792 |
|
- type: nauc_precision_at_3_std |
|
value: -24.1555216494731 |
|
- type: nauc_precision_at_5_diff1 |
|
value: 1.89724542441865 |
|
- type: nauc_precision_at_5_max |
|
value: -10.916266272968988 |
|
- type: nauc_precision_at_5_std |
|
value: -19.996228467499165 |
|
- type: nauc_recall_at_1000_diff1 |
|
value: -0.3248897031222208 |
|
- type: nauc_recall_at_1000_max |
|
value: -25.08629526651275 |
|
- type: nauc_recall_at_1000_std |
|
value: 72.42326605733102 |
|
- type: nauc_recall_at_100_diff1 |
|
value: 0.20011224230233096 |
|
- type: nauc_recall_at_100_max |
|
value: -25.71382782994985 |
|
- type: nauc_recall_at_100_std |
|
value: 31.40559917674001 |
|
- type: nauc_recall_at_10_diff1 |
|
value: -7.502107897824034 |
|
- type: nauc_recall_at_10_max |
|
value: -26.197156105779833 |
|
- type: nauc_recall_at_10_std |
|
value: -20.067019662396106 |
|
- type: nauc_recall_at_1_diff1 |
|
value: -0.6946291628228135 |
|
- type: nauc_recall_at_1_max |
|
value: -27.928780525228326 |
|
- type: nauc_recall_at_1_std |
|
value: -26.644256007057386 |
|
- type: nauc_recall_at_20_diff1 |
|
value: -16.829462200879107 |
|
- type: nauc_recall_at_20_max |
|
value: -29.55978083865099 |
|
- type: nauc_recall_at_20_std |
|
value: -11.329177422867945 |
|
- type: nauc_recall_at_3_diff1 |
|
value: -4.487251181022699 |
|
- type: nauc_recall_at_3_max |
|
value: -26.28852595660599 |
|
- type: nauc_recall_at_3_std |
|
value: -30.010933869743877 |
|
- type: nauc_recall_at_5_diff1 |
|
value: -7.4729339604681515 |
|
- type: nauc_recall_at_5_max |
|
value: -22.995431038489112 |
|
- type: nauc_recall_at_5_std |
|
value: -27.623494423158906 |
|
- type: ndcg_at_1 |
|
value: 35.714 |
|
- type: ndcg_at_10 |
|
value: 27.095999999999997 |
|
- type: ndcg_at_100 |
|
value: 37.577 |
|
- type: ndcg_at_1000 |
|
value: 50.234 |
|
- type: ndcg_at_20 |
|
value: 28.706 |
|
- type: ndcg_at_3 |
|
value: 34.808 |
|
- type: ndcg_at_5 |
|
value: 31.657999999999998 |
|
- type: precision_at_1 |
|
value: 38.775999999999996 |
|
- type: precision_at_10 |
|
value: 23.061 |
|
- type: precision_at_100 |
|
value: 7.388 |
|
- type: precision_at_1000 |
|
value: 1.5650000000000002 |
|
- type: precision_at_20 |
|
value: 18.776 |
|
- type: precision_at_3 |
|
value: 36.735 |
|
- type: precision_at_5 |
|
value: 31.429000000000002 |
|
- type: recall_at_1 |
|
value: 2.708 |
|
- type: recall_at_10 |
|
value: 16.645 |
|
- type: recall_at_100 |
|
value: 45.953 |
|
- type: recall_at_1000 |
|
value: 84.553 |
|
- type: recall_at_20 |
|
value: 26.259 |
|
- type: recall_at_3 |
|
value: 7.869 |
|
- type: recall_at_5 |
|
value: 11.166 |
|
task: |
|
type: Retrieval |
|
- dataset: |
|
config: default |
|
name: MTEB ToxicConversationsClassification |
|
revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de |
|
split: test |
|
type: mteb/toxic_conversations_50k |
|
metrics: |
|
- type: accuracy |
|
value: 75.87890625 |
|
- type: ap |
|
value: 16.4629793865873 |
|
- type: ap_weighted |
|
value: 16.4629793865873 |
|
- type: f1 |
|
value: 58.32993265544471 |
|
- type: f1_weighted |
|
value: 80.94360012442658 |
|
- type: main_score |
|
value: 75.87890625 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB TweetSentimentExtractionClassification |
|
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a |
|
split: test |
|
type: mteb/tweet_sentiment_extraction |
|
metrics: |
|
- type: accuracy |
|
value: 65.21788341822298 |
|
- type: f1 |
|
value: 65.00914562845475 |
|
- type: f1_weighted |
|
value: 63.672388825903845 |
|
- type: main_score |
|
value: 65.21788341822298 |
|
task: |
|
type: Classification |
|
- dataset: |
|
config: default |
|
name: MTEB TwentyNewsgroupsClustering |
|
revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 |
|
split: test |
|
type: mteb/twentynewsgroups-clustering |
|
metrics: |
|
- type: main_score |
|
value: 57.152337838073485 |
|
- type: v_measure |
|
value: 57.152337838073485 |
|
- type: v_measure_std |
|
value: 0.8799366494028795 |
|
task: |
|
type: Clustering |
|
- dataset: |
|
config: default |
|
name: MTEB TwitterSemEval2015 |
|
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 |
|
split: test |
|
type: mteb/twittersemeval2015-pairclassification |
|
metrics: |
|
- type: max_accuracy |
|
value: 88.63920844012637 |
|
- type: max_ap |
|
value: 81.41444048232692 |
|
- type: max_f1 |
|
value: 74.84396892115653 |
|
- type: accuracy |
|
value: 88.63920844012637 |
|
- type: accuracy_threshold |
|
value: 84.5294713973999 |
|
- type: ap |
|
value: 81.41443623323144 |
|
- type: f1 |
|
value: 74.84396892115653 |
|
- type: f1_threshold |
|
value: 82.87262320518494 |
|
- type: precision |
|
value: 72.34671263235656 |
|
- type: recall |
|
value: 77.5197889182058 |
|
- type: main_score |
|
value: 81.41444048232692 |
|
task: |
|
type: PairClassification |
|
- dataset: |
|
config: default |
|
name: MTEB TwitterURLCorpus |
|
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf |
|
split: test |
|
type: mteb/twitterurlcorpus-pairclassification |
|
metrics: |
|
- type: max_accuracy |
|
value: 89.90569332867622 |
|
- type: max_ap |
|
value: 87.91825594329686 |
|
- type: max_f1 |
|
value: 80.35081439054949 |
|
- type: accuracy |
|
value: 89.90569332867622 |
|
- type: accuracy_threshold |
|
value: 81.01733326911926 |
|
- type: ap |
|
value: 87.91824445175028 |
|
- type: f1 |
|
value: 80.35081439054949 |
|
- type: f1_threshold |
|
value: 78.65387201309204 |
|
- type: precision |
|
value: 75.0853013982739 |
|
- type: recall |
|
value: 86.41053279950724 |
|
- type: main_score |
|
value: 87.91825594329686 |
|
task: |
|
type: PairClassification |
|
tags: |
|
- mteb |
|
--- |
|
## Zeta-Alpha-E5-Mistral |
|
|
|
We introduce Zeta Alpha's first public embedding model, a retrieval-specialized, 7B parameter embedding model trained on top of [E5-mistral-7b-instruct](https://huggingface.co/intfloat/e5-mistral-7b-instruct). |
|
This model marks the first published model from Zeta Alpha's open science embedding models. The full dataset and detailed training recipe will be made available (very) soon. |
|
|
|
We will also make available our evaluation dataset (Called `Mini` and `Nano` versions of `BEIR`), which allowed us to quickly evaluate the model's performance while training. |
|
|
|
|
|
### Lora Weights |
|
|
|
The lora weights are also available, so there is no need to download the full model. |
|
## How to Run |
|
|
|
The model was trained with the same instruction-tuning strategy as the original [E5-mistral-7b-instruct](https://huggingface.co/intfloat/e5-mistral-7b-instruct) model. Therefore, queries should be formatted as follows: |
|
|
|
``` |
|
Instruct: <task description>\nQuery: <query> |
|
``` |
|
|
|
### Transformers |
|
``` python |
|
import torch |
|
import torch.nn.functional as F |
|
from torch import Tensor |
|
from transformers import AutoTokenizer, AutoModel |
|
|
|
def last_token_pool(last_hidden_states: Tensor, |
|
attention_mask: Tensor) -> Tensor: |
|
left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0]) |
|
if left_padding: |
|
return last_hidden_states[:, -1] |
|
else: |
|
sequence_lengths = attention_mask.sum(dim=1) - 1 |
|
batch_size = last_hidden_states.shape[0] |
|
return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), sequence_lengths] |
|
def get_detailed_instruct(task_description: str, query: str) -> str: |
|
return f'Instruct: {task_description}\nQuery: {query}' |
|
|
|
task = "Given a claim about climate change, retrieve documents that support or refute the claim" |
|
queries = [ |
|
get_detailed_instruct(task, "In Alaska, brown bears are changing their feeding habits to eat elderberries that ripen earlier."), |
|
get_detailed_instruct(task, "Local and regional sea levels continue to exhibit typical natural variability—in some places rising and in others falling.") |
|
] |
|
|
|
passages = [ |
|
"The brown bear ( Ursus arctos ) is a large bear with the widest distribution of any living ursid . The species is distributed across much of northern Eurasia and North America . It is one of the two largest terrestrial carnivorans alive today , rivaled in body size only by its close cousin , the polar bear ( Ursus maritimus ) , which is much less variable in size and averages larger due to this . There are several recognized subspecies , many of which are quite well-known within their native ranges , found in the brown bear species . The brown bear 's principal range includes parts of Russia , Central Asia , China , Canada , the United States ( mostly Alaska ) , Scandinavia and the Carpathian region ( especially Romania ) , Anatolia , and Caucasus . The brown bear is recognized as a national and state animal in several European countries . While the brown bear 's range has shrunk and it has faced local extinctions , it remains listed as a least concern species by the International Union for Conservation of Nature ( IUCN ) with a total population of approximately 200,000 . As of 2012 , this and the American black bear are the only bear species not classified as threatened by the IUCN . However , the Californian , North African ( Atlas bear ) , and Mexican subspecies were hunted to extinction in the nineteenth and early twentieth centuries , and many of the southern Asian subspecies are highly endangered . One of the smaller-bodied subspecies , the Himalayan brown bear , is critically endangered , occupying only 2 % of its former range and threatened by uncontrolled poaching for its parts . The Marsican brown bear , one of several currently isolated populations of the main Eurasian brown bear race , in central Italy is believed to have a population of just 30 to 40 bears .", |
|
"ean sea level ( MSL ) ( abbreviated simply sea level ) is an average level of the surface of one or more of Earth 's oceans from which heights such as elevations may be measured . MSL is a type of vertical datuma standardised geodetic reference pointthat is used , for example , as a chart datum in cartography and marine navigation , or , in aviation , as the standard sea level at which atmospheric pressure is measured in order to calibrate altitude and , consequently , aircraft flight levels . A common and relatively straightforward mean sea-level standard is the midpoint between a mean low and mean high tide at a particular location . Sea levels can be affected by many factors and are known to have varied greatly over geological time scales . The careful measurement of variations in MSL can offer insights into ongoing climate change , and sea level rise has been widely quoted as evidence of ongoing global warming . The term above sea level generally refers to above mean sea level ( AMSL ) ." |
|
] |
|
|
|
# load model and tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("zeta-alpha-ai/Zeta-Alpha-E5-Mistral") |
|
model = AutoModel.from_pretrained("zeta-alpha-ai/Zeta-Alpha-E5-Mistral") |
|
|
|
# get the embeddings |
|
max_length = 4096 |
|
input_texts = queries + passages |
|
batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors="pt") |
|
outputs = model(**batch_dict) |
|
embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask']) |
|
|
|
# normalize embeddings |
|
embeddings = F.normalize(embeddings, p=2, dim=1) |
|
scores = (embeddings[:2] @ embeddings[2:].T) * 100 |
|
print(scores.tolist()) |
|
[[66.15530395507812, 43.65541458129883], [47.681705474853516, 63.67986297607422]] |
|
|
|
``` |
|
|
|
### Zeta Alpha |
|
Zeta Alpha is the premier Neural Discovery Platform for AI and more. We are an Amsterdam-based R&D and product lab with a passion for AI technology, with offices on the Science Park campus of the University of Amsterdam. and in San Francisco. |
|
|
|
|
|
The Zeta Alpha Research team: |
|
- Arthur Câmara |
|
- Dinos Papakostas |
|
- Mathias Parisot |
|
- Fernando Rejon Barrera |
|
- Jakub Zavrel |
|
|
