bwang0911 commited on
Commit
521abc0
1 Parent(s): 92252ab

chore: update readme

Browse files
Files changed (1) hide show
  1. README.md +9 -47
README.md CHANGED
@@ -21684,53 +21684,6 @@ embeddings = model.encode(
21684
  ```
21685
 
21686
 
21687
-
21688
- ## Performance
21689
-
21690
- ### English MTEB
21691
- | Model | Dimension | Average | Classification | Clustering | Pair Classification | Reranking | Retrieval | STS | Summarization |
21692
- |:------------------------------:|:-----------:|:---------:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|
21693
- | jina-embeddings-v3 | 1024 | **65.60** | **82.58**| 45.27| 84.01| 58.13| 53.87| **85.8** | 30.98|
21694
- | jina-embeddings-v2-en | 768 | 58.12 | 68.82 | 40.08| 84.44| 55.09| 45.64| 80.00| 30.56|
21695
- | text-embedding-3-large | 3072 | 62.03 | 75.45 | 49.01| 84.22| 59.16| 55.44| 81.04| 29.92|
21696
- | multilingual-e5-large-instruct | 1024 | 64.41 | 77.56 | 47.1 | 86.19| 58.58| 52.47| 84.78| 30.39|
21697
- | Cohere-embed-multilingual-v3.0 | 1024 | 60.08 | 64.01 | 46.6 | 86.15| 57.86| 53.84| 83.15| 30.99|
21698
-
21699
- ### Multilingual MTEB
21700
-
21701
- | Model | Dimension | Average | Classification | Clustering | Pair Classification | Reranking | Retrieval | STS | Summarization |
21702
- |:------------------------------:|:---------:|:---------:|:--------------:|:----------:|:-------------------:|:---------:|:---------:|:---------:|:-------------:|
21703
- | jina-embeddings-v3 | 1024 | **64.44** | **71.46** | 46.71 | 76.91 | 63.98 | 57.98 | **69.83** | - |
21704
- | multilingual-e5-large | 1024 | 59.58 | 65.22 | 42.12 | 76.95 | 63.4 | 52.37 | 64.65 | - |
21705
- | multilingual-e5-large-instruct | 1024 | 64.25 | 67.45 | **52.12** | 77.79 | **69.02** | **58.38** | 68.77 | - |
21706
-
21707
-
21708
- ### Long Context Tasks (LongEmbed)
21709
-
21710
- | Model | Dimension | Average | NarrativeQA | Needle | Passkey | QMSum | SummScreen | WikiQA |
21711
- |:----------------------:|:---------:|:---------:|:-----------:|:---------:|:----------:|:---------:|:----------:|:---------:|
21712
- | jina-embeddings-v3* | 1024 | **70.39** | 33.32 | **84.00** | **100.00** | **39.75** | 92.78 | 72.46 |
21713
- | jina-embeddings-v2 | 768 | 58.12 | 37.89 | 54.25 | 50.25 | 38.87 | 93.48 | 73.99 |
21714
- | text-embedding-3-large | 3072 | 51.30 | 44.09 | 29.25 | 63.00 | 32.49 | 84.80 | 54.16 |
21715
- | baai-bge-m3 | 1024 | 56.56 | **45.76** | 40.25 | 46.00 | 35.54 | **94.09** | **77.73** |
21716
-
21717
- Notes: `*`, use the text-matching adapter
21718
-
21719
-
21720
- #### Matryoshka Embeddings
21721
-
21722
- | Dimension | Retrieval | STS |
21723
- |:-----------:|:-----------:|:-------:|
21724
- | 32 | 52.54 | 76.35 |
21725
- | 64 | 58.54 | 77.03 |
21726
- | 128 | 61.64 | 77.43 |
21727
- | 256 | 62.72 | 77.56 |
21728
- | 512 | 63.16 | 77.59 |
21729
- | 768 | 63.3 | 77.59 |
21730
- | 1024 | 63.35 | 77.58 |
21731
-
21732
- For a comprehensive evaluation and detailed metrics, please refer to the full paper available here (coming soon).
21733
-
21734
  ## Contact
21735
 
21736
  Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.
@@ -21740,5 +21693,14 @@ Join our [Discord community](https://discord.jina.ai) and chat with other commun
21740
  If you find `jina-embeddings-v3` useful in your research, please cite the following paper:
21741
 
21742
  ```bibtex
 
 
 
 
 
 
 
 
 
21743
 
21744
  ```
 
21684
  ```
21685
 
21686
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21687
  ## Contact
21688
 
21689
  Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.
 
21693
  If you find `jina-embeddings-v3` useful in your research, please cite the following paper:
21694
 
21695
  ```bibtex
21696
+ @misc{sturua2024jinaembeddingsv3multilingualembeddingstask,
21697
+ title={jina-embeddings-v3: Multilingual Embeddings With Task LoRA},
21698
+ author={Saba Sturua and Isabelle Mohr and Mohammad Kalim Akram and Michael Günther and Bo Wang and Markus Krimmel and Feng Wang and Georgios Mastrapas and Andreas Koukounas and Andreas Koukounas and Nan Wang and Han Xiao},
21699
+ year={2024},
21700
+ eprint={2409.10173},
21701
+ archivePrefix={arXiv},
21702
+ primaryClass={cs.CL},
21703
+ url={https://arxiv.org/abs/2409.10173},
21704
+ }
21705
 
21706
  ```