Model Card for SynCSE-scratch

Model Details

Model Description

More information needed

Developed by: SJTU-LIT
Shared by [Optional]: SJTU-LIT
Model type: Feature Extraction
Language(s) (NLP): More information needed
License: More information needed
Parent Model: RoBERTa-base
Resources for more information: - GitHub Repo - Associated Paper

Uses

Direct Use

This model can be used for the task of feature extraction.

Out-of-Scope Use

The model should not be used to intentionally create hostile or alienating environments for people.

Bias, Risks, and Limitations

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Training Data

The model craters note in the Github Repository

We use 27.5k generated synthetic train SynCSE-sractch-RoBERTa-base.

Citation

BibTeX:

@article{zhang2023contrastive,
  title={Contrastive Learning of Sentence Embeddings from Scratch},
  author={Zhang, Junlei and Lan, Zhenzhong and He, Junxian},
  journal={arXiv preprint arXiv:2305.15077},
  year={2023}
}

Model Card Contact

If you have any questions related to the code or the paper, feel free to email Junlei ([email protected]). If you encounter any problems when using the code, or want to report a bug, you can open an issue. Please try to specify the problem with details so we can help you better and quicker!

How to Get Started with the Model

Use the code below to get started with the model.

Click to expand

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("sjtu-lit/SynCSE-partial-RoBERTa-base")
model = AutoModel.from_pretrained("sjtu-lit/SynCSE-partial-RoBERTa-base")