README.md

metadata

base_model: sentence-transformers/all-mpnet-base-v2
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
  - dot_accuracy@1
  - dot_accuracy@3
  - dot_accuracy@5
  - dot_accuracy@10
  - dot_precision@1
  - dot_precision@3
  - dot_precision@5
  - dot_precision@10
  - dot_recall@1
  - dot_recall@3
  - dot_recall@5
  - dot_recall@10
  - dot_ndcg@10
  - dot_mrr@10
  - dot_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:714
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      What does the term 'rights, opportunities, or access' encompass in this
      framework?
    sentences:
      - >-
        10 

        GAI systems can ease the unintentional production or dissemination of
        false, inaccurate, or misleading 

        content (misinformation) at scale, particularly if the content stems
        from confabulations.  

        GAI systems can also ease the deliberate production or dissemination of
        false or misleading information 

        (disinformation) at scale, where an actor has the explicit intent to
        deceive or cause harm to others. Even 

        very subtle changes to text or images can manipulate human and machine
        perception. 

        Similarly, GAI systems could enable a higher degree of sophistication
        for malicious actors to produce 

        disinformation that is targeted towards specific demographics. Current
        and emerging multimodal models 

        make it possible to generate both text-based disinformation and highly
        realistic “deepfakes” – that is, 

        synthetic audiovisual content and photorealistic images.12 Additional
        disinformation threats could be 

        enabled by future GAI models trained on new data modalities.
      - >-
        74. See, e.g., Heather Morrison. Virtual Testing Puts Disabled Students
        at a Disadvantage. Government

        Technology. May 24, 2022.

        https://www.govtech.com/education/k-12/virtual-testing-puts-disabled-students-at-a-disadvantage;

        Lydia X. Z. Brown, Ridhi Shetty, Matt Scherer, and Andrew Crawford.
        Ableism And Disability

        Discrimination In New Surveillance Technologies: How new surveillance
        technologies in education,

        policing, health care, and the workplace disproportionately harm
        disabled people. Center for Democracy

        and Technology Report. May 24, 2022.

        https://cdt.org/insights/ableism-and-disability-discrimination-in-new-surveillance-technologies-how­

        new-surveillance-technologies-in-education-policing-health-care-and-the-workplace­

        disproportionately-harm-disabled-people/

        69
      - >-
        persons, Asian Americans and Pacific Islanders and other persons of
        color; members of religious minorities; 

        women, girls, and non-binary people; lesbian, gay, bisexual,
        transgender, queer, and intersex (LGBTQI+) 

        persons; older adults; persons with disabilities; persons who live in
        rural areas; and persons otherwise adversely 

        affected by persistent poverty or inequality. 

        RIGHTS, OPPORTUNITIES, OR ACCESS: “Rights, opportunities, or access” is
        used to indicate the scoping 

        of this framework. It describes the set of: civil rights, civil
        liberties, and privacy, including freedom of speech, 

        voting, and protections from discrimination, excessive punishment,
        unlawful surveillance, and violations of 

        privacy and other freedoms in both public and private sector contexts;
        equal opportunities, including equitable 

        access to education, housing, credit, employment, and other programs;
        or, access to critical resources or
  - source_sentence: >-
      What are some broad negative risks associated with GAI design,
      development, and deployment?
    sentences:
      - >-
        actually occurring, or large-scale risks could occur); and broad GAI
        negative risks, 

        including: Immature safety or risk cultures related to AI and GAI
        design, 

        development and deployment, public information integrity risks,
        including impacts 

        on democratic processes, unknown long-term performance characteristics
        of GAI. 

        Information Integrity; Dangerous, 

        Violent, or Hateful Content; CBRN 

        Information or Capabilities 

        GV-1.3-007 Devise a plan to halt development or deployment of a GAI
        system that poses 

        unacceptable negative risk. 

        CBRN Information and Capability; 

        Information Security; Information 

        Integrity 

        AI Actor Tasks: Governance and Oversight 
         
        GOVERN 1.4: The risk management process and its outcomes are established
        through transparent policies, procedures, and other 

        controls based on organizational risk priorities. 

        Action ID 

        Suggested Action 

        GAI Risks 

        GV-1.4-001 

        Establish policies and mechanisms to prevent GAI systems from generating
      - >-
        39 

        MS-3.3-004 

        Provide input for training materials about the capabilities and
        limitations of GAI 

        systems related to digital content transparency for AI Actors, other 

        professionals, and the public about the societal impacts of AI and the
        role of 

        diverse and inclusive content generation. 

        Human-AI Configuration; 

        Information Integrity; Harmful Bias 

        and Homogenization 

        MS-3.3-005 

        Record and integrate structured feedback about content provenance from 

        operators, users, and potentially impacted communities through the use
        of 

        methods such as user research studies, focus groups, or community
        forums. 

        Actively seek feedback on generated content quality and potential
        biases. 

        Assess the general awareness among end users and impacted communities 

        about the availability of these feedback channels. 

        Human-AI Configuration; 

        Information Integrity; Harmful Bias 

        and Homogenization 

        AI Actor Tasks: AI Deployment, Affected Individuals and Communities,
        End-Users, Operation and Monitoring, TEVV
      - >-
        NOTICE & 

        EXPLANATION 

        WHY THIS PRINCIPLE IS IMPORTANT

        This section provides a brief summary of the problems which the
        principle seeks to address and protect 

        against, including illustrative examples. 

        Automated systems now determine opportunities, from employment to
        credit, and directly shape the American 

        public’s experiences, from the courtroom to online classrooms, in ways
        that profoundly impact people’s lives. But this 

        expansive impact is not always visible. An applicant might not know
        whether a person rejected their resume or a 

        hiring algorithm moved them to the bottom of the list. A defendant in
        the courtroom might not know if a judge deny­

        ing their bail is informed by an automated system that labeled them
        “high risk.” From correcting errors to contesting 

        decisions, people are often denied the knowledge they need to address
        the impact of automated systems on their lives.
  - source_sentence: >-
      Who should conduct the assessment of the impact of surveillance on rights
      and opportunities?
    sentences:
      - >-
        APPENDIX

        •

        Julia Simon-Mishel, Supervising Attorney, Philadelphia Legal Assistance

        •

        Dr. Zachary Mahafza, Research & Data Analyst, Southern Poverty Law
        Center

        •

        J. Khadijah Abdurahman, Tech Impact Network Research Fellow, AI Now
        Institute, UCLA C2I1, and

        UWA Law School

        Panelists separately described the increasing scope of technology use in
        providing for social welfare, including 

        in fraud detection, digital ID systems, and other methods focused on
        improving efficiency and reducing cost. 

        However, various panelists individually cautioned that these systems may
        reduce burden for government 

        agencies by increasing the burden and agency of people using and
        interacting with these technologies. 

        Additionally, these systems can produce feedback loops and compounded
        harm, collecting data from 

        communities and using it to reinforce inequality. Various panelists
        suggested that these harms could be
      - >-
        assessments, including data retention timelines and associated
        justification, and an assessment of the 

        impact of surveillance or data collection on rights, opportunities, and
        access. Where possible, this 

        assessment of the impact of surveillance should be done by an
        independent party. Reporting should be 

        provided in a clear and machine-readable manner.  

        35
      - >-
        access to education, housing, credit, employment, and other programs;
        or, access to critical resources or 

        services, such as healthcare, financial services, safety, social
        services, non-deceptive information about goods 

        and services, and government benefits. 

        10
  - source_sentence: How can voting-related systems impact privacy and security?
    sentences:
      - >-
        as custody and divorce information, and home, work, or school
        environmental data); or have the reasonable potential 

        to be used in ways that are likely to expose individuals to meaningful
        harm, such as a loss of privacy or financial harm 

        due to identity theft. Data and metadata generated by or about those who
        are not yet legal adults is also sensitive, even 

        if not related to a sensitive domain. Such data includes, but is not
        limited to, numerical, text, image, audio, or video 

        data. “Sensitive domains” are those in which activities being conducted
        can cause material harms, including signifi­

        cant adverse effects on human rights such as autonomy and dignity, as
        well as civil liberties and civil rights. Domains 

        that have historically been singled out as deserving of enhanced data
        protections or where such enhanced protections 

        are reasonably expected by the public include, but are not limited to,
        health, family planning and care, employment,
      - >-
        agreed upon the importance of advisory boards and compensated community
        input early in the design process 

        (before the technology is built and instituted). Various panelists also
        emphasized the importance of regulation 

        that includes limits to the type and cost of such technologies. 

        56
      - >-
        Surveillance and criminal justice system algorithms such as risk
        assessments, predictive  
            policing, automated license plate readers, real-time facial recognition systems (especially  
            those used in public places or during protected activities like peaceful protests), social media  
            monitoring, and ankle monitoring devices; 
        Voting-related systems such as signature matching tools; 

        Systems with a potential privacy impact such as smart home systems and
        associated data,  
            systems that use or collect health-related data, systems that use or collect education-related  
            data, criminal justice system data, ad-targeting systems, and systems that perform big data  
            analytics in order to build profiles or infer personal information about individuals; and 
        Any system that has the meaningful potential to lead to algorithmic
        discrimination. 

        • Equal opportunities, including but not limited to:
  - source_sentence: What impact do automated systems have on underserved communities?
    sentences:
      - >-
        generation, summarization, search, and chat. These activities can take
        place within organizational 

        settings or in the public domain. 

        Organizations can restrict AI applications that cause harm, exceed
        stated risk tolerances, or that conflict 

        with their tolerances or values. Governance tools and protocols that are
        applied to other types of AI 

        systems can be applied to GAI systems. These plans and actions include: 

        • Accessibility and reasonable 

        accommodations 

        • AI actor credentials and qualifications  

        • Alignment to organizational values 

        • Auditing and assessment 

        • Change-management controls 

        • Commercial use 

        • Data provenance
      - >-
        automated systems make on underserved communities and to institute
        proactive protections that support these 

        communities. 

        •

        An automated system using nontraditional factors such as educational
        attainment and employment history as

        part of its loan underwriting and pricing model was found to be much
        more likely to charge an applicant who

        attended a Historically Black College or University (HBCU) higher loan
        prices for refinancing a student loan

        than an applicant who did not attend an HBCU. This was found to be true
        even when controlling for

        other credit-related factors.32

        •

        A hiring tool that learned the features of a company's employees
        (predominantly men) rejected women appli­

        cants for spurious and discriminatory reasons; resumes with the word
        “women’s,” such as “women’s

        chess club captain,” were penalized in the candidate ranking.33

        •

        A predictive model marketed as being able to predict whether students
        are likely to drop out of school was
      - >-
        on a principle of local control, such that those individuals closest to
        the data subject have more access while 

        those who are less proximate do not (e.g., a teacher has access to their
        students’ daily progress data while a 

        superintendent does not). 

        Reporting. In addition to the reporting on data privacy (as listed above
        for non-sensitive data), entities devel-

        oping technologies related to a sensitive domain and those collecting,
        using, storing, or sharing sensitive data 

        should, whenever appropriate, regularly provide public reports
        describing: any data security lapses or breaches 

        that resulted in sensitive data leaks; the number, type, and outcomes of
        ethical pre-reviews undertaken; a 

        description of any data sold, shared, or made public, and how that data
        was assessed to determine it did not pres-

        ent a sensitive data risk; and ongoing risk identification and
        management procedures, and any mitigation added
model-index:
  - name: SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.8881578947368421
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.993421052631579
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.993421052631579
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 1
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8881578947368421
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.331140350877193
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19868421052631577
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09999999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.8881578947368421
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.993421052631579
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.993421052631579
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 1
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.9550417755482483
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.9395363408521302
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.9395363408521302
            name: Cosine Map@100
          - type: dot_accuracy@1
            value: 0.8881578947368421
            name: Dot Accuracy@1
          - type: dot_accuracy@3
            value: 0.993421052631579
            name: Dot Accuracy@3
          - type: dot_accuracy@5
            value: 0.993421052631579
            name: Dot Accuracy@5
          - type: dot_accuracy@10
            value: 1
            name: Dot Accuracy@10
          - type: dot_precision@1
            value: 0.8881578947368421
            name: Dot Precision@1
          - type: dot_precision@3
            value: 0.331140350877193
            name: Dot Precision@3
          - type: dot_precision@5
            value: 0.19868421052631577
            name: Dot Precision@5
          - type: dot_precision@10
            value: 0.09999999999999999
            name: Dot Precision@10
          - type: dot_recall@1
            value: 0.8881578947368421
            name: Dot Recall@1
          - type: dot_recall@3
            value: 0.993421052631579
            name: Dot Recall@3
          - type: dot_recall@5
            value: 0.993421052631579
            name: Dot Recall@5
          - type: dot_recall@10
            value: 1
            name: Dot Recall@10
          - type: dot_ndcg@10
            value: 0.9550417755482483
            name: Dot Ndcg@10
          - type: dot_mrr@10
            value: 0.9395363408521302
            name: Dot Mrr@10
          - type: dot_map@100
            value: 0.9395363408521302
            name: Dot Map@100

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jet-taekyo/mpnet_finetuned_recursive")
# Run inference
sentences = [
    'What impact do automated systems have on underserved communities?',
    "automated systems make on underserved communities and to institute proactive protections that support these \ncommunities. \n•\nAn automated system using nontraditional factors such as educational attainment and employment history as\npart of its loan underwriting and pricing model was found to be much more likely to charge an applicant who\nattended a Historically Black College or University (HBCU) higher loan prices for refinancing a student loan\nthan an applicant who did not attend an HBCU. This was found to be true even when controlling for\nother credit-related factors.32\n•\nA hiring tool that learned the features of a company's employees (predominantly men) rejected women appli\xad\ncants for spurious and discriminatory reasons; resumes with the word “women’s,” such as “women’s\nchess club captain,” were penalized in the candidate ranking.33\n•\nA predictive model marketed as being able to predict whether students are likely to drop out of school was",
    'on a principle of local control, such that those individuals closest to the data subject have more access while \nthose who are less proximate do not (e.g., a teacher has access to their students’ daily progress data while a \nsuperintendent does not). \nReporting. In addition to the reporting on data privacy (as listed above for non-sensitive data), entities devel-\noping technologies related to a sensitive domain and those collecting, using, storing, or sharing sensitive data \nshould, whenever appropriate, regularly provide public reports describing: any data security lapses or breaches \nthat resulted in sensitive data leaks; the number, type, and outcomes of ethical pre-reviews undertaken; a \ndescription of any data sold, shared, or made public, and how that data was assessed to determine it did not pres-\nent a sensitive data risk; and ongoing risk identification and management procedures, and any mitigation added',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8882
cosine_accuracy@3	0.9934
cosine_accuracy@5	0.9934
cosine_accuracy@10	1.0
cosine_precision@1	0.8882
cosine_precision@3	0.3311
cosine_precision@5	0.1987
cosine_precision@10	0.1
cosine_recall@1	0.8882
cosine_recall@3	0.9934
cosine_recall@5	0.9934
cosine_recall@10	1.0
cosine_ndcg@10	0.955
cosine_mrr@10	0.9395
cosine_map@100	0.9395
dot_accuracy@1	0.8882
dot_accuracy@3	0.9934
dot_accuracy@5	0.9934
dot_accuracy@10	1.0
dot_precision@1	0.8882
dot_precision@3	0.3311
dot_precision@5	0.1987
dot_precision@10	0.1
dot_recall@1	0.8882
dot_recall@3	0.9934
dot_recall@5	0.9934
dot_recall@10	1.0
dot_ndcg@10	0.955
dot_mrr@10	0.9395
dot_map@100	0.9395

Training Details

Training Dataset

Unnamed Dataset

• Size: 714 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 714 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 10 tokens mean: 18.43 tokens max: 32 tokens	min: 22 tokens mean: 175.4 tokens max: 384 tokens

• Samples:

  sentence_0	sentence_1
  What information should designers and developers provide about automated systems to ensure transparency?	You should know that an automated system is being used, and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automat­ ed systems should provide generally accessible plain language docu­ mentation including clear descriptions of the overall system func­ tioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and ex­ planations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality chang­ es. You should know how and why an outcome impacting you was de­ termined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who
  Why is it important for individuals impacted by automated systems to be notified of significant changes in functionality?	You should know that an automated system is being used, and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automat­ ed systems should provide generally accessible plain language docu­ mentation including clear descriptions of the overall system func­ tioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and ex­ planations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality chang­ es. You should know how and why an outcome impacting you was de­ termined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who
  What specific technical questions does the questionnaire for evaluating software workers cover?	questionnaire that businesses can use proactively when procuring software to evaluate workers. It covers specific technical questions such as the training data used, model training process, biases identified, and mitigation steps employed.55 Standards organizations have developed guidelines to incorporate accessibility criteria into technology design processes. The most prevalent in the United States is the Access Board’s Section 508 regulations,56 which are the technical standards for federal information communication technology (software, hardware, and web). Other standards include those issued by the International Organization for Standardization,57 and the World Wide Web Consortium Web Content Accessibility Guidelines,58 a globally recognized voluntary consensus standard for web content and other information and communications technology. NIST has released Special Publication 1270, Towards a Standard for Identifying and Managing Bias

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 20
• per_device_eval_batch_size: 20
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 20
• per_device_eval_batch_size: 20
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_map@100
1.0	36	0.9395

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 3.1.0
• Transformers: 4.44.2
• PyTorch: 2.4.1+cu121
• Accelerate: 0.34.2
• Datasets: 3.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}