baconnier/Finance_embedding_large_en-V0.1

SentenceTransformer based on BAAI/bge-large-en

This is a sentence-transformers model finetuned from BAAI/bge-large-en. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-large-en
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • baconnier/finance2_dataset_private

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("baconnier/Finance_embedding_large_en-V0.1")
# Run inference
sentences = [
    'How many companies are listed on the NYSE?',
    'What are the trading hours of the New York Stock Exchange?',
    'Why do Maple Leaf coins often trade at a premium over their metal content value?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Dataset: Original_Embedding_model_Metric
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.5006
dot_accuracy	0.4977
manhattan_accuracy	0.5015
euclidean_accuracy	0.5003
max_accuracy	0.5015

Triplet

• Dataset: Finance_model_Embedding_Metric
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9872
dot_accuracy	0.0112
manhattan_accuracy	0.9869
euclidean_accuracy	0.9872
max_accuracy	0.9872

Training Details

Training Dataset

baconnier/finance2_dataset_private

• Dataset: baconnier/finance2_dataset_private at f384fe0
• Size: 36,223 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 9 tokens mean: 25.02 tokens max: 63 tokens	min: 22 tokens mean: 152.04 tokens max: 460 tokens	min: 16 tokens mean: 54.96 tokens max: 225 tokens

• Samples:

  anchor	positive	negative
  When was the Libyan Dinar (LYD) introduced, and what was the exchange rate with the previous currency?	According to the context, the Libyan Dinar (LYD) was introduced in 1971, replacing the Libyan pound at a rate of 1 dinar = 1 pound. The Libyan Dinar (LYD) was introduced in 1971, replacing the Libyan pound at a rate of 1 dinar to 1 pound.	The Libyan Dinar was introduced sometime in the 20th century. The Libyan Dinar was introduced in the 20th century.
  How many fillér would you have if you exchanged 10 USD for Hungarian Forints at the given exchange rate?	First, calculate the HUF equivalent of 10 USD using the exchange rate: 1 USD ≈ 339 HUF, so 10 USD ≈ 10 × 339 = 3,390 HUF. The context also states that 1 HUF = 100 fillér, so to find the number of fillér, multiply the HUF amount by 100: 3,390 HUF × 100 fillér/HUF = 339,000 fillér. At the given exchange rate, exchanging 10 USD would give you approximately 339,000 fillér.	The context provides the exchange rate between USD and HUF, but it doesn't mention how many HUF equal one fillér. Without knowing the conversion rate between HUF and fillér, it's impossible to calculate the number of fillér you'd get for 10 USD. There is not enough information provided to determine the number of fillér you would get for 10 USD.
  What is the total value of John's vintage car collection and his wife's jewelry collection combined?	The passage states that John's vintage car collection is valued at $500,000 and his wife's jewelry collection is worth $200,000. To find the total value, we add these two amounts: Vintage car collection: $500,000 Jewelry collection: $200,000 $500,000 + $200,000 = $700,000 Therefore, the total value of John's vintage car collection and his wife's jewelry collection combined is $700,000. The total value of John's vintage car collection and his wife's jewelry collection combined is $700,000.	The passage mentions that John has a vintage car collection and his wife has a jewelry collection. However, the values of these collections are not provided. The total value of John's vintage car collection and his wife's jewelry collection cannot be determined from the given information.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

baconnier/finance2_dataset_private

• Dataset: baconnier/finance2_dataset_private at f384fe0
• Size: 7,762 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 9 tokens mean: 25.52 tokens max: 74 tokens	min: 22 tokens mean: 153.66 tokens max: 512 tokens	min: 12 tokens mean: 53.73 tokens max: 181 tokens

• Samples:

  anchor	positive	negative
  What factors have contributed to Acme Inc.'s stock becoming a wallflower?	Several factors have contributed to Acme Inc.'s stock becoming a wallflower: 1. Declining sales: Acme Inc. has experienced a decline in sales, which has negatively impacted its financial performance. 2. Decreasing profit margins: Along with declining sales, Acme Inc.'s profit margins have also decreased, further affecting its bottom line. 3. Falling stock price: As a result of the declining sales and profit margins, Acme Inc.'s stock price has dropped significantly. 4. Low P/E ratio: The company's P/E ratio has decreased to 8, which is much lower than the industry average of 15. This low P/E ratio indicates that investors are not willing to pay a premium for Acme Inc.'s stock due to its poor financial performance. These factors have collectively led to Acme Inc.'s stock falling out of favor with investors, making it a wallflower stock. Acme Inc.'s stock has become a wallflower due to a combination of factors, including declining sales, decreasing profit margins, a falling stock price, and a low P/E ratio compared to the industry average, which have led to investors losing interest in the company's stock.	Acme Inc.'s stock has become a wallflower because its P/E ratio is lower than the industry average. Acme Inc.'s low P/E ratio has caused its stock to become a wallflower.
  How does the Accumulated Benefit Obligation (ABO) differ from the Projected Benefit Obligation (PBO) in terms of assumptions about future salary increases?	The Accumulated Benefit Obligation (ABO) assumes that the pension plan will terminate immediately and does not take into account any future salary increases. In contrast, the Projected Benefit Obligation (PBO) includes assumptions about future salary increases when calculating the present value of an employee's pension benefits. The ABO does not consider future salary increases, assuming immediate plan termination, while the PBO incorporates assumptions about future salary increases in its calculations.	The ABO and PBO are the same things and both include assumptions about future salary increases for employees. There is no difference between ABO and PBO in terms of salary increase assumptions.
  What is the annual interest rate of the annuity, and how is it compounded?	According to the context, the annuity has an annual interest rate of 3%. This interest is compounded monthly, meaning the 3% annual rate is divided by 12 (the number of months in a year) and applied to the account balance each month. This results in a slightly higher effective annual rate due to the compound growth. The annuity has an annual interest rate of 3%, which is compounded monthly, resulting in compound growth of the account balance.	The annuity has an interest rate that is compounded. The annuity's interest rate is compounded.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	Finance_model_Embedding_Metric_max_accuracy	Original_Embedding_model_Metric_max_accuracy
0	0	-	-	-	0.5015
0.0044	10	1.0947	-	-	-
0.0088	20	0.9611	-	-	-
0.0133	30	0.6565	-	-	-
0.0177	40	0.4234	-	-	-
0.0221	50	0.1672	-	-	-
0.0265	60	0.1305	-	-	-
0.0309	70	0.1381	-	-	-
0.0353	80	0.0846	-	-	-
0.0398	90	0.1078	-	-	-
0.0442	100	0.0867	-	-	-
0.0486	110	0.0935	-	-	-
0.0530	120	0.1197	-	-	-
0.0574	130	0.0841	-	-	-
0.0618	140	0.0792	-	-	-
0.0663	150	0.0811	-	-	-
0.0707	160	0.1032	-	-	-
0.0751	170	0.1051	-	-	-
0.0795	180	0.1091	-	-	-
0.0839	190	0.0778	-	-	-
0.0883	200	0.1006	-	-	-
0.0928	210	0.0738	-	-	-
0.0972	220	0.1105	-	-	-
0.1003	227	-	0.1181	-	-
0.1016	230	0.0697	-	-	-
0.1060	240	0.064	-	-	-
0.1104	250	0.1204	-	-	-
0.1148	260	0.0664	-	-	-
0.1193	270	0.0776	-	-	-
0.1237	280	0.0574	-	-	-
0.1281	290	0.054	-	-	-
0.1325	300	0.0681	-	-	-
0.1369	310	0.1315	-	-	-
0.1413	320	0.1005	-	-	-
0.1458	330	0.0613	-	-	-
0.1502	340	0.0476	-	-	-
0.1546	350	0.0735	-	-	-
0.1590	360	0.106	-	-	-
0.1634	370	0.1082	-	-	-
0.1678	380	0.0437	-	-	-
0.1723	390	0.0782	-	-	-
0.1767	400	0.0858	-	-	-
0.1811	410	0.0563	-	-	-
0.1855	420	0.0798	-	-	-
0.1899	430	0.0674	-	-	-
0.1943	440	0.0887	-	-	-
0.1988	450	0.1032	-	-	-
0.2005	454	-	0.0720	-	-
0.2032	460	0.0591	-	-	-
0.2076	470	0.0581	-	-	-
0.2120	480	0.1544	-	-	-
0.2164	490	0.0169	-	-	-
0.2208	500	0.0593	-	-	-
0.2253	510	0.0971	-	-	-
0.2297	520	0.0567	-	-	-
0.2341	530	0.0501	-	-	-
0.2385	540	0.0452	-	-	-
0.2429	550	0.0574	-	-	-
0.2473	560	0.0616	-	-	-
0.2518	570	0.1414	-	-	-
0.2562	580	0.0776	-	-	-
0.2606	590	0.0828	-	-	-
0.2650	600	0.1046	-	-	-
0.2694	610	0.1248	-	-	-
0.2739	620	0.0547	-	-	-
0.2783	630	0.0424	-	-	-
0.2827	640	0.1401	-	-	-
0.2871	650	0.0746	-	-	-
0.2915	660	0.0279	-	-	-
0.2959	670	0.1115	-	-	-
0.3004	680	0.0846	-	-	-
0.3008	681	-	0.0655	-	-
0.3048	690	0.063	-	-	-
0.3092	700	0.0949	-	-	-
0.3136	710	0.0482	-	-	-
0.3180	720	0.063	-	-	-
0.3224	730	0.0524	-	-	-
0.3269	740	0.0752	-	-	-
0.3313	750	0.0964	-	-	-
0.3357	760	0.0378	-	-	-
0.3401	770	0.0611	-	-	-
0.3445	780	0.0764	-	-	-
0.3489	790	0.0391	-	-	-
0.3534	800	0.0549	-	-	-
0.3578	810	0.0717	-	-	-
0.3622	820	0.0688	-	-	-
0.3666	830	0.0891	-	-	-
0.3710	840	0.034	-	-	-
0.3754	850	0.0773	-	-	-
0.3799	860	0.0377	-	-	-
0.3843	870	0.0629	-	-	-
0.3887	880	0.0544	-	-	-
0.3931	890	0.0384	-	-	-
0.3975	900	0.0489	-	-	-
0.4011	908	-	0.0708	-	-
0.4019	910	0.0757	-	-	-
0.4064	920	0.0904	-	-	-
0.4108	930	0.0569	-	-	-
0.4152	940	0.0875	-	-	-
0.4196	950	0.0452	-	-	-
0.4240	960	0.0791	-	-	-
0.4284	970	0.0721	-	-	-
0.4329	980	0.0354	-	-	-
0.4373	990	0.0171	-	-	-
0.4417	1000	0.0726	-	-	-
0.4461	1010	0.0546	-	-	-
0.4505	1020	0.0352	-	-	-
0.4549	1030	0.0424	-	-	-
0.4594	1040	0.063	-	-	-
0.4638	1050	0.0928	-	-	-
0.4682	1060	0.0648	-	-	-
0.4726	1070	0.0591	-	-	-
0.4770	1080	0.0506	-	-	-
0.4814	1090	0.0991	-	-	-
0.4859	1100	0.0268	-	-	-
0.4903	1110	0.039	-	-	-
0.4947	1120	0.0913	-	-	-
0.4991	1130	0.0413	-	-	-
0.5013	1135	-	0.0542	-	-
0.5035	1140	0.0706	-	-	-
0.5080	1150	0.0476	-	-	-
0.5124	1160	0.0567	-	-	-
0.5168	1170	0.0425	-	-	-
0.5212	1180	0.0378	-	-	-
0.5256	1190	0.0531	-	-	-
0.5300	1200	0.0839	-	-	-
0.5345	1210	0.0378	-	-	-
0.5389	1220	0.0309	-	-	-
0.5433	1230	0.0213	-	-	-
0.5477	1240	0.0769	-	-	-
0.5521	1250	0.0543	-	-	-
0.5565	1260	0.0587	-	-	-
0.5610	1270	0.0658	-	-	-
0.5654	1280	0.0621	-	-	-
0.5698	1290	0.0558	-	-	-
0.5742	1300	0.0521	-	-	-
0.5786	1310	0.0481	-	-	-
0.5830	1320	0.0373	-	-	-
0.5875	1330	0.0652	-	-	-
0.5919	1340	0.0685	-	-	-
0.5963	1350	0.077	-	-	-
0.6007	1360	0.0521	-	-	-
0.6016	1362	-	0.0516	-	-
0.6051	1370	0.0378	-	-	-
0.6095	1380	0.0442	-	-	-
0.6140	1390	0.0435	-	-	-
0.6184	1400	0.0288	-	-	-
0.6228	1410	0.0565	-	-	-
0.6272	1420	0.0449	-	-	-
0.6316	1430	0.0226	-	-	-
0.6360	1440	0.0395	-	-	-
0.6405	1450	0.059	-	-	-
0.6449	1460	0.1588	-	-	-
0.6493	1470	0.0562	-	-	-
0.6537	1480	0.117	-	-	-
0.6581	1490	0.107	-	-	-
0.6625	1500	0.0972	-	-	-
0.6670	1510	0.0684	-	-	-
0.6714	1520	0.0743	-	-	-
0.6758	1530	0.0784	-	-	-
0.6802	1540	0.0892	-	-	-
0.6846	1550	0.0676	-	-	-
0.6890	1560	0.0312	-	-	-
0.6935	1570	0.0834	-	-	-
0.6979	1580	0.0241	-	-	-
0.7019	1589	-	0.0495	-	-
0.7023	1590	0.0391	-	-	-
0.7067	1600	0.043	-	-	-
0.7111	1610	0.045	-	-	-
0.7155	1620	0.0216	-	-	-
0.7200	1630	0.0715	-	-	-
0.7244	1640	0.0173	-	-	-
0.7288	1650	0.0249	-	-	-
0.7332	1660	0.0187	-	-	-
0.7376	1670	0.0647	-	-	-
0.7420	1680	0.0199	-	-	-
0.7465	1690	0.0333	-	-	-
0.7509	1700	0.0718	-	-	-
0.7553	1710	0.0373	-	-	-
0.7597	1720	0.0744	-	-	-
0.7641	1730	0.0185	-	-	-
0.7686	1740	0.0647	-	-	-
0.7730	1750	0.0289	-	-	-
0.7774	1760	0.034	-	-	-
0.7818	1770	0.0184	-	-	-
0.7862	1780	0.0537	-	-	-
0.7906	1790	0.0724	-	-	-
0.7951	1800	0.0511	-	-	-
0.7995	1810	0.0165	-	-	-
0.8021	1816	-	0.0488	-	-
0.8039	1820	0.0364	-	-	-
0.8083	1830	0.1126	-	-	-
0.8127	1840	0.0148	-	-	-
0.8171	1850	0.0722	-	-	-
0.8216	1860	0.0586	-	-	-
0.8260	1870	0.0496	-	-	-
0.8304	1880	0.026	-	-	-
0.8348	1890	0.0417	-	-	-
0.8392	1900	0.0586	-	-	-
0.8436	1910	0.0255	-	-	-
0.8481	1920	0.0329	-	-	-
0.8525	1930	0.015	-	-	-
0.8569	1940	0.0657	-	-	-
0.8613	1950	0.0465	-	-	-
0.8657	1960	0.0107	-	-	-
0.8701	1970	0.0401	-	-	-
0.8746	1980	0.022	-	-	-
0.8790	1990	0.061	-	-	-
0.8834	2000	0.0474	-	-	-
0.8878	2010	0.0358	-	-	-
0.8922	2020	0.0599	-	-	-
0.8966	2030	0.0522	-	-	-
0.9011	2040	0.0312	-	-	-
0.9024	2043	-	0.0421	-	-
0.9055	2050	0.024	-	-	-
0.9099	2060	0.1085	-	-	-
0.9143	2070	0.0144	-	-	-
0.9187	2080	0.038	-	-	-
0.9231	2090	0.0948	-	-	-
0.9276	2100	0.0317	-	-	-
0.9320	2110	0.0674	-	-	-
0.9364	2120	0.081	-	-	-
0.9408	2130	0.036	-	-	-
0.9452	2140	0.0649	-	-	-
0.9496	2150	0.0235	-	-	-
0.9541	2160	0.0291	-	-	-
0.9585	2170	0.0293	-	-	-
0.9629	2180	0.0703	-	-	-
0.9673	2190	0.0148	-	-	-
0.9717	2200	0.0397	-	-	-
0.9761	2210	0.0552	-	-	-
0.9806	2220	0.0097	-	-	-
0.9850	2230	0.0723	-	-	-
0.9894	2240	0.0379	-	-	-
0.9938	2250	0.0289	-	-	-
0.9982	2260	0.0267	-	-	-
1.0	2264	-	-	0.9872	-

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.0
• Transformers: 4.41.1
• PyTorch: 2.3.0+cu121
• Accelerate: 0.30.1
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}