SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A ClassifierChain instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a ClassifierChain instance
Maximum Sequence Length: 512 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Evaluation

Metrics

Label	Metric
all	0.6881

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("CrisisNarratives/setfit-9classes-multi_label")
# Run inference
preds = model("my dad had huge ones..so they may be real..")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	25.8891	1681

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (3, 3)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 40
body_learning_rate: (1.752e-05, 1.752e-05)
head_learning_rate: 1.752e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 30
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0004	1	0.3395	-
0.0185	50	0.3628	-
0.0370	100	0.2538	-
0.0555	150	0.2044	-
0.0739	200	0.1831	-
0.0924	250	0.2218	-
0.1109	300	0.2014	-
0.1294	350	0.2405	-
0.1479	400	0.1238	-
0.1664	450	0.1658	-
0.1848	500	0.1974	-
0.2033	550	0.1565	-
0.2218	600	0.1131	-
0.2403	650	0.0994	-
0.2588	700	0.0743	-
0.2773	750	0.0259	-
0.2957	800	0.1852	-
0.3142	850	0.1896	-
0.3327	900	0.1102	-
0.3512	950	0.0951	-
0.3697	1000	0.0619	-
0.3882	1050	0.0227	-
0.4067	1100	0.0986	-
0.4251	1150	0.0375	-
0.4436	1200	0.1151	-
0.4621	1250	0.1128	-
0.4806	1300	0.0334	-
0.4991	1350	0.1012	-
0.5176	1400	0.0895	-
0.5360	1450	0.072	-
0.5545	1500	0.0619	-
0.5730	1550	0.0852	-
0.5915	1600	0.0611	-
0.6100	1650	0.0679	-
0.6285	1700	0.0238	-
0.6470	1750	0.1776	-
0.6654	1800	0.081	-
0.6839	1850	0.1059	-
0.7024	1900	0.045	-
0.7209	1950	0.0664	-
0.7394	2000	0.0666	-
0.7579	2050	0.0714	-
0.7763	2100	0.0312	-
0.7948	2150	0.0461	-
0.8133	2200	0.0946	-
0.8318	2250	0.047	-
0.8503	2300	0.0906	-
0.8688	2350	0.0186	-
0.8872	2400	0.0937	-
0.9057	2450	0.1674	-
0.9242	2500	0.0311	-
0.9427	2550	0.0884	-
0.9612	2600	0.0787	-
0.9797	2650	0.192	-
0.9982	2700	0.0689	-
1.0166	2750	0.0945	-
1.0351	2800	0.066	-
1.0536	2850	0.0592	-
1.0721	2900	0.068	-
1.0906	2950	0.0619	-
1.1091	3000	0.0329	-
1.1275	3050	0.0986	-
1.1460	3100	0.0468	-
1.1645	3150	0.0717	-
1.1830	3200	0.0721	-
1.2015	3250	0.0345	-
1.2200	3300	0.0317	-
1.2384	3350	0.0476	-
1.2569	3400	0.122	-
1.2754	3450	0.0576	-
1.2939	3500	0.0375	-
1.3124	3550	0.1074	-
1.3309	3600	0.113	-
1.3494	3650	0.0564	-
1.3678	3700	0.0437	-
1.3863	3750	0.0623	-
1.4048	3800	0.0213	-
1.4233	3850	0.0629	-
1.4418	3900	0.059	-
1.4603	3950	0.0807	-
1.4787	4000	0.0946	-
1.4972	4050	0.0381	-
1.5157	4100	0.0451	-
1.5342	4150	0.0742	-
1.5527	4200	0.0899	-
1.5712	4250	0.0722	-
1.5896	4300	0.1022	-
1.6081	4350	0.0446	-
1.6266	4400	0.022	-
1.6451	4450	0.0586	-
1.6636	4500	0.0585	-
1.6821	4550	0.0409	-
1.7006	4600	0.0253	-
1.7190	4650	0.0363	-
1.7375	4700	0.0492	-
1.7560	4750	0.0154	-
1.7745	4800	0.0427	-
1.7930	4850	0.0284	-
1.8115	4900	0.022	-
1.8299	4950	0.0335	-
1.8484	5000	0.0222	-
1.8669	5050	0.0291	-
1.8854	5100	0.0824	-
1.9039	5150	0.0563	-
1.9224	5200	0.0355	-
1.9409	5250	0.064	-
1.9593	5300	0.0596	-
1.9778	5350	0.0789	-
1.9963	5400	0.0901	-
2.0148	5450	0.0388	-
2.0333	5500	0.0738	-
2.0518	5550	0.0712	-
2.0702	5600	0.0825	-
2.0887	5650	0.0406	-
2.1072	5700	0.0623	-
2.1257	5750	0.0423	-
2.1442	5800	0.0566	-
2.1627	5850	0.0745	-
2.1811	5900	0.0271	-
2.1996	5950	0.0257	-
2.2181	6000	0.0347	-
2.2366	6050	0.0291	-
2.2551	6100	0.0401	-
2.2736	6150	0.0222	-
2.2921	6200	0.0217	-
2.3105	6250	0.0589	-
2.3290	6300	0.0685	-
2.3475	6350	0.1191	-
2.3660	6400	0.0626	-
2.3845	6450	0.0615	-
2.4030	6500	0.0327	-
2.4214	6550	0.0431	-
2.4399	6600	0.1037	-
2.4584	6650	0.0318	-
2.4769	6700	0.062	-
2.4954	6750	0.0183	-
2.5139	6800	0.0568	-
2.5323	6850	0.0581	-
2.5508	6900	0.0363	-
2.5693	6950	0.0413	-
2.5878	7000	0.076	-
2.6063	7050	0.046	-
2.6248	7100	0.0401	-
2.6433	7150	0.0552	-
2.6617	7200	0.0767	-
2.6802	7250	0.0167	-
2.6987	7300	0.0459	-
2.7172	7350	0.0306	-
2.7357	7400	0.0559	-
2.7542	7450	0.0688	-
2.7726	7500	0.0417	-
2.7911	7550	0.033	-
2.8096	7600	0.0404	-
2.8281	7650	0.0391	-
2.8466	7700	0.0254	-
2.8651	7750	0.0635	-
2.8835	7800	0.0739	-
2.9020	7850	0.0274	-
2.9205	7900	0.0394	-
2.9390	7950	0.0606	-
2.9575	8000	0.0098	-
2.9760	8050	0.0997	-
2.9945	8100	0.0369	-

Framework Versions

Python: 3.9.16
SetFit: 1.0.1
Sentence Transformers: 2.2.2
Transformers: 4.35.0
PyTorch: 2.1.0+cu121
Datasets: 2.14.6
Tokenizers: 0.14.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

CrisisNarratives
/

setfit-9classes-multi_label