sembr2023-distilbert-base-uncased

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2391
• Precision: 0.8025
• Recall: 0.8289
• F1: 0.8155
• Iou: 0.6885
• Accuracy: 0.9655
• Balanced Accuracy: 0.9041
• Overall Accuracy: 0.9498

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Iou	Accuracy	Balanced Accuracy	Overall Accuracy
0.4046	0.06	10	0.4081	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.376	0.12	20	0.3776	0	0.0	0.0	0.0	0.9080	0.5	0.9080
0.3291	0.18	30	0.2954	0.8417	0.0813	0.1482	0.0801	0.9141	0.5399	0.9134
0.2309	0.24	40	0.2165	0.8158	0.6008	0.6920	0.5290	0.9508	0.7935	0.9357
0.1704	0.3	50	0.1928	0.8703	0.6428	0.7394	0.5866	0.9583	0.8166	0.9420
0.1711	0.36	60	0.1810	0.7918	0.7620	0.7766	0.6348	0.9597	0.8709	0.9404
0.1716	0.42	70	0.1841	0.8143	0.7735	0.7934	0.6575	0.9629	0.8778	0.9426
0.1452	0.48	80	0.1610	0.9013	0.6827	0.7769	0.6352	0.9639	0.8376	0.9486
0.1233	0.55	90	0.1604	0.8569	0.7587	0.8048	0.6734	0.9662	0.8729	0.9487
0.1291	0.61	100	0.1694	0.8405	0.7794	0.8088	0.6789	0.9661	0.8822	0.9475
0.1335	0.67	110	0.1529	0.8730	0.7445	0.8036	0.6717	0.9665	0.8667	0.9494
0.0921	0.73	120	0.1675	0.8238	0.8090	0.8164	0.6897	0.9665	0.8958	0.9462
0.0936	0.79	130	0.1469	0.8672	0.7738	0.8178	0.6918	0.9683	0.8809	0.9520
0.1168	0.85	140	0.1570	0.8190	0.8088	0.8139	0.6862	0.9660	0.8953	0.9483
0.091	0.91	150	0.1630	0.8330	0.7842	0.8079	0.6777	0.9657	0.8841	0.9477
0.0795	0.97	160	0.1413	0.8477	0.7934	0.8196	0.6944	0.9679	0.8895	0.9539
0.0927	1.03	170	0.1678	0.8071	0.8218	0.8144	0.6869	0.9655	0.9009	0.9467
0.0748	1.09	180	0.1346	0.8785	0.7628	0.8166	0.6900	0.9685	0.8761	0.9563
0.0731	1.15	190	0.1691	0.8290	0.8018	0.8152	0.6880	0.9666	0.8925	0.9485
0.0842	1.21	200	0.1561	0.8264	0.8127	0.8195	0.6942	0.9671	0.8977	0.9512
0.0702	1.27	210	0.1419	0.8459	0.7931	0.8187	0.6930	0.9677	0.8892	0.9539
0.0577	1.33	220	0.1506	0.8375	0.8057	0.8213	0.6968	0.9678	0.8950	0.9521
0.0596	1.39	230	0.1628	0.7907	0.8377	0.8135	0.6856	0.9647	0.9076	0.9485
0.0562	1.45	240	0.1585	0.8413	0.7953	0.8176	0.6915	0.9674	0.8900	0.9518
0.0519	1.52	250	0.1740	0.8062	0.8189	0.8125	0.6842	0.9652	0.8995	0.9490
0.0547	1.58	260	0.1751	0.8593	0.7752	0.8151	0.6879	0.9677	0.8812	0.9521
0.0604	1.64	270	0.1693	0.8155	0.8116	0.8136	0.6857	0.9658	0.8965	0.9503
0.043	1.7	280	0.1939	0.8106	0.8264	0.8184	0.6926	0.9663	0.9034	0.9490
0.0538	1.76	290	0.1800	0.8320	0.8146	0.8232	0.6996	0.9678	0.8990	0.9515
0.0557	1.82	300	0.1762	0.7867	0.8289	0.8073	0.6768	0.9636	0.9031	0.9473
0.0493	1.88	310	0.1646	0.8411	0.7894	0.8144	0.6870	0.9669	0.8872	0.9519
0.0524	1.94	320	0.1674	0.8155	0.8205	0.8180	0.6921	0.9664	0.9009	0.9507
0.0445	2.0	330	0.1616	0.8186	0.8075	0.8130	0.6849	0.9658	0.8947	0.9511
0.0399	2.06	340	0.2141	0.7818	0.8352	0.8076	0.6773	0.9634	0.9058	0.9455
0.039	2.12	350	0.1765	0.8189	0.8134	0.8161	0.6894	0.9663	0.8976	0.9509
0.0353	2.18	360	0.1900	0.8097	0.8130	0.8113	0.6826	0.9652	0.8968	0.9495
0.0269	2.24	370	0.2080	0.8338	0.7953	0.8141	0.6865	0.9666	0.8896	0.9508
0.0333	2.3	380	0.2032	0.7753	0.8341	0.8036	0.6717	0.9625	0.9048	0.9466
0.0339	2.36	390	0.1867	0.8309	0.8045	0.8175	0.6913	0.9670	0.8939	0.9520
0.035	2.42	400	0.1826	0.8078	0.8213	0.8145	0.6870	0.9656	0.9007	0.9500
0.039	2.48	410	0.2028	0.7762	0.8335	0.8038	0.6720	0.9626	0.9046	0.9466
0.0303	2.55	420	0.1964	0.8096	0.8153	0.8124	0.6841	0.9654	0.8979	0.9495
0.0296	2.61	430	0.1908	0.7883	0.8349	0.8109	0.6820	0.9642	0.9061	0.9485
0.0356	2.67	440	0.2016	0.8151	0.8140	0.8145	0.6871	0.9659	0.8977	0.9497
0.0343	2.73	450	0.1963	0.8016	0.8248	0.8130	0.6850	0.9651	0.9021	0.9497
0.0271	2.79	460	0.2077	0.8172	0.8127	0.8150	0.6877	0.9661	0.8972	0.9500
0.0311	2.85	470	0.1889	0.8080	0.8254	0.8166	0.6900	0.9659	0.9027	0.9512
0.029	2.91	480	0.1995	0.7698	0.8443	0.8053	0.6741	0.9625	0.9094	0.9469
0.0265	2.97	490	0.1828	0.8245	0.8148	0.8196	0.6944	0.9670	0.8986	0.9528
0.0281	3.03	500	0.2061	0.8321	0.8070	0.8194	0.6940	0.9673	0.8953	0.9520
0.0273	3.09	510	0.2072	0.8105	0.8218	0.8161	0.6894	0.9659	0.9012	0.9502
0.0268	3.15	520	0.2113	0.8084	0.8222	0.8152	0.6881	0.9657	0.9012	0.9502
0.0241	3.21	530	0.2015	0.8318	0.8038	0.8176	0.6914	0.9670	0.8937	0.9524
0.0198	3.27	540	0.2252	0.7922	0.8331	0.8122	0.6837	0.9646	0.9055	0.9483
0.0268	3.33	550	0.2107	0.7931	0.8343	0.8132	0.6852	0.9647	0.9061	0.9493
0.0185	3.39	560	0.2202	0.7874	0.8311	0.8086	0.6787	0.9638	0.9042	0.9485
0.0224	3.45	570	0.2256	0.8057	0.8247	0.8151	0.6879	0.9656	0.9023	0.9497
0.0189	3.52	580	0.2125	0.8111	0.8204	0.8157	0.6888	0.9659	0.9005	0.9504
0.026	3.58	590	0.2163	0.8122	0.8182	0.8152	0.6881	0.9659	0.8995	0.9506
0.0184	3.64	600	0.2143	0.8057	0.8204	0.8130	0.6849	0.9653	0.9002	0.9502
0.0196	3.7	610	0.2113	0.8171	0.8164	0.8167	0.6903	0.9663	0.8990	0.9509
0.0205	3.76	620	0.2081	0.8127	0.8213	0.8170	0.6906	0.9662	0.9011	0.9512
0.0231	3.82	630	0.2200	0.8170	0.8225	0.8198	0.6946	0.9667	0.9019	0.9508
0.0209	3.88	640	0.2179	0.8148	0.8261	0.8204	0.6955	0.9667	0.9035	0.9512
0.0164	3.94	650	0.2201	0.8338	0.8124	0.8229	0.6992	0.9679	0.8980	0.9522
0.0222	4.0	660	0.2174	0.8193	0.8124	0.8158	0.6890	0.9663	0.8971	0.9505
0.0209	4.06	670	0.2169	0.7975	0.8336	0.8151	0.6880	0.9652	0.9061	0.9502
0.0165	4.12	680	0.2248	0.7973	0.8317	0.8141	0.6865	0.9651	0.9052	0.9494
0.0136	4.18	690	0.2254	0.8073	0.8245	0.8158	0.6889	0.9658	0.9023	0.9501
0.016	4.24	700	0.2259	0.8029	0.8231	0.8129	0.6847	0.9651	0.9013	0.9499
0.018	4.3	710	0.2291	0.8013	0.8264	0.8136	0.6858	0.9652	0.9028	0.9496
0.0148	4.36	720	0.2316	0.7923	0.8283	0.8099	0.6806	0.9642	0.9031	0.9483
0.0163	4.42	730	0.2423	0.7882	0.8321	0.8096	0.6801	0.9640	0.9047	0.9475
0.0162	4.48	740	0.2312	0.8042	0.8243	0.8141	0.6865	0.9654	0.9020	0.9496
0.0121	4.55	750	0.2355	0.8003	0.8257	0.8128	0.6847	0.9650	0.9024	0.9494
0.0182	4.61	760	0.2393	0.7911	0.8313	0.8107	0.6817	0.9643	0.9046	0.9485
0.0182	4.67	770	0.2292	0.7954	0.8308	0.8127	0.6845	0.9648	0.9046	0.9494
0.0153	4.73	780	0.2312	0.8043	0.8266	0.8153	0.6882	0.9656	0.9031	0.9498
0.0182	4.79	790	0.2410	0.7953	0.8326	0.8135	0.6857	0.9649	0.9055	0.9489
0.0133	4.85	800	0.2350	0.8110	0.8219	0.8164	0.6898	0.9660	0.9013	0.9505
0.0185	4.91	810	0.2432	0.7912	0.8363	0.8132	0.6851	0.9647	0.9070	0.9486
0.0157	4.97	820	0.2318	0.8103	0.8214	0.8158	0.6889	0.9659	0.9010	0.9503
0.0171	5.03	830	0.2392	0.7992	0.8294	0.8140	0.6864	0.9651	0.9042	0.9492
0.0121	5.09	840	0.2422	0.8043	0.8271	0.8155	0.6885	0.9656	0.9034	0.9497
0.0103	5.15	850	0.2382	0.8009	0.8296	0.8150	0.6877	0.9654	0.9043	0.9495
0.0148	5.21	860	0.2358	0.8057	0.8256	0.8155	0.6885	0.9656	0.9027	0.9501
0.0115	5.27	870	0.2398	0.8008	0.8290	0.8147	0.6873	0.9653	0.9041	0.9497
0.0115	5.33	880	0.2385	0.8035	0.8245	0.8138	0.6861	0.9653	0.9020	0.9496
0.0122	5.39	890	0.2387	0.7983	0.8303	0.8140	0.6863	0.9651	0.9045	0.9494
0.0134	5.45	900	0.2395	0.8009	0.8279	0.8142	0.6866	0.9652	0.9035	0.9495
0.0169	5.52	910	0.2391	0.8012	0.8282	0.8145	0.6870	0.9653	0.9037	0.9496
0.0178	5.58	920	0.2403	0.7997	0.8290	0.8141	0.6865	0.9652	0.9040	0.9493
0.0162	5.64	930	0.2391	0.8017	0.8274	0.8143	0.6868	0.9653	0.9033	0.9496
0.0115	5.7	940	0.2384	0.8035	0.8265	0.8148	0.6875	0.9655	0.9030	0.9498
0.0175	5.76	950	0.2386	0.8029	0.8282	0.8153	0.6882	0.9655	0.9038	0.9498
0.0165	5.82	960	0.2390	0.8017	0.8288	0.8150	0.6878	0.9654	0.9040	0.9497
0.0186	5.88	970	0.2389	0.8021	0.8289	0.8153	0.6882	0.9655	0.9041	0.9498
0.0124	5.94	980	0.2390	0.8024	0.8288	0.8154	0.6883	0.9655	0.9041	0.9498
0.0138	6.0	990	0.2391	0.8025	0.8289	0.8155	0.6885	0.9655	0.9041	0.9498
0.0113	6.06	1000	0.2391	0.8025	0.8289	0.8155	0.6885	0.9655	0.9041	0.9498

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1
• Datasets 2.14.6
• Tokenizers 0.14.1