nl_electra

This model is a pretrained version of ELECTRA on the Dutch subset of the CC100 dataset. It achieves the following results on the evaluation set:

• Loss: 2.4650
• Accuracy: 0.5392

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 703
• gradient_accumulation_steps: 32
• total_train_batch_size: 512
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 8000
• num_epochs: 400.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.67	500	9.9977	0.0486
No log	1.35	1000	9.5620	0.0543
No log	2.02	1500	8.9306	0.0741
No log	2.69	2000	8.2617	0.0826
No log	3.36	2500	7.6880	0.0792
No log	4.04	3000	7.3316	0.0757
No log	4.71	3500	7.1944	0.0747
No log	5.38	4000	7.1349	0.0802
No log	6.06	4500	7.0752	0.0887
8.201	6.73	5000	7.0046	0.1021
8.201	7.4	5500	6.9295	0.1090
8.201	8.08	6000	6.8483	0.1132
8.201	8.75	6500	6.7750	0.1171
8.201	9.42	7000	6.7116	0.1187
8.201	10.09	7500	6.6560	0.1218
8.201	10.77	8000	6.6178	0.1239
8.201	11.44	8500	6.5824	0.1255
8.201	12.11	9000	6.5521	0.1273
8.201	12.79	9500	6.5203	0.1292
6.7257	13.46	10000	6.5027	0.1303
6.7257	14.13	10500	6.4809	0.1314
6.7257	14.8	11000	6.4631	0.1322
6.7257	15.48	11500	6.4483	0.1329
6.7257	16.15	12000	6.4320	0.1338
6.7257	16.82	12500	6.4169	0.1348
6.7257	17.5	13000	6.4067	0.1359
6.7257	18.17	13500	6.3994	0.1359
6.7257	18.84	14000	6.3823	0.1368
6.7257	19.52	14500	6.3759	0.1373
6.4502	20.19	15000	6.3629	0.1374
6.4502	20.86	15500	6.3638	0.1373
6.4502	21.53	16000	6.3505	0.1382
6.4502	22.21	16500	6.3416	0.1387
6.4502	22.88	17000	6.3420	0.1383
6.4502	23.55	17500	6.3330	0.1389
6.4502	24.23	18000	6.3289	0.1388
6.4502	24.9	18500	6.3184	0.1389
6.4502	25.57	19000	6.3099	0.1396
6.4502	26.24	19500	6.2789	0.1405
6.3474	26.92	20000	6.2398	0.1404
6.3474	27.59	20500	6.2012	0.1412
6.3474	28.26	21000	6.1803	0.1414
6.3474	28.94	21500	6.1579	0.1414
6.3474	29.61	22000	6.1403	0.1431
6.3474	30.28	22500	6.1316	0.1423
6.3474	30.96	23000	6.1102	0.1435
6.3474	31.63	23500	6.0998	0.1439
6.3474	32.3	24000	6.0867	0.1446
6.3474	32.97	24500	6.0700	0.1451
6.1758	33.65	25000	6.0554	0.1457
6.1758	34.32	25500	6.0487	0.1457
6.1758	34.99	26000	6.0328	0.1469
6.1758	35.67	26500	6.0265	0.1469
6.1758	36.34	27000	5.9992	0.1486
6.1758	37.01	27500	5.9934	0.1485
6.1758	37.68	28000	5.9702	0.1501
6.1758	38.36	28500	5.9661	0.1503
6.1758	39.03	29000	5.9558	0.1512
6.1758	39.7	29500	5.9321	0.1528
6.052	40.38	30000	5.9147	0.1532
6.052	41.05	30500	5.8975	0.1545
6.052	41.72	31000	5.8784	0.1566
6.052	42.4	31500	5.8584	0.1586
6.052	43.07	32000	5.8325	0.1603
6.052	43.74	32500	5.7583	0.1664
6.052	44.41	33000	5.6158	0.1787
6.052	45.09	33500	5.4580	0.1917
6.052	45.76	34000	5.3396	0.2010
6.052	46.43	34500	5.2568	0.2082
5.7995	47.11	35000	5.1775	0.2146
5.7995	47.78	35500	5.1076	0.2204
5.7995	48.45	36000	5.0457	0.2258
5.7995	49.13	36500	4.9932	0.2313
5.7995	49.8	37000	4.9164	0.2384
5.7995	50.47	37500	4.7844	0.2521
5.7995	51.14	38000	4.6598	0.2642
5.7995	51.82	38500	4.5472	0.2757
5.7995	52.49	39000	4.4374	0.2871
5.7995	53.16	39500	4.3399	0.2982
5.0341	53.84	40000	4.2549	0.3083
5.0341	54.51	40500	4.1795	0.3170
5.0341	55.18	41000	4.1017	0.3274
5.0341	55.85	41500	4.0308	0.3375
5.0341	56.53	42000	3.9673	0.3462
5.0341	57.2	42500	3.9130	0.3538
5.0341	57.87	43000	3.8672	0.3599
5.0341	58.55	43500	3.8249	0.3656
5.0341	59.22	44000	3.7748	0.3728
5.0341	59.89	44500	3.7459	0.3768
4.2119	60.57	45000	3.7089	0.3808
4.2119	61.24	45500	3.6732	0.3857
4.2119	61.91	46000	3.6546	0.3881
4.2119	62.58	46500	3.6205	0.3921
4.2119	63.26	47000	3.5908	0.3960
4.2119	63.93	47500	3.5627	0.3991
4.2119	64.6	48000	3.5466	0.4019
4.2119	65.28	48500	3.5262	0.4039
4.2119	65.95	49000	3.4987	0.4074
4.2119	66.62	49500	3.4817	0.4093
3.8182	67.29	50000	3.4608	0.4119
3.8182	67.97	50500	3.4467	0.4142
3.8182	68.64	51000	3.4280	0.4163
3.8182	69.31	51500	3.4165	0.4175
3.8182	69.99	52000	3.3970	0.4199
3.8182	70.66	52500	3.3738	0.4227
3.8182	71.33	53000	3.3640	0.4242
3.8182	72.01	53500	3.3583	0.4252
3.8182	72.68	54000	3.3319	0.4279
3.8182	73.35	54500	3.3153	0.4303
3.5946	74.02	55000	3.3098	0.4304
3.5946	74.7	55500	3.2949	0.4328
3.5946	75.37	56000	3.2820	0.4335
3.5946	76.04	56500	3.2686	0.4355
3.5946	76.72	57000	3.2663	0.4359
3.5946	77.39	57500	3.2482	0.4379
3.5946	78.06	58000	3.2344	0.4393
3.5946	78.73	58500	3.2281	0.4407
3.5946	79.41	59000	3.2172	0.4412
3.5946	80.08	59500	3.2110	0.4420
3.4435	80.75	60000	3.1940	0.4443
3.4435	81.43	60500	3.1837	0.4455
3.4435	82.1	61000	3.1744	0.4469
3.4435	82.77	61500	3.1611	0.4483
3.4435	83.45	62000	3.1531	0.4496
3.4435	84.12	62500	3.1524	0.4499
3.4435	84.79	63000	3.1431	0.4501
3.4435	85.46	63500	3.1287	0.4527
3.4435	86.14	64000	3.1192	0.4533
3.4435	86.81	64500	3.1107	0.4547
3.3301	87.48	65000	3.1041	0.4553
3.3301	88.16	65500	3.0999	0.4562
3.3301	88.83	66000	3.0882	0.4576
3.3301	89.5	66500	3.0777	0.4589
3.3301	90.17	67000	3.0726	0.4588
3.3301	90.85	67500	3.0676	0.4601
3.3301	91.52	68000	3.0616	0.4602
3.3301	92.19	68500	3.0523	0.4621
3.3301	92.87	69000	3.0464	0.4624
3.3301	93.54	69500	3.0405	0.4635
3.2418	94.21	70000	3.0312	0.4649
3.2418	94.89	70500	3.0209	0.4653
3.2418	95.56	71000	3.0202	0.4657
3.2418	96.23	71500	3.0101	0.4676
3.2418	96.9	72000	3.0105	0.4666
3.2418	97.58	72500	3.0023	0.4685
3.2418	98.25	73000	3.0008	0.4680
3.2418	98.92	73500	2.9882	0.4691
3.2418	99.6	74000	2.9855	0.4702
3.2418	100.27	74500	2.9790	0.4709
3.1698	100.94	75000	2.9680	0.4716
3.1698	101.61	75500	2.9667	0.4724
3.1698	102.29	76000	2.9657	0.4726
3.1698	102.96	76500	2.9623	0.4731
3.1698	103.63	77000	2.9515	0.4745
3.1698	104.31	77500	2.9471	0.4753
3.1698	104.98	78000	2.9407	0.4756
3.1698	105.65	78500	2.9388	0.4761
3.1698	106.33	79000	2.9369	0.4766
3.1698	107.0	79500	2.9297	0.4762
3.1101	107.67	80000	2.9291	0.4776
3.1101	108.34	80500	2.9139	0.4788
3.1101	109.02	81000	2.9113	0.4790
3.1101	109.69	81500	2.9057	0.4798
3.1101	110.36	82000	2.9058	0.4804
3.1101	111.04	82500	2.9019	0.4807
3.1101	111.71	83000	2.8934	0.4818
3.1101	112.38	83500	2.8864	0.4825
3.1101	113.06	84000	2.8926	0.4815
3.1101	113.73	84500	2.8812	0.4830
3.058	114.4	85000	2.8740	0.4840
3.058	115.07	85500	2.8730	0.4840
3.058	115.75	86000	2.8694	0.4847
3.058	116.42	86500	2.8694	0.4848
3.058	117.09	87000	2.8601	0.4862
3.058	117.77	87500	2.8547	0.4862
3.058	118.44	88000	2.8538	0.4861
3.058	119.11	88500	2.8494	0.4876
3.058	119.78	89000	2.8430	0.4882
3.058	120.46	89500	2.8436	0.4875
3.0129	121.13	90000	2.8402	0.4884
3.0129	121.8	90500	2.8353	0.4888
3.0129	122.48	91000	2.8271	0.4896
3.0129	123.15	91500	2.8236	0.4900
3.0129	123.82	92000	2.8199	0.4913
3.0129	124.5	92500	2.8119	0.4916
3.0129	125.17	93000	2.8138	0.4916
3.0129	125.84	93500	2.8089	0.4925
3.0129	126.51	94000	2.8067	0.4928
3.0129	127.19	94500	2.8010	0.4939
2.9701	127.86	95000	2.7992	0.4938
2.9701	128.53	95500	2.7953	0.4948
2.9701	129.21	96000	2.7964	0.4942
2.9701	129.88	96500	2.7838	0.4955
2.9701	130.55	97000	2.7770	0.4968
2.9701	131.22	97500	2.7800	0.4962
2.9701	131.9	98000	2.7743	0.4972
2.9701	132.57	98500	2.7696	0.4973
2.9701	133.24	99000	2.7691	0.4980
2.9701	133.92	99500	2.7612	0.4989
2.9289	134.59	100000	2.7606	0.4987
2.9289	135.26	100500	2.7545	0.4993
2.9289	135.94	101000	2.7544	0.4999
2.9289	136.61	101500	2.7550	0.4999
2.9289	137.28	102000	2.7510	0.5001
2.9289	137.95	102500	2.7480	0.5002
2.9289	138.63	103000	2.7422	0.5012
2.9289	139.3	103500	2.7419	0.5014
2.9289	139.97	104000	2.7416	0.5009
2.9289	140.65	104500	2.7412	0.5017
2.8968	141.32	105000	2.7356	0.5023
2.8968	141.99	105500	2.7303	0.5027
2.8968	142.66	106000	2.7359	0.5029
2.8968	143.34	106500	2.7283	0.5032
2.8968	144.01	107000	2.7226	0.5033
2.8968	144.68	107500	2.7247	0.5039
2.8968	145.36	108000	2.7209	0.5044
2.8968	146.03	108500	2.7210	0.5039
2.8968	146.7	109000	2.7135	0.5046
2.8968	147.38	109500	2.7139	0.5048
2.8697	148.05	110000	2.7167	0.5050
2.8697	148.72	110500	2.7125	0.5058
2.8697	149.39	111000	2.7064	0.5060
2.8697	150.07	111500	2.7024	0.5067
2.8697	150.74	112000	2.7035	0.5067
2.8697	151.41	112500	2.7034	0.5067
2.8697	152.09	113000	2.6967	0.5073
2.8697	152.76	113500	2.6982	0.5070
2.8697	153.43	114000	2.6948	0.5079
2.8697	154.1	114500	2.6946	0.5076
2.8457	154.78	115000	2.6918	0.5078
2.8457	155.45	115500	2.6917	0.5078
2.8457	156.12	116000	2.6868	0.5086
2.8457	156.8	116500	2.6870	0.5084
2.8457	157.47	117000	2.6830	0.5091
2.8457	158.14	117500	2.6824	0.5090
2.8457	158.82	118000	2.6812	0.5092
2.8457	159.49	118500	2.6747	0.5098
2.8457	160.16	119000	2.6747	0.5105
2.8457	160.83	119500	2.6750	0.5102
2.825	161.51	120000	2.6761	0.5102
2.825	162.18	120500	2.6670	0.5115
2.825	162.85	121000	2.6740	0.5104
2.825	163.53	121500	2.6700	0.5106
2.825	164.2	122000	2.6629	0.5119
2.825	164.87	122500	2.6642	0.5117
2.825	165.54	123000	2.6664	0.5109
2.825	166.22	123500	2.6626	0.5117
2.825	166.89	124000	2.6561	0.5130
2.825	167.56	124500	2.6612	0.5125
2.8059	168.24	125000	2.6594	0.5123
2.8059	168.91	125500	2.6508	0.5132
2.8059	169.58	126000	2.6477	0.5134
2.8059	170.26	126500	2.6527	0.5133
2.8059	170.93	127000	2.6519	0.5136
2.8059	171.6	127500	2.6456	0.5141
2.8059	172.27	128000	2.6473	0.5139
2.8059	172.95	128500	2.6426	0.5144
2.8059	173.62	129000	2.6454	0.5137
2.8059	174.29	129500	2.6427	0.5147
2.788	174.97	130000	2.6376	0.5150
2.788	175.64	130500	2.6366	0.5154
2.788	176.31	131000	2.6327	0.5156
2.788	176.98	131500	2.6328	0.5157
2.788	177.66	132000	2.6335	0.5156
2.788	178.33	132500	2.6302	0.5166
2.788	179.0	133000	2.6333	0.5160
2.788	179.68	133500	2.6253	0.5171
2.788	180.35	134000	2.6237	0.5167
2.788	181.02	134500	2.6269	0.5165
2.7723	181.7	135000	2.6283	0.5164
2.7723	182.37	135500	2.6255	0.5174
2.7723	183.04	136000	2.6254	0.5175
2.7723	183.71	136500	2.6231	0.5172
2.7723	184.39	137000	2.6181	0.5173
2.7723	185.06	137500	2.6260	0.5168
2.7723	185.73	138000	2.6183	0.5176
2.7723	186.41	138500	2.6174	0.5182
2.7723	187.08	139000	2.6144	0.5182
2.7723	187.75	139500	2.6152	0.5186
2.7575	188.43	140000	2.6150	0.5183
2.7575	189.1	140500	2.6110	0.5190
2.7575	189.77	141000	2.6044	0.5194
2.7575	190.44	141500	2.6083	0.5186
2.7575	191.12	142000	2.6102	0.5189
2.7575	191.79	142500	2.6063	0.5195
2.7575	192.46	143000	2.6071	0.5198
2.7575	193.14	143500	2.6090	0.5191
2.7575	193.81	144000	2.6068	0.5200
2.7575	194.48	144500	2.6032	0.5200
2.7445	195.15	145000	2.6031	0.5200
2.7445	195.83	145500	2.6007	0.5201
2.7445	196.5	146000	2.5998	0.5203
2.7445	197.17	146500	2.5980	0.5208
2.7445	197.85	147000	2.5952	0.5211
2.7445	198.52	147500	2.5977	0.5210
2.7445	199.19	148000	2.5922	0.5212
2.7445	199.87	148500	2.5936	0.5211
2.7445	200.54	149000	2.5933	0.5219
2.7445	201.21	149500	2.5887	0.5219
2.7324	201.88	150000	2.5911	0.5215
2.7324	202.56	150500	2.5892	0.5219
2.7324	203.23	151000	2.5875	0.5218
2.7324	203.9	151500	2.5898	0.5220
2.7324	204.58	152000	2.5872	0.5223
2.7324	205.25	152500	2.5805	0.5226
2.7324	205.92	153000	2.5861	0.5225
2.7324	206.59	153500	2.5839	0.5223
2.7324	207.27	154000	2.5804	0.5232
2.7324	207.94	154500	2.5766	0.5235
2.7212	208.61	155000	2.5764	0.5233
2.7212	209.29	155500	2.5801	0.5233
2.7212	209.96	156000	2.5737	0.5241
2.7212	210.63	156500	2.5769	0.5236
2.7212	211.31	157000	2.5769	0.5237
2.7212	211.98	157500	2.5748	0.5239
2.7212	212.65	158000	2.5745	0.5230
2.7212	213.32	158500	2.5725	0.5240
2.7212	214.0	159000	2.5736	0.5239
2.7212	214.67	159500	2.5675	0.5252
2.7103	215.34	160000	2.5678	0.5245
2.7103	216.02	160500	2.5691	0.5250
2.7103	216.69	161000	2.5688	0.5245
2.7103	217.36	161500	2.5681	0.5251
2.7103	218.03	162000	2.5582	0.5255
2.7103	218.71	162500	2.5675	0.5247
2.7103	219.38	163000	2.5609	0.5259
2.7103	220.05	163500	2.5625	0.5254
2.7103	220.73	164000	2.5630	0.5254
2.7103	221.4	164500	2.5607	0.5265
2.7003	222.07	165000	2.5615	0.5260
2.7003	222.75	165500	2.5660	0.5248
2.7003	223.42	166000	2.5569	0.5263
2.7003	224.09	166500	2.5610	0.5255
2.7003	224.76	167000	2.5569	0.5263
2.7003	225.44	167500	2.5534	0.5265
2.7003	226.11	168000	2.5573	0.5259
2.7003	226.78	168500	2.5559	0.5264
2.7003	227.46	169000	2.5508	0.5277
2.7003	228.13	169500	2.5500	0.5276
2.6915	228.8	170000	2.5501	0.5270
2.6915	229.47	170500	2.5508	0.5273
2.6915	230.15	171000	2.5523	0.5267
2.6915	230.82	171500	2.5464	0.5276
2.6915	231.49	172000	2.5482	0.5271
2.6915	232.17	172500	2.5486	0.5270
2.6915	232.84	173000	2.5474	0.5275
2.6915	233.51	173500	2.5483	0.5270
2.6915	234.19	174000	2.5480	0.5276
2.6915	234.86	174500	2.5486	0.5278
2.6833	235.53	175000	2.5484	0.5273
2.6833	236.2	175500	2.5436	0.5277
2.6833	236.88	176000	2.5448	0.5278
2.6833	237.55	176500	2.5430	0.5284
2.6833	238.22	177000	2.5433	0.5279
2.6833	238.9	177500	2.5398	0.5288
2.6833	239.57	178000	2.5424	0.5282
2.6833	240.24	178500	2.5371	0.5291
2.6833	240.91	179000	2.5372	0.5294
2.6833	241.59	179500	2.5368	0.5290
2.6753	242.26	180000	2.5383	0.5289
2.6753	242.93	180500	2.5387	0.5289
2.6753	243.61	181000	2.5351	0.5295
2.6753	244.28	181500	2.5340	0.5296
2.6753	244.95	182000	2.5349	0.5289
2.6753	245.63	182500	2.5358	0.5295
2.6753	246.3	183000	2.5333	0.5299
2.6753	246.97	183500	2.5363	0.5292
2.6753	247.64	184000	2.5323	0.5298
2.6753	248.32	184500	2.5286	0.5299
2.6679	248.99	185000	2.5276	0.5306
2.6679	249.66	185500	2.5249	0.5308
2.6679	250.34	186000	2.5308	0.5302
2.6679	251.01	186500	2.5307	0.5297
2.6679	251.68	187000	2.5293	0.5305
2.6679	252.36	187500	2.5255	0.5306
2.6679	253.03	188000	2.5244	0.5312
2.6679	253.7	188500	2.5278	0.5305
2.6679	254.37	189000	2.5212	0.5317
2.6679	255.05	189500	2.5256	0.5307
2.6611	255.72	190000	2.5273	0.5303
2.6611	256.39	190500	2.5222	0.5310
2.6611	257.07	191000	2.5237	0.5311
2.6611	257.74	191500	2.5258	0.5309
2.6611	258.41	192000	2.5219	0.5313
2.6611	259.08	192500	2.5243	0.5314
2.6611	259.76	193000	2.5203	0.5319
2.6611	260.43	193500	2.5205	0.5313
2.6611	261.1	194000	2.5205	0.5322
2.6611	261.78	194500	2.5196	0.5317
2.655	262.45	195000	2.5199	0.5315
2.655	263.12	195500	2.5226	0.5315
2.655	263.8	196000	2.5175	0.5316
2.655	264.47	196500	2.5160	0.5322
2.655	265.14	197000	2.5185	0.5316
2.655	265.81	197500	2.5133	0.5322
2.655	266.49	198000	2.5163	0.5318
2.655	267.16	198500	2.5135	0.5325
2.655	267.83	199000	2.5132	0.5326
2.655	268.51	199500	2.5148	0.5323
2.6486	269.18	200000	2.5194	0.5317
2.6486	269.85	200500	2.5162	0.5321
2.6486	270.52	201000	2.5090	0.5332
2.6486	271.2	201500	2.5126	0.5325
2.6486	271.87	202000	2.5155	0.5320
2.6486	272.54	202500	2.5099	0.5329
2.6486	273.22	203000	2.5130	0.5325
2.6486	273.89	203500	2.5064	0.5329
2.6486	274.56	204000	2.5154	0.5319
2.6486	275.24	204500	2.5097	0.5329
2.6433	275.91	205000	2.5075	0.5334
2.6433	276.58	205500	2.5120	0.5325
2.6433	277.25	206000	2.5100	0.5329
2.6433	277.93	206500	2.5115	0.5332
2.6433	278.6	207000	2.5071	0.5332
2.6433	279.27	207500	2.5075	0.5335
2.6433	279.95	208000	2.5020	0.5338
2.6433	280.62	208500	2.5025	0.5340
2.6433	281.29	209000	2.5064	0.5333
2.6433	281.96	209500	2.5038	0.5336
2.6383	282.64	210000	2.5041	0.5340
2.6383	283.31	210500	2.5075	0.5336
2.6383	283.98	211000	2.5028	0.5333
2.6383	284.66	211500	2.5008	0.5340
2.6383	285.33	212000	2.5005	0.5345
2.6383	286.0	212500	2.5020	0.5334
2.6383	286.68	213000	2.5011	0.5344
2.6383	287.35	213500	2.5028	0.5338
2.6383	288.02	214000	2.4970	0.5340
2.6383	288.69	214500	2.4995	0.5343
2.6336	289.37	215000	2.5010	0.5343
2.6336	290.04	215500	2.5060	0.5336
2.6336	290.71	216000	2.4955	0.5347
2.6336	291.39	216500	2.4972	0.5349
2.6336	292.06	217000	2.4977	0.5349
2.6336	292.73	217500	2.4973	0.5346
2.6336	293.4	218000	2.4981	0.5346
2.6336	294.08	218500	2.4941	0.5346
2.6336	294.75	219000	2.4978	0.5350
2.6336	295.42	219500	2.4990	0.5343
2.6288	296.1	220000	2.4929	0.5347
2.6288	296.77	220500	2.4937	0.5349
2.6288	297.44	221000	2.4938	0.5349
2.6288	298.12	221500	2.4938	0.5343
2.6288	298.79	222000	2.4924	0.5354
2.6288	299.46	222500	2.4953	0.5348
2.6288	300.13	223000	2.4930	0.5347
2.6288	300.81	223500	2.4934	0.5353
2.6288	301.48	224000	2.4942	0.5348
2.6288	302.15	224500	2.4960	0.5344
2.6246	302.83	225000	2.4875	0.5357
2.6246	303.5	225500	2.4898	0.5355
2.6246	304.17	226000	2.4847	0.5366
2.6246	304.84	226500	2.4970	0.5348
2.6246	305.52	227000	2.4905	0.5356
2.6246	306.19	227500	2.4873	0.5361
2.6246	306.86	228000	2.4939	0.5350
2.6246	307.54	228500	2.4910	0.5360
2.6246	308.21	229000	2.4886	0.5355
2.6246	308.88	229500	2.4890	0.5369
2.6207	309.56	230000	2.4900	0.5360
2.6207	310.23	230500	2.4885	0.5354
2.6207	310.9	231000	2.4895	0.5358
2.6207	311.57	231500	2.4871	0.5358
2.6207	312.25	232000	2.4914	0.5352
2.6207	312.92	232500	2.4843	0.5366
2.6207	313.59	233000	2.4837	0.5365
2.6207	314.27	233500	2.4883	0.5360
2.6207	314.94	234000	2.4839	0.5366
2.6207	315.61	234500	2.4854	0.5366
2.6171	316.29	235000	2.4833	0.5367
2.6171	316.96	235500	2.4783	0.5374
2.6171	317.63	236000	2.4807	0.5370
2.6171	318.3	236500	2.4824	0.5366
2.6171	318.98	237000	2.4857	0.5361
2.6171	319.65	237500	2.4817	0.5366
2.6171	320.32	238000	2.4855	0.5364
2.6171	321.0	238500	2.4834	0.5367
2.6171	321.67	239000	2.4831	0.5363
2.6171	322.34	239500	2.4806	0.5370
2.6134	323.01	240000	2.4842	0.5365
2.6134	323.69	240500	2.4830	0.5364
2.6134	324.36	241000	2.4822	0.5367
2.6134	325.03	241500	2.4805	0.5373
2.6134	325.71	242000	2.4838	0.5365
2.6134	326.38	242500	2.4776	0.5371
2.6134	327.05	243000	2.4786	0.5376
2.6134	327.73	243500	2.4824	0.5371
2.6134	328.4	244000	2.4842	0.5363
2.6134	329.07	244500	2.4790	0.5375
2.6107	329.74	245000	2.4770	0.5372
2.6107	330.42	245500	2.4805	0.5375
2.6107	331.09	246000	2.4839	0.5370
2.6107	331.76	246500	2.4802	0.5367
2.6107	332.44	247000	2.4796	0.5373
2.6107	333.11	247500	2.4736	0.5377
2.6107	333.78	248000	2.4789	0.5374
2.6107	334.45	248500	2.4761	0.5375
2.6107	335.13	249000	2.4728	0.5379
2.6107	335.8	249500	2.4702	0.5386
2.608	336.47	250000	2.4764	0.5377
2.608	337.15	250500	2.4738	0.5380
2.608	337.82	251000	2.4795	0.5371
2.608	338.49	251500	2.4702	0.5387
2.608	339.17	252000	2.4823	0.5369
2.608	339.84	252500	2.4741	0.5382
2.608	340.51	253000	2.4718	0.5382
2.608	341.18	253500	2.4731	0.5378
2.608	341.86	254000	2.4780	0.5373
2.608	342.53	254500	2.4706	0.5388
2.6058	343.2	255000	2.4707	0.5386
2.6058	343.88	255500	2.4725	0.5380
2.6058	344.55	256000	2.4744	0.5382
2.6058	345.22	256500	2.4766	0.5374
2.6058	345.89	257000	2.4736	0.5378
2.6058	346.57	257500	2.4731	0.5383
2.6058	347.24	258000	2.4754	0.5377
2.6058	347.91	258500	2.4749	0.5382
2.6058	348.59	259000	2.4735	0.5378
2.6058	349.26	259500	2.4716	0.5384
2.6027	349.93	260000	2.4726	0.5378
2.6027	350.61	260500	2.4733	0.5378
2.6027	351.28	261000	2.4698	0.5386
2.6027	351.95	261500	2.4702	0.5388
2.6027	352.62	262000	2.4673	0.5390
2.6027	353.3	262500	2.4683	0.5390
2.6027	353.97	263000	2.4739	0.5379
2.6027	354.64	263500	2.4743	0.5382
2.6027	355.32	264000	2.4694	0.5388
2.6027	355.99	264500	2.4671	0.5391
2.6009	356.66	265000	2.4747	0.5383
2.6009	357.34	265500	2.4703	0.5382
2.6009	358.01	266000	2.4695	0.5388
2.6009	358.68	266500	2.4725	0.5380
2.6009	359.35	267000	2.4639	0.5397
2.6009	360.03	267500	2.4686	0.5385
2.6009	360.7	268000	2.4698	0.5386
2.6009	361.37	268500	2.4694	0.5386
2.6009	362.05	269000	2.4680	0.5390
2.6009	362.72	269500	2.4728	0.5383
2.5989	363.39	270000	2.4697	0.5385
2.5989	364.06	270500	2.4701	0.5387
2.5989	364.74	271000	2.4702	0.5387
2.5989	365.41	271500	2.4687	0.5390
2.5989	366.08	272000	2.4725	0.5382
2.5989	366.76	272500	2.4673	0.5384
2.5989	367.43	273000	2.4659	0.5390
2.5989	368.1	273500	2.4686	0.5389
2.5989	368.78	274000	2.4677	0.5382
2.5989	369.45	274500	2.4632	0.5389
2.5977	370.12	275000	2.4676	0.5385
2.5977	370.79	275500	2.4699	0.5388
2.5977	371.47	276000	2.4629	0.5394
2.5977	372.14	276500	2.4720	0.5380
2.5977	372.81	277000	2.4678	0.5391
2.5977	373.49	277500	2.4643	0.5396
2.5977	374.16	278000	2.4654	0.5395
2.5977	374.83	278500	2.4645	0.5395
2.5977	375.5	279000	2.4649	0.5391
2.5977	376.18	279500	2.4639	0.5392
2.5961	376.85	280000	2.4659	0.5389
2.5961	377.52	280500	2.4681	0.5385
2.5961	378.2	281000	2.4641	0.5390
2.5961	378.87	281500	2.4658	0.5393
2.5961	379.54	282000	2.4687	0.5388
2.5961	380.22	282500	2.4690	0.5385
2.5961	380.89	283000	2.4679	0.5391
2.5961	381.56	283500	2.4612	0.5395
2.5961	382.23	284000	2.4624	0.5395
2.5961	382.91	284500	2.4668	0.5390
2.5947	383.58	285000	2.4663	0.5389
2.5947	384.25	285500	2.4654	0.5387
2.5947	384.93	286000	2.4708	0.5385
2.5947	385.6	286500	2.4669	0.5388
2.5947	386.27	287000	2.4612	0.5396
2.5947	386.94	287500	2.4666	0.5392
2.5947	387.62	288000	2.4653	0.5393
2.5947	388.29	288500	2.4666	0.5390
2.5947	388.96	289000	2.4684	0.5388
2.5947	389.64	289500	2.4660	0.5394
2.5936	390.31	290000	2.4642	0.5395
2.5936	390.98	290500	2.4627	0.5403
2.5936	391.66	291000	2.4683	0.5389
2.5936	392.33	291500	2.4667	0.5387
2.5936	393.0	292000	2.4660	0.5389
2.5936	393.67	292500	2.4673	0.5390
2.5936	394.35	293000	2.4645	0.5391
2.5936	395.02	293500	2.4693	0.5389
2.5936	395.69	294000	2.4692	0.5385
2.5936	396.37	294500	2.4653	0.5385
2.5934	397.04	295000	2.4661	0.5390
2.5934	397.71	295500	2.4630	0.5394
2.5934	398.38	296000	2.4641	0.5390
2.5934	399.06	296500	2.4636	0.5392
2.5934	399.73	297000	2.4650	0.5392

Framework versions

• Transformers 4.20.1
• Pytorch 1.12.0+cu102
• Datasets 2.3.2
• Tokenizers 0.12.1

Additional configurations

data:
  dataset_name: cc100
  lang: nl
  overwrite_cache: False
  validation_split_percentage: 5
  max_seq_length: 512
  preprocessing_num_workers: 8
  mlm_probability: 0.15
  line_by_line: False
  pad_to_max_length: False
  max_train_samples: -1
  max_eval_samples: -1

training:
  do_train: True
  do_eval: True
  do_predict: True
  resume_from_checkpoint: False
  evaluation_strategy: steps
  eval_steps: 500
  per_device_train_batch_size: 16
  per_device_eval_batch_size: 16
  gradient_accumulation_steps: 32
  eval_accumulation_steps: 1
  learning_rate: 5e-5
  weight_decay: 0.0
  adam_beta1: 0.9
  adam_beta2: 0.999
  adam_epsilon: 1e-8
  max_grad_norm: 1.0
  num_train_epochs: 400.0
  lr_scheduler_type: linear
  fp16: False
  warmup_steps: 8000
  seed: 703