|
--- |
|
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct |
|
library_name: peft |
|
--- |
|
|
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
gretelai/synthetic_text_to_sql |
|
https://huggingface.co/datasets/gretelai/synthetic_text_to_sql |
|
gretelai/synthetic_text_to_sql is a rich dataset of high quality synthetic Text-to-SQL samples. The dataset includes 105,851 records partitioned into 100,000 train and 5,851 test records. But i used only 50k records for my training. |
|
### Training Result |
|
|
|
|
|
Step Training Loss |
|
|
|
10 1.296000 |
|
|
|
20 1.331600 |
|
|
|
30 1.279400 |
|
|
|
40 1.312900 |
|
|
|
50 1.274100 |
|
|
|
60 1.271700 |
|
|
|
70 1.209100 |
|
|
|
80 1.192600 |
|
|
|
90 1.176700 |
|
|
|
100 1.118300 |
|
|
|
110 1.086800 |
|
|
|
120 1.048000 |
|
|
|
130 1.019500 |
|
|
|
140 1.001400 |
|
|
|
150 0.994300 |
|
|
|
160 0.934900 |
|
|
|
170 0.904500 |
|
|
|
180 0.879900 |
|
|
|
190 0.850400 |
|
|
|
200 0.828000 |
|
|
|
210 0.811400 |
|
|
|
220 0.846000 |
|
|
|
230 0.791100 |
|
|
|
240 0.766900 |
|
|
|
250 0.782000 |
|
|
|
260 0.718300 |
|
|
|
270 0.701800 |
|
|
|
280 0.720000 |
|
|
|
290 0.693600 |
|
|
|
300 0.676500 |
|
|
|
310 0.679900 |
|
|
|
320 0.673200 |
|
|
|
330 0.669500 |
|
|
|
340 0.692800 |
|
|
|
350 0.662200 |
|
|
|
360 0.761200 |
|
|
|
370 0.659600 |
|
|
|
380 0.683700 |
|
|
|
390 0.681200 |
|
|
|
400 0.674000 |
|
|
|
410 0.651800 |
|
|
|
420 0.641800 |
|
|
|
430 0.646500 |
|
|
|
440 0.664200 |
|
|
|
450 0.633600 |
|
|
|
460 0.646900 |
|
|
|
470 0.643400 |
|
|
|
480 0.658800 |
|
|
|
490 0.631500 |
|
|
|
500 0.678200 |
|
|
|
510 0.633400 |
|
|
|
520 0.623300 |
|
|
|
530 0.655700 |
|
|
|
540 0.631500 |
|
|
|
550 0.617700 |
|
|
|
560 0.644000 |
|
|
|
570 0.650200 |
|
|
|
580 0.618500 |
|
|
|
590 0.615400 |
|
|
|
600 0.614000 |
|
|
|
610 0.612800 |
|
|
|
620 0.616900 |
|
|
|
630 0.640200 |
|
|
|
640 0.613000 |
|
|
|
650 0.611400 |
|
|
|
660 0.617000 |
|
|
|
670 0.629800 |
|
|
|
680 0.648800 |
|
|
|
690 0.608800 |
|
|
|
700 0.603200 |
|
|
|
710 0.628200 |
|
|
|
720 0.629700 |
|
|
|
730 0.604400 |
|
|
|
740 0.610700 |
|
|
|
750 0.621300 |
|
|
|
760 0.617900 |
|
|
|
770 0.596500 |
|
|
|
780 0.612800 |
|
|
|
790 0.611700 |
|
|
|
800 0.618600 |
|
|
|
810 0.590900 |
|
|
|
820 0.590300 |
|
|
|
830 0.592900 |
|
|
|
840 0.611700 |
|
|
|
850 0.628300 |
|
|
|
860 0.590100 |
|
|
|
870 0.584800 |
|
|
|
880 0.591200 |
|
|
|
890 0.585900 |
|
|
|
900 0.607000 |
|
|
|
910 0.578800 |
|
|
|
920 0.576600 |
|
|
|
930 0.597600 |
|
|
|
940 0.602100 |
|
|
|
950 0.579000 |
|
|
|
960 0.597900 |
|
|
|
970 0.590600 |
|
|
|
980 0.606100 |
|
|
|
990 0.577600 |
|
|
|
1000 0.584000 |
|
|
|
1010 0.569300 |
|
|
|
1020 0.594000 |
|
|
|
1030 0.596100 |
|
|
|
1040 0.590600 |
|
|
|
1050 0.570300 |
|
|
|
1060 0.572800 |
|
|
|
1070 0.572200 |
|
|
|
1080 0.569900 |
|
|
|
1090 0.587200 |
|
|
|
1100 0.572200 |
|
|
|
1110 0.569700 |
|
|
|
1120 0.612500 |
|
|
|
1130 0.587800 |
|
|
|
1140 0.568100 |
|
|
|
1150 0.573100 |
|
|
|
1160 0.568300 |
|
|
|
1170 0.620800 |
|
|
|
1180 0.570600 |
|
|
|
1190 0.561500 |
|
|
|
1200 0.560200 |
|
|
|
1210 0.592400 |
|
|
|
1220 0.580500 |
|
|
|
1230 0.578300 |
|
|
|
1240 0.573400 |
|
|
|
1250 0.568800 |
|
|
|
1260 0.600500 |
|
|
|
1270 0.578800 |
|
|
|
1280 0.561300 |
|
|
|
1290 0.570900 |
|
|
|
1300 0.567700 |
|
|
|
1310 0.589800 |
|
|
|
1320 0.598200 |
|
|
|
1330 0.564900 |
|
|
|
1340 0.577500 |
|
|
|
1350 0.565700 |
|
|
|
1360 0.581400 |
|
|
|
1370 0.562000 |
|
|
|
1380 0.588200 |
|
|
|
1390 0.603800 |
|
|
|
1400 0.560300 |
|
|
|
1410 0.559600 |
|
|
|
1420 0.567000 |
|
|
|
1430 0.562700 |
|
|
|
1440 0.564200 |
|
|
|
1450 0.563700 |
|
|
|
1460 0.561100 |
|
|
|
1470 0.561100 |
|
|
|
1480 0.561600 |
|
|
|
1490 0.564800 |
|
|
|
1500 0.579100 |
|
|
|
1510 0.564100 |
|
|
|
1520 0.562900 |
|
|
|
1530 0.569800 |
|
|
|
1540 0.566200 |
|
|
|
1550 0.599100 |
|
|
|
1560 0.562000 |
|
|
|
1570 0.580600 |
|
|
|
1580 0.564900 |
|
|
|
1590 0.571900 |
|
|
|
1600 0.580000 |
|
|
|
1610 0.559200 |
|
|
|
1620 0.566900 |
|
|
|
1630 0.556100 |
|
|
|
|
|
 |
|
|
|
#### Training Hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
num_train_epochs=3, |
|
per_device_train_batch_size=2, |
|
gradient_accumulation_steps=4, |
|
optim="adamw_torch_fused", |
|
learning_rate=2e-4, |
|
max_grad_norm=0.3, |
|
weight_decay=0.01, |
|
lr_scheduler_type="cosine", |
|
warmup_steps=50, |
|
bf16=True, |
|
tf32=True, |
|
) |
|
|
|
|
