File size: 3,640 Bytes
6527b45 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 1fcccde 32a4714 6527b45 32a4714 6527b45 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 |
---
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
library_name: peft
---
## Training Details
### Training Data
gretelai/synthetic_text_to_sql
https://huggingface.co/datasets/gretelai/synthetic_text_to_sql
gretelai/synthetic_text_to_sql is a rich dataset of high quality synthetic Text-to-SQL samples. The dataset includes 105,851 records partitioned into 100,000 train and 5,851 test records. But i used only 50k records for my training.
### Training Result
Step Training Loss
10 1.296000
20 1.331600
30 1.279400
40 1.312900
50 1.274100
60 1.271700
70 1.209100
80 1.192600
90 1.176700
100 1.118300
110 1.086800
120 1.048000
130 1.019500
140 1.001400
150 0.994300
160 0.934900
170 0.904500
180 0.879900
190 0.850400
200 0.828000
210 0.811400
220 0.846000
230 0.791100
240 0.766900
250 0.782000
260 0.718300
270 0.701800
280 0.720000
290 0.693600
300 0.676500
310 0.679900
320 0.673200
330 0.669500
340 0.692800
350 0.662200
360 0.761200
370 0.659600
380 0.683700
390 0.681200
400 0.674000
410 0.651800
420 0.641800
430 0.646500
440 0.664200
450 0.633600
460 0.646900
470 0.643400
480 0.658800
490 0.631500
500 0.678200
510 0.633400
520 0.623300
530 0.655700
540 0.631500
550 0.617700
560 0.644000
570 0.650200
580 0.618500
590 0.615400
600 0.614000
610 0.612800
620 0.616900
630 0.640200
640 0.613000
650 0.611400
660 0.617000
670 0.629800
680 0.648800
690 0.608800
700 0.603200
710 0.628200
720 0.629700
730 0.604400
740 0.610700
750 0.621300
760 0.617900
770 0.596500
780 0.612800
790 0.611700
800 0.618600
810 0.590900
820 0.590300
830 0.592900
840 0.611700
850 0.628300
860 0.590100
870 0.584800
880 0.591200
890 0.585900
900 0.607000
910 0.578800
920 0.576600
930 0.597600
940 0.602100
950 0.579000
960 0.597900
970 0.590600
980 0.606100
990 0.577600
1000 0.584000
1010 0.569300
1020 0.594000
1030 0.596100
1040 0.590600
1050 0.570300
1060 0.572800
1070 0.572200
1080 0.569900
1090 0.587200
1100 0.572200
1110 0.569700
1120 0.612500
1130 0.587800
1140 0.568100
1150 0.573100
1160 0.568300
1170 0.620800
1180 0.570600
1190 0.561500
1200 0.560200
1210 0.592400
1220 0.580500
1230 0.578300
1240 0.573400
1250 0.568800
1260 0.600500
1270 0.578800
1280 0.561300
1290 0.570900
1300 0.567700
1310 0.589800
1320 0.598200
1330 0.564900
1340 0.577500
1350 0.565700
1360 0.581400
1370 0.562000
1380 0.588200
1390 0.603800
1400 0.560300
1410 0.559600
1420 0.567000
1430 0.562700
1440 0.564200
1450 0.563700
1460 0.561100
1470 0.561100
1480 0.561600
1490 0.564800
1500 0.579100
1510 0.564100
1520 0.562900
1530 0.569800
1540 0.566200
1550 0.599100
1560 0.562000
1570 0.580600
1580 0.564900
1590 0.571900
1600 0.580000
1610 0.559200
1620 0.566900
1630 0.556100
#### Training Hyperparameters
The following hyperparameters were used during training:
num_train_epochs=3,
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
optim="adamw_torch_fused",
learning_rate=2e-4,
max_grad_norm=0.3,
weight_decay=0.01,
lr_scheduler_type="cosine",
warmup_steps=50,
bf16=True,
tf32=True,
)
|