bol20162021
commited on
Commit
•
5f5bfaa
1
Parent(s):
8f2efda
Update README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,7 @@ We evaluate our models on three text-to-SQL benchmarks: Spider, Bird, and text2s
|
|
32 |
|
33 |
## Training Procedure
|
34 |
|
35 |
-
|
36 |
|
37 |
### Hyperparameters
|
38 |
|
|
|
32 |
|
33 |
## Training Procedure
|
34 |
|
35 |
+
SambaCoder-nsql-llama-2-70b was trained using cross-entropy loss to maximize the likelihood of sequential inputs. For finetuning on text-to-SQL pairs, we only compute the loss over the SQL portion of the pair. The model is trained using SambaNova's in-house Reconfigurable Dataflow Unit (RDU), leveraging data and model parallelism. We pre-trained for 2 epochs and fine-tuned for 10 epochs.
|
36 |
|
37 |
### Hyperparameters
|
38 |
|