codefuse-ai

/

CodeFuse-DeepSeek-33B

like 59

Follow

CodeFuse AI 120

Hello, we haven't tested its performance on SQL generation; we'll test it.
In the fine-tuning training data, we did not use a dedicated SQL generation dataset.

@waytohou Hello,
I have tested the performance of our model in SQL generation. I selected the Spider evaluation dataset as our test set, and after using our model for inference generation, I utilized GPT-4 to determine the semantic equivalence between the predictions and the reference answers, resulting in an accuracy of 66.8% (690/1033). The format I used for inference is as follows:

prompt:

<s>system
You are a MySQL SQL Writer. You must generate clean SQL statements using MySQL Syntax according to the user's requirements.
<s>human
What are the names, countries, and ages for every singer in descending order of age?
singer(singer_id|name|country|song_name|song_release_year|age|is_male)
<s>bot

predict:

SELECT name, country, age
FROM singer
ORDER BY age DESC;

label:

SELECT name ,  country ,  age FROM singer ORDER BY age DESC