Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+license: llama2
+inference:
+  parameters:
+    do_sample: false
+    max_length: 200
+widget:
+- text: "CREATE TABLE stadium (\n    stadium_id number,\n    location text,\n    name text,\n    capacity number,\n)\n\n-- Using valid SQLite, answer the following questions for the tables provided above.\n\n-- how many stadiums in total?\n\nSELECT"
+  example_title: "Number stadiums"
+- text: "CREATE TABLE work_orders ( ID NUMBER, CREATED_AT TEXT, COST FLOAT, INVOICE_AMOUNT FLOAT, IS_DUE BOOLEAN, IS_OPEN BOOLEAN, IS_OVERDUE BOOLEAN, COUNTRY_NAME TEXT, )\n\n-- Using valid SQLite, answer the following questions for the tables provided above.\n\n-- how many work orders are open?\n\nSELECT"
+  example_title: "Open work orders"
+- text: "CREATE TABLE stadium ( stadium_id number, location text, name text, capacity number, highest number, lowest number, average number )\n\nCREATE TABLE singer ( singer_id number, name text, country text, song_name text, song_release_year text, age number, is_male others )\n\nCREATE TABLE concert ( concert_id number, concert_name text, theme text, stadium_id text, year text )\n\nCREATE TABLE singer_in_concert ( concert_id number, singer_id text )\n\n-- Using valid SQLite, answer the following questions for the tables provided above.\n\n-- What is the maximum, the average, and the minimum capacity of stadiums ?\n\nSELECT"
+  example_title: "Stadium capacity"
+---
+
+# NSQL-Llama-2-70B
+
+## Model Description
+
+NSQL is a family of autoregressive open-source large foundation models (FMs) designed specifically for SQL generation tasks.
+
+In this repository we are introducing a new member of NSQL, NSQL-Llama-2-70B. It's based on Meta's original [Llama-2 70B model](https://huggingface.co/meta-llama/Llama-2-70b) and further pre-trained on a dataset of general SQL queries and then fine-tuned on a dataset composed of text-to-SQL pairs.
+
+### Basic Information
+
+<!-- Provide the basic links for the model. -->
+- **Blog Post**: [Link](TBA)
+- **Discord**: [Link](TBA)
+- **HF Hosting**: [Chat with me!](TBA)
+
+## Training Data
+
+The general SQL queries are the SQL subset from [The Stack](https://huggingface.co/datasets/bigcode/the-stack), containing 1M training samples. The labeled text-to-SQL pairs come from the NSText2SQL dataset (https://huggingface.co/datasets/NumbersStation/NSText2SQL).
+
+## Evaluation Data
+
+We evaluate our models on three text-to-SQL benchmarks: Spider, Bird, and text2sql.
+
+## Training Procedure
+
+NSQL was trained using cross-entropy loss to maximize the likelihood of sequential inputs. For finetuning on text-to-SQL pairs, we only compute the loss over the SQL portion of the pair. The model is trained using SambaNova's in-house Reconfigurable Dataflow Unit (RDU), leveraging data and model parallelism. We pre-trained for 2 epochs and fine-tuned for 10 epochs.
+
+## Intended Use and Limitations
+
+The model was designed for text-to-SQL generation tasks from given table schema and natural language prompts. The model works best with the prompt format defined below and outputting `SELECT` queries.
+
+## How to Use
+
+Example 1:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+tokenizer = AutoTokenizer.from_pretrained("sambanovasystems/nsql-Llama-2-70B")
+model = AutoModelForCausalLM.from_pretrained("sambanovasystems/nsql-Llama-2-70B", torch_dtype=torch.bfloat16)
+
+```