--- language: - en - fr - ro - de - multilingual widget: - text: "Continue the dialogue as a task-oriented dialogue system called SYSTEM. The answer of SYSTEM should follow the ACTION provided next while answering the USER's last utterance: \n Hello, I am looking for a restaurant in Cambridge. I believe it is called Golden Wok. \n ACTION: {'Restaurant-Inform': [['address', '191 Histon Road Chesterton']]}" example_title: "Dialog Act to Response Generation" - text: "Translate to German: My name is Arthur" example_title: "Translation" - text: "Please answer to the following question. Who is going to be the next Ballon d'or?" example_title: "Question Answering" - text: "Please answer the following question. What is the boiling point of Nitrogen?" example_title: "Scientific knowledge" - text: "Answer the following yes/no question. Can you write 200 words in a single tweet?" example_title: "Yes/no question" - text: "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?" example_title: "Reasoning task" - text: "Q: Is the statement ( `Jianguo is a research scientist at Salesforce AI` and `Jianguo is a student at UIC` ) True or Flase? A: Let's think step by step" example_title: "Boolean Expressions" - text: "The square root of x is the cube root of y. What is y to the power of 2, if x = 4?" example_title: "Math reasoning" - text: "Premise: At my age you will probably have learnt one lesson. Hypothesis: It's not certain how many lessons you'll learn by your thirties. Does the premise entail the hypothesis?" example_title: "Premise and hypothesis" inference: parameters: max_length: 256 tags: - text2text-generation - dialog datasets: - Salesforce/dialogstudio - flan license: apache-2.0 --- # Model Card for DialogStudio-T5 base drawing # Table of Contents 0. [TL;DR](#TL;DR) 1. [Model Details](#model-details) 2. [Usage](#usage) 3. [Uses](#uses) 4. [Bias, Risks, and Limitations](#bias-risks-and-limitations) 5. [Training Details](#training-details) 6. [Evaluation](#evaluation) 7. [Environmental Impact](#environmental-impact) 8. [Citation](#citation) 9. [Model Card Authors](#model-card-authors) # TL;DR If you already know T5 and Flan-T5, DialogStudio-T5 is better at many things. With the same number of parameters, the models are fine-tuned from a selected amount of dialogues from [DialogStudio](https://github.com/salesforce/DialogStudio) and also 1000 additional tasks. **Disclaimer**: Content from **this** model card are modified from contents written by the Hugging Face team, and parts of it were copy pasted from the [T5 model card](https://huggingface.co/t5-large) and [Flan-T5 model card](https://huggingface.co/google/flan-t5-large). **Follow the [DialogStudio](https://github.com/salesforce/DialogStudio) GitHub repository for latest information.** # Model Details ## Data We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). Note that this version does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder. drawing ## Model Description - **Model type:** Language model - **Language(s) (NLP):** English, Spanish, Japanese, Persian, Hindi, French, Chinese, Bengali, Gujarati, German, Telugu, Italian, Arabic, Polish, Tamil, Marathi, Malayalam, Oriya, Panjabi, Portuguese, Urdu, Galician, Hebrew, Korean, Catalan, Thai, Dutch, Indonesian, Vietnamese, Bulgarian, Filipino, Central Khmer, Lao, Turkish, Russian, Croatian, Swedish, Yoruba, Kurdish, Burmese, Malay, Czech, Finnish, Somali, Tagalog, Swahili, Sinhala, Kannada, Zhuang, Igbo, Xhosa, Romanian, Haitian, Estonian, Slovak, Lithuanian, Greek, Nepali, Assamese, Norwegian - **License:** Apache 2.0 - **Related Models:** [All DialogStudio-T5 Checkpoints](https://huggingface.co/models?search=dialogstudio-t5) - **Resources for more information:** - [Research paper](https://arxiv.org/abs/2307.10172) - [GitHub Repo](https://github.com/salesforce/DialogStudio) - **Maximum model length:**: - Maximum input length: 1200 - Maximum output length: 256 - **Training formats:** - We process dialogue data into below input format : - With instruction and external knowledge: ```Instruction: your instruction user utterance 1 system utterance 1 ... user utterance N your external knowledge``` - Without instruction: ``` user utterance 1 system utterance 1 ... user utterance N your external knowledge``` - Without external knowledge: ```Instruction: your instruction user utterance 1 system utterance 1 ... user utterance N``` - Without both: ``` user utterance 1 system utterance 1 ... user utterance N``` - Note: output is final the system response; ``, `` and `` are special tokens - For sampled FLAN data: - We follow their original data format, i.e., we did not set special tokens to separate in-context learning examples. - In summary: - We recommend you use our format and add our special tokens (such as `` and `` ) to get better performance. However, you may not necessary need to exactly follow our format if you do observe random behavios. - We found that T5 model series such as Flan-t5 and DialogStudio-T5 may generate repetitive tokens during inference. If you find such repetition issues, you can set the `repetition_penalty` in model.generate(), such as 1.5, to mitigate them. Note that `repetition_penalty=1.0` by default. # Usage Find below some example scripts on how to use the model in `transformers`: ## Using the Pytorch model ### Running the model on a CPU
Click to expand ```python from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0") model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0") input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?" input_ids = tokenizer(input_text, return_tensors="pt").input_ids outputs = model.generate(input_ids, max_new_tokens=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ```
### Running the model on a GPU
Click to expand ```python # pip install accelerate from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0") model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0", device_map="auto") input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?" input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda") outputs = model.generate(input_ids, max_new_tokens=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ```
### Running the model on a GPU using different precisions #### FP16
Click to expand ```python # pip install accelerate import torch from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0") model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0", device_map="auto", torch_dtype=torch.float16) input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?" input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda") outputs = model.generate(input_ids, max_new_tokens=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ```
#### INT8
Click to expand ```python # pip install bitsandbytes accelerate from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0") model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-base-v1.0", device_map="auto", load_in_8bit=True) input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?" input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda") outputs = model.generate(input_ids, max_new_tokens=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ```
# Uses ## Direct Use and Downstream Use > The primary use is research on language models, including: research on zero-shot NLP tasks and in-context few-shot learning NLP tasks, such as dialogue response generation, reasoning, and question answering; advancing fairness and safety research, and understanding limitations of current large language models ## Out-of-Scope Use More information needed. # Bias, Risks, and Limitations The information below in this section are copied and modified from Flan-T5's models card: > Language models, including DialogStudio-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). DialogStudio-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application. ## Ethical considerations and risks > DialogStudio-T5 is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data. ## Known Limitations > DialogStudio-T5 has not been tested in real world applications. ## Sensitive Use: > DialogStudio-T5 should not be applied for any unacceptable use cases, e.g., generation of abusive speech. # Training Details ## Training Data We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). Note that this version does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder. See above **Training formats:** for details of the training formats. ## Training Procedure > These models are based on Flan-T5 and are fine-tuned with instructions for better zero-shot and few-shot performance. There is one fine-tuned DialogStudio model per T5 model size. The model has been trained on 16 A100 GPUs, each with 40G memory, using public [transformer](https://github.com/huggingface/transformers) codebase. # Evaluation ## Testing Data, Factors & Metrics The authors evaluated the model on several dialogue tasks and general tasks such as 0-shot/5-shot MMLU and 3-shot BBH. ## Results For full results for DialogStudio, see the [research paper](https://arxiv.org/abs/2307.10172). ## Environmental Impact More information needed. # Citation **BibTeX:** ```bibtex @misc{zhang2023dialogstudio, title={DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI}, author={Jianguo Zhang and Kun Qian and Zhiwei Liu and Shelby Heinecke and Rui Meng and Ye Liu and Zhou Yu and and Huan Wang and Silvio Savarese and Caiming Xiong}, year={2023}, eprint={2307.10172}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```