README.md

---
language: 
- en
- fr
- ro
- de
- multilingual

widget:
- text: "Continue the dialogue as a task-oriented dialogue system called SYSTEM. The answer of SYSTEM should follow the ACTION provided next while answering the USER's last utterance: \n<USER> Hello, I am looking for a restaurant in Cambridge. I believe it is called Golden Wok. \n<EXTERNAL KNOWLEDGE> ACTION: {'Restaurant-Inform': [['address', '191 Histon Road Chesterton']]}"
  example_title: "Dialog Act to Response Generation"
- text: "Translate to German:  My name is Arthur"
  example_title: "Translation"
- text: "Please answer to the following question. Who is going to be the next Ballon d'or?"
  example_title: "Question Answering"
- text: "Q: Can Geoffrey Hinton have a conversation with George Washington? Give the rationale before answering."
  example_title: "Logical reasoning"
- text: "Please answer the following question. What is the boiling point of Nitrogen?"
  example_title: "Scientific knowledge"
- text: "Answer the following yes/no question. Can you write 200 words in a single tweet?"
  example_title: "Yes/no question"
- text: "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
  example_title: "Reasoning task"
- text: "Q: Is the statement ( `Jianguo is a research scientist at Salesforce AI` and `Jianguo is a student at UIC` ) True or Flase? A: Let's think step by step"
  example_title: "Boolean Expressions"
- text: "The square root of x is the cube root of y. What is y to the power of 2, if x = 4?"
  example_title: "Math reasoning"
- text: "Premise:  At my age you will probably have learnt one lesson. Hypothesis:  It's not certain how many lessons you'll learn by your thirties. Does the premise entail the hypothesis?"
  example_title: "Premise and hypothesis"

inference:
  parameters:
    max_length: 256

tags:
  - text2text-generation
  - dialog

datasets:
- Salesforce/dialogstudio
- flan


license: apache-2.0
---

# Model Card for DialogStudio-T5 large

<img src="https://huggingface.co/datasets/Salesforce/dialogstudio/resolve/main/logo.png"
alt="drawing" width="510"/>

#  Table of Contents

0. [TL;DR](#TL;DR)
1. [Model Details](#model-details)
2. [Usage](#usage)
3. [Uses](#uses)
4. [Bias, Risks, and Limitations](#bias-risks-and-limitations)
5. [Training Details](#training-details)
6. [Evaluation](#evaluation)
7. [Environmental Impact](#environmental-impact)
8. [Citation](#citation)
9. [Model Card Authors](#model-card-authors)

# TL;DR

If you already know T5 and Flan-T5, DialogStudio-T5 is better at many things. With the same number of parameters, the models are fine-tuned from a selected amount of dialogues from [DialogStudio](https://github.com/salesforce/DialogStudio) and also 1000 additional tasks. 


**Disclaimer**: Content from **this** model card are modified from contents written by the Hugging Face team, and parts of it were copy pasted from the [T5 model card](https://huggingface.co/t5-large) and [Flan-T5 model card](https://huggingface.co/google/flan-t5-large).


**Follow the [DialogStudio](https://github.com/salesforce/DialogStudio) GitHub repository for latest information.**


# Model Details
## Data

We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). 


**Note** that this model version 1.0 does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder.


<img src="https://huggingface.co/datasets/Salesforce/dialogstudio/resolve/main/DialogStudio_Stats.jpg"
alt="drawing" width="700"/>



## Model Description


- **Model type:** Language model
- **Language(s) (NLP):** English, Spanish, Japanese, Persian, Hindi, French, Chinese, Bengali, Gujarati, German, Telugu, Italian, Arabic, Polish, Tamil, Marathi, Malayalam, Oriya, Panjabi, Portuguese, Urdu, Galician, Hebrew, Korean, Catalan, Thai, Dutch, Indonesian, Vietnamese, Bulgarian, Filipino, Central Khmer, Lao, Turkish, Russian, Croatian, Swedish, Yoruba, Kurdish, Burmese, Malay, Czech, Finnish, Somali, Tagalog, Swahili, Sinhala, Kannada, Zhuang, Igbo, Xhosa, Romanian, Haitian, Estonian, Slovak, Lithuanian, Greek, Nepali, Assamese, Norwegian
- **License:** Apache 2.0
- **Related Models:** [All DialogStudio-T5 Checkpoints](https://huggingface.co/models?search=dialogstudio-t5)
- **Resources for more information:**
  - [Research paper](https://arxiv.org/abs/2307.10172)
  - [GitHub Repo](https://github.com/salesforce/DialogStudio)
- **Maximum model length:**: 
  - Maximum input length: 1200
  - Maximum output length: 256
- **Training formats:**
  - We process dialogue data into below input format :
    - With instruction and external knowledge: ```Instruction: your instruction <USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N <EXTERNAL KNOWLEDGE> your external knowledge```
    - Without instruction: ```<USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N <EXTERNAL KNOWLEDGE> your external knowledge```
    - Without external knowledge: ```Instruction: your instruction <USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N```
    - Without both: ```<USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N```
    - Note:  output is final the system response; `<USER>`, `<SYSTEM>` and `<EXTERNAL KNOWLEDGE>` are special tokens
  - For sampled FLAN data:
    - We follow their original data format, i.e., we did not set special tokens to separate in-context learning examples.
  - In summary:
    - We recommend you use our format and add our special tokens (such as `<USER>` and `<SYSTEM>` ) to get better performance. However, you may not necessary need to exactly follow our format if you do not observe random behavios.
    - We found that T5 model series such as Flan-t5 and DialogStudio-T5 may generate repetitive tokens during inference. If you find such repetition issues, you can set the `repetition_penalty` in model.generate(), such as 1.5, to mitigate them. Note that `repetition_penalty=1.0` by default.
# Usage

Find below some example scripts on how to use the model in `transformers`:

## Using the Pytorch model

### Running the model on a CPU

<details>
<summary> Click to expand </summary>

```python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

### Running the model on a GPU

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto")

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

### Running the model on a GPU using different precisions

#### FP16

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto", torch_dtype=torch.float16)

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

#### INT8

<details>
<summary> Click to expand </summary>

```python
# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto", load_in_8bit=True)

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

# Uses

## Direct Use and Downstream Use

<!-- The authors write in [the original paper's model card](https://arxiv.org/pdf/2210.11416.pdf) that:  -->

> The primary use is research on language models, including: research on zero-shot NLP tasks and in-context few-shot learning NLP tasks, such as dialogue response generation, reasoning, and question answering; advancing fairness and safety research, and understanding limitations of current large language models


## Out-of-Scope Use

More information needed.

# Bias, Risks, and Limitations

The information below in this section are copied and modified from Flan-T5's models card:

> Language models, including DialogStudio-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). DialogStudio-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.

## Ethical considerations and risks

> DialogStudio-T5 is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data.

## Known Limitations

> DialogStudio-T5 has not been tested in real world applications.

## Sensitive Use:

> DialogStudio-T5 should not be applied for any unacceptable use cases, e.g., generation of abusive speech.

# Training Details

## Training Data

We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). 

**Note:** 

Model Version 1.0 is built on small-scale pre-trained models, this version does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder.  As a result, it has certain limitations in terms of writing and creative capabilities. Our initial focus is to update the model versions to enhance existing abilities. Further improvements, including expansion of other capabilities, are part of our roadmap and will be responsive to community requests.


See above **Training formats:** for details of the training formats.

## Training Procedure


> These models are based on Flan-T5 and are fine-tuned with instructions for better zero-shot and few-shot performance. There is one fine-tuned DialogStudio model per T5 model size.

The model has been trained on 16 A100 GPUs, each with 40G memory, using public [transformer](https://github.com/huggingface/transformers) codebase.


# Evaluation

## Testing Data, Factors & Metrics

The authors evaluated the model on several dialogue tasks and general tasks such as 0-shot/5-shot MMLU and 3-shot BBH. 

## Results 

For full results for DialogStudio, see the [research paper](https://arxiv.org/abs/2307.10172).

## Environmental Impact
More information needed.

# Citation

**BibTeX:**

```bibtex
@misc{zhang2023dialogstudio,
      title={DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI}, 
      author={Jianguo Zhang and Kun Qian and Zhiwei Liu and Shelby Heinecke and Rui Meng and Ye Liu and Zhou Yu and and Huan Wang and Silvio Savarese and Caiming Xiong},
      year={2023},
      eprint={2307.10172},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```