README.md

---
language: 
- en
- fr
- ro
- de
- multilingual

widget:
- text: "Continue the dialogue as a task-oriented dialogue system called SYSTEM. The answer of SYSTEM should follow the ACTION provided next while answering the USER's last utterance: \n<USER> Hello, I am looking for a restaurant in Cambridge. I believe it is called Golden Wok. \n<EXTERNAL KNOWLEDGE> ACTION: {'Restaurant-Inform': [['address', '191 Histon Road Chesterton']]}"
  example_title: "Dialog Act to Response Generation"
- text: "Translate to German:  My name is Arthur"
  example_title: "Translation"
- text: "Please answer to the following question. Who is going to be the next Ballon d'or?"
  example_title: "Question Answering"
- text: "Q: Can Geoffrey Hinton have a conversation with George Washington? Give the rationale before answering."
  example_title: "Logical reasoning"
- text: "Please answer the following question. What is the boiling point of Nitrogen?"
  example_title: "Scientific knowledge"
- text: "Answer the following yes/no question. Can you write 200 words in a single tweet?"
  example_title: "Yes/no question"
- text: "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
  example_title: "Reasoning task"
- text: "Q: Is the statement ( `Jianguo is a research scientist at Salesforce AI` and `Jianguo is a student at UIC` ) True or Flase? A: Let's think step by step"
  example_title: "Boolean Expressions"
- text: "The square root of x is the cube root of y. What is y to the power of 2, if x = 4?"
  example_title: "Math reasoning"
- text: "Premise:  At my age you will probably have learnt one lesson. Hypothesis:  It's not certain how many lessons you'll learn by your thirties. Does the premise entail the hypothesis?"
  example_title: "Premise and hypothesis"

inference:
  parameters:
    max_length: 256

tags:
  - text2text-generation
  - dialog

datasets:
- Salesforce/dialogstudio
- flan


license: apache-2.0
---

# Model Card for DialogStudio-T5 large

<img src="https://huggingface.co/datasets/Salesforce/dialogstudio/resolve/main/logo.png"
alt="drawing" width="510"/>

#  Table of Contents

0. [TL;DR](#TL;DR)
1. [Model Details](#model-details)
2. [Usage](#usage)
3. [Uses](#uses)
4. [Bias, Risks, and Limitations](#bias-risks-and-limitations)
5. [Training Details](#training-details)
6. [Evaluation](#evaluation)
7. [Environmental Impact](#environmental-impact)
8. [Citation](#citation)
9. [Model Card Authors](#model-card-authors)

# TL;DR

If you already know T5 and Flan-T5, DialogStudio-T5 is better at many things. With the same number of parameters, the models are fine-tuned from a selected amount of dialogues from [DialogStudio](https://github.com/salesforce/DialogStudio) and also 1000 additional tasks. 


**Disclaimer**: Content from **this** model card are modified from contents written by the Hugging Face team, and parts of it were copy pasted from the [T5 model card](https://huggingface.co/t5-large) and [Flan-T5 model card](https://huggingface.co/google/flan-t5-large).


**Follow the [DialogStudio](https://github.com/salesforce/DialogStudio) GitHub repository for latest information.**


# Model Details
## Data

We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). 

Note that this version does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder.


<img src="https://huggingface.co/datasets/Salesforce/dialogstudio/resolve/main/DialogStudio_Stats.jpg"
alt="drawing" width="700"/>



## Model Description


- **Model type:** Language model
- **Language(s) (NLP):** English, Spanish, Japanese, Persian, Hindi, French, Chinese, Bengali, Gujarati, German, Telugu, Italian, Arabic, Polish, Tamil, Marathi, Malayalam, Oriya, Panjabi, Portuguese, Urdu, Galician, Hebrew, Korean, Catalan, Thai, Dutch, Indonesian, Vietnamese, Bulgarian, Filipino, Central Khmer, Lao, Turkish, Russian, Croatian, Swedish, Yoruba, Kurdish, Burmese, Malay, Czech, Finnish, Somali, Tagalog, Swahili, Sinhala, Kannada, Zhuang, Igbo, Xhosa, Romanian, Haitian, Estonian, Slovak, Lithuanian, Greek, Nepali, Assamese, Norwegian
- **License:** Apache 2.0
- **Related Models:** [All DialogStudio-T5 Checkpoints](https://huggingface.co/models?search=dialogstudio-t5)
- **Resources for more information:**
  - [Research paper](https://arxiv.org/abs/2307.10172)
  - [GitHub Repo](https://github.com/salesforce/DialogStudio)
- **Maximum model length:**: 
  - Maximum input length: 1200
  - Maximum output length: 256
- **Training formats:**
  - We process dialogue data into below input format :
    - With instruction and external knowledge: ```Instruction: your instruction <USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N <EXTERNAL KNOWLEDGE> your external knowledge```
    - Without instruction: ```<USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N <EXTERNAL KNOWLEDGE> your external knowledge```
    - Without external knowledge: ```Instruction: your instruction <USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N```
    - Without both: ```<USER> user utterance 1 <SYSTEM> system utterance 1 ... <USER> user utterance N```
    - Note:  output is final the system response; `<USER>`, `<SYSTEM>` and `<EXTERNAL KNOWLEDGE>` are special tokens
  - For sampled FLAN data:
    - We follow their original data format, i.e., we did not set special tokens to separate in-context learning examples.
  - In summary:
    - We recommend you use our format and add our special tokens (such as `<USER>` and `<SYSTEM>` ) to get better performance. However, you may not necessary need to exactly follow our format if you do observe random behavios.
    - We found that T5 model series such as Flan-t5 and DialogStudio-T5 may generate repetitive tokens during inference. If you find such repetition issues, you can set the `repetition_penalty` in model.generate(), such as 1.5, to mitigate them. Note that `repetition_penalty=1.0` by default.
# Usage

Find below some example scripts on how to use the model in `transformers`:

## Using the Pytorch model

### Running the model on a CPU

<details>
<summary> Click to expand </summary>

```python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

### Running the model on a GPU

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto")

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

### Running the model on a GPU using different precisions

#### FP16

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto", torch_dtype=torch.float16)

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

#### INT8

<details>
<summary> Click to expand </summary>

```python
# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0")
model = AutoModelForSeq2SeqLM.from_pretrained("Salesforce/dialogstudio-t5-large-v1.0", device_map="auto", load_in_8bit=True)

input_text = "Answer the following yes/no question by reasoning step-by-step. Can you write 200 words in a single tweet?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

</details>

# Uses

## Direct Use and Downstream Use

<!-- The authors write in [the original paper's model card](https://arxiv.org/pdf/2210.11416.pdf) that:  -->

> The primary use is research on language models, including: research on zero-shot NLP tasks and in-context few-shot learning NLP tasks, such as dialogue response generation, reasoning, and question answering; advancing fairness and safety research, and understanding limitations of current large language models


## Out-of-Scope Use

More information needed.

# Bias, Risks, and Limitations

The information below in this section are copied and modified from Flan-T5's models card:

> Language models, including DialogStudio-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). DialogStudio-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.

## Ethical considerations and risks

> DialogStudio-T5 is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data.

## Known Limitations

> DialogStudio-T5 has not been tested in real world applications.

## Sensitive Use:

> DialogStudio-T5 should not be applied for any unacceptable use cases, e.g., generation of abusive speech.

# Training Details

## Training Data

We sample a small amount of dialogues from each commercial supported dataset under three categories of [DialogStudio](https://huggingface.co/datasets/Salesforce/dialogstudio), i.e., KG-Dial, TOD and Open-Domain dialogues. Additionally, we sample at most 150 examples for each non-translation task from [FLAN](https://github.com/google-research/FLAN/tree/main/flan/v2). 

Note that this version does not incorporate datasets utilized for training large-scale models (>=7B) like Alpaca, ShareGPT, GPT4ALL, UltraChat from OpenAI's 'GPT-3.5/4', or other datasets such as OASST1 and WizardCoder.


See above **Training formats:** for details of the training formats.

## Training Procedure


> These models are based on Flan-T5 and are fine-tuned with instructions for better zero-shot and few-shot performance. There is one fine-tuned DialogStudio model per T5 model size.

The model has been trained on 16 A100 GPUs, each with 40G memory, using public [transformer](https://github.com/huggingface/transformers) codebase.


# Evaluation

## Testing Data, Factors & Metrics

The authors evaluated the model on several dialogue tasks and general tasks such as 0-shot/5-shot MMLU and 3-shot BBH. 

## Results 

For full results for DialogStudio, see the [research paper](https://arxiv.org/abs/2307.10172).

## Environmental Impact
More information needed.

# Citation

**BibTeX:**

```bibtex
@misc{zhang2023dialogstudio,
      title={DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI}, 
      author={Jianguo Zhang and Kun Qian and Zhiwei Liu and Shelby Heinecke and Rui Meng and Ye Liu and Zhou Yu and and Huan Wang and Silvio Savarese and Caiming Xiong},
      year={2023},
      eprint={2307.10172},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```