init

README.md

@@ -0,0 +1,73 @@
+# Transformer QG on DRCD
+The inputs of the model refers to 
+```
+we integrate C and A into a new C' in the following form.
+C' = [c1, c2, ..., [HL], a1, ..., a|A|, [HL], ..., c|C|]
+```
+> Proposed by [Ying-Hong Chan & Yao-Chung Fan. (2019). A Re-current BERT-based Model for Question Generation.](https://www.aclweb.org/anthology/D19-5821/)
+
+## Features
+- Fully pipline from fine-tune to evaluation
+- Support most of state of the art models
+- Fast deploy as a API server
+
+## DRCD dataset
+[台達閱讀理解資料集 Delta Reading Comprehension Dataset (DRCD)](https://github.com/DRCKnowledgeTeam/DRCD) 屬於通用領域繁體中文機器閱讀理解資料集。 DRCD資料集從2,108篇維基條目中整理出10,014篇段落，並從段落中標註出30,000多個問題。
+
+## Available models
+- BART (base on **[uer/bart-base-chinese-cluecorpussmall](https://huggingface.co/uer/bart-base-chinese-cluecorpussmall)**)
+
+## Expriments
+Model             |Bleu 1|Bleu 2|Bleu 3|Bleu 4|METEOR|ROUGE-L|
+------------------|------|------|------|------|------|-------|
+BART-HLSQG        |34.25 |27.70 |22.43 |18.13 |23.58 |36.88  |
+
+## Environment requirements
+The hole development is based on Ubuntu system
+
+1. If you don't have pytorch 1.6+ please install or update first
+> https://pytorch.org/get-started/locally/
+
+2. Install packages `pip install -r requirements.txt`
+
+3. Setup scorer `python setup_scorer.py`
+
+5. Download dataset `python init_dataset.py`
+
+
+## Training
+### Seq2Seq LM
+```
+usage: train_seq2seq_lm.py [-h]
+                           [--base_model {facebook/bart-base,facebook/bart-large,t5-small,t5-base,t5-large}]
+                           [-d {squad,squad-nqg}] [--epoch EPOCH] [--lr LR]
+                           [--dev DEV] [--server] [--run_test]
+                           [-fc FROM_CHECKPOINT]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --base_model {facebook/bart-base,facebook/bart-large,t5-small,t5-base,t5-large}
+  -d {squad,squad-nqg}, --dataset {squad,squad-nqg}
+  --epoch EPOCH
+  --lr LR
+  --dev DEV
+  --server
+  --run_test
+  -fc FROM_CHECKPOINT, --from_checkpoint FROM_CHECKPOINT
+```
+
+## Deploy
+### Start up
+```
+python train_seq2seq_lm.py --server --base_model YOUR_BASE_MODEL --from_checkpoint FROM_CHECKPOINT
+```
+### Request example
+```
+curl --location --request POST 'http://127.0.0.1:5000/' \
+--header 'Content-Type: application/x-www-form-urlencoded' \
+--data-urlencode 'context=哈利·波特是英國作家[HL]羅琳[HL]撰寫的七部幻想小說系列'
+```
+```json
+{"predict": "誰撰寫哈利·波特?"}
+```
+

config.json

@@ -0,0 +1,49 @@
+{
+  "_name_or_path": "uer/bart-base-chinese-cluecorpussmall",
+  "activation_dropout": 0.1,
+  "activation_function": "gelu",
+  "architectures": [
+    "BartForConditionalGeneration"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": 0.0,
+  "d_model": 768,
+  "decoder_attention_heads": 12,
+  "decoder_ffn_dim": 3072,
+  "decoder_layerdrop": 0.1,
+  "decoder_layers": 6,
+  "decoder_start_token_id": 101,
+  "dropout": 0.1,
+  "early_stopping": true,
+  "encoder_attention_heads": 12,
+  "encoder_ffn_dim": 3072,
+  "encoder_layerdrop": 0.1,
+  "encoder_layers": 6,
+  "eos_token_id": 0,
+  "forced_eos_token_id": 0,
+  "gradient_checkpointing": false,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "init_std": 0.02,
+  "is_encoder_decoder": true,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "max_length": 256,
+  "max_position_embeddings": 1024,
+  "model_type": "bart",
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "scale_embedding": false,
+  "tie_word_embeddings": 0,
+  "tokenizer_class": "BertTokenizer",
+  "transformers_version": "4.5.0",
+  "use_cache": true,
+  "vocab_size": 21128
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66ed0b84b7e07533abd9a7323173682f62ae1c24c9c218a1020159aa23790b9f
+size 533263126

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": false, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "do_basic_tokenize": true, "never_split": null, "model_max_length": 128, "special_tokens_map_file": "gpt2-base-chinese-cluecorpussmall\\special_tokens_map.json", "name_or_path": "uer/bart-base-chinese-cluecorpussmall"}