Update README.md

README.md

@@ -25,30 +25,30 @@ This metric can be used in relation extraction evaluation.
 ## How to Use
 This metric takes 2 inputs, prediction and references(ground truth). Both of them are a list of list of dictionary of entity's name and entity's type:
 ```python
->>> import evaluate
->>> metric_path = "Ikala-allen/relation_extraction"
->>> module = evaluate.load(metric_path)
->>> references = [
-...    [
-...       {"head": "phip igments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...        {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...    ]
-... ]
->>> predictions = [
-...    [
-...        {"head": "phipigments", "head_type": "product", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...        {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...    ]
-... ]
->>> evaluation_scores = module.compute(predictions=predictions, references=references)
->>> print(evaluation_scores)
-{'sell': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0}, 'ALL': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0, 'Macro_f1': 50.0, 'Macro_p': 50.0, 'Macro_r': 50.0}}
+import evaluate
+metric_path = "Ikala-allen/relation_extraction"
+module = evaluate.load(metric_path)
+references = [
+  [
+    {"head": "phip igments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+    {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+  ]
+]
+predictions = [
+  [
+    {"head": "phipigments", "head_type": "product", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+    {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+  ]
+]
+evaluation_scores = module.compute(predictions=predictions, references=references)
 ```
 
-
 ### Inputs
 - **predictions** (`list` of `list` of `dictionary`): relation and its type of prediction
-- **references** (`list` of `list` of `dictionary`): references for each relation and its type.
+- **references** (`list` of `list` of `dictionary`): references for each relation and its type
+- **mode** (`str`): define strict or boundaries mode
+- **only_all** (`bool`): define whether only output ["ALL"] relation_type score or every relation_type score, default True
+- **relation_types** (`list`): define relation type that need to be evaluate, if not given, it will construct relation_types from ground truth, default []
  
 ### Output Values
 
@@ -73,7 +73,7 @@ This metric takes 2 inputs, prediction and references(ground truth). Both of the
  
 Output Example:
 ```python
-{'sell': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0}, 'ALL': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0, 'Macro_f1': 50.0, 'Macro_p': 50.0, 'Macro_r': 50.0}}
+{'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0, 'Macro_f1': 50.0, 'Macro_p': 50.0, 'Macro_r': 50.0}
 ```
 
 Remind : Macro_f1、Macro_p、Macro_r、p、r、f1 are always a number between 0 and 1. And tp、fp、fn depend on how many data inputs.
@@ -81,23 +81,23 @@ Remind : Macro_f1、Macro_p、Macro_r、p、r、f1 are always a number between 0
 ### Examples
 Example of only one prediction and reference:
 ```python
->>> metric_path = "Ikala-allen/relation_extraction"
->>> module = evaluate.load(metric_path)
->>> references = [
-...    [
-...       {"head": "phip igments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...        {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...    ]
-... ]
->>> predictions = [
-...    [
-...        {"head": "phipigments", "head_type": "product", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...        {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
-...    ]
-... ]
->>> evaluation_scores = module.compute(predictions=predictions, references=references)
->>> print(evaluation_scores)
-{'sell': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0}, 'ALL': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0, 'Macro_f1': 50.0, 'Macro_p': 50.0, 'Macro_r': 50.0}}
+metric_path = "Ikala-allen/relation_extraction"
+module = evaluate.load(metric_path)
+references = [
+  [
+    {"head": "phip igments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+    {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+  ]
+]
+predictions = [
+  [
+    {"head": "phipigments", "head_type": "product", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+    {"head": "tinadaviespigments", "head_type": "brand", "type": "sell", "tail": "國際認證之色乳", "tail_type": "product"},
+  ]
+]
+evaluation_scores = module.compute(predictions=predictions, references=references)
+print(evaluation_scores)
+>>> {'sell': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0}, 'ALL': {'tp': 1, 'fp': 1, 'fn': 1, 'p': 50.0, 'r': 50.0, 'f1': 50.0, 'Macro_f1': 50.0, 'Macro_p': 50.0, 'Macro_r': 50.0}}
 ```
 
 Example with two or more prediction and reference: