Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,6 @@ pipeline_tag: text-classification
|
|
13 |
|
14 |
[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17b7vrZqH0Yun2AJaOXydYZxr3cw20Ga6?usp=sharing)
|
15 |
|
16 |
-
|
17 |
QA-Evaluation-Metrics is a fast and lightweight Python package for evaluating question-answering models. It provides various basic metrics to assess the performance of QA models. Check out our paper [**PANDA**](https://arxiv.org/abs/2402.11161), a matching method going beyond token-level matching and is more efficient than LLM matchings but still retains competitive evaluation performance of transformer LLM models.
|
18 |
|
19 |
|
@@ -81,9 +80,9 @@ question = "Which movie is loosley based off the Brother Grimm's Iron Henry?"
|
|
81 |
cfm = CFMatcher()
|
82 |
scores = cfm.get_scores(reference_answer, candidate_answer, question)
|
83 |
match_result = cfm.cf_match(reference_answer, candidate_answer, question)
|
84 |
-
print("Score: %s;
|
85 |
'''
|
86 |
-
Score: {'the frog prince': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.7131625951317375}, 'the princess and the frog': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.854451712151719}};
|
87 |
'''
|
88 |
```
|
89 |
|
|
|
13 |
|
14 |
[![PyPI version qa-metrics](https://img.shields.io/pypi/v/qa-metrics.svg)](https://pypi.org/project/qa-metrics/) [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17b7vrZqH0Yun2AJaOXydYZxr3cw20Ga6?usp=sharing)
|
15 |
|
|
|
16 |
QA-Evaluation-Metrics is a fast and lightweight Python package for evaluating question-answering models. It provides various basic metrics to assess the performance of QA models. Check out our paper [**PANDA**](https://arxiv.org/abs/2402.11161), a matching method going beyond token-level matching and is more efficient than LLM matchings but still retains competitive evaluation performance of transformer LLM models.
|
17 |
|
18 |
|
|
|
80 |
cfm = CFMatcher()
|
81 |
scores = cfm.get_scores(reference_answer, candidate_answer, question)
|
82 |
match_result = cfm.cf_match(reference_answer, candidate_answer, question)
|
83 |
+
print("Score: %s; PD Match: %s" % (scores, match_result))
|
84 |
'''
|
85 |
+
Score: {'the frog prince': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.7131625951317375}, 'the princess and the frog': {'The movie "The Princess and the Frog" is loosely based off the Brother Grimm\'s "Iron Henry"': 0.854451712151719}}; PD Match: True
|
86 |
'''
|
87 |
```
|
88 |
|