nithinraok
commited on
Commit
•
df9cc09
1
Parent(s):
74ec759
Update README.md
Browse filesAdd Fairness evaluation
README.md
CHANGED
@@ -270,6 +270,28 @@ The following tables summarizes the performance of the available models in this
|
|
270 |
|
271 |
These are greedy WER numbers without external LM. More details on evaluation can be found at [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)
|
272 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
273 |
## NVIDIA Riva: Deployment
|
274 |
|
275 |
[NVIDIA Riva](https://developer.nvidia.com/riva), is an accelerated speech AI SDK deployable on-prem, in all clouds, multi-cloud, hybrid, on edge, and embedded.
|
@@ -295,6 +317,8 @@ Check out [Riva live demo](https://developer.nvidia.com/riva#demos).
|
|
295 |
|
296 |
[6] [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)
|
297 |
|
|
|
|
|
298 |
|
299 |
## Licence
|
300 |
|
|
|
270 |
|
271 |
These are greedy WER numbers without external LM. More details on evaluation can be found at [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)
|
272 |
|
273 |
+
|
274 |
+
## Model Fairness Evaluation
|
275 |
+
|
276 |
+
As presented in paper "Towards Measuring Fairness in AI: the Casual Conversations Dataset" for Fairness evaluation of Models, we evaluated parakeet-tdt-1.1b on the CausalConversations-v1
|
277 |
+
dataset and results are reported as follows:
|
278 |
+
|
279 |
+
### Gender Bias:
|
280 |
+
|
281 |
+
| Gender | Male | Female | N/A | Other |
|
282 |
+
| :--- | :--- | :--- | :--- | :--- |
|
283 |
+
| Num utterances | 19325 | 24532 | 926 | 33 |
|
284 |
+
| % WER | 19.56 | 16.87 | 21.45 | 38.88 |
|
285 |
+
|
286 |
+
### Age Bias:
|
287 |
+
|
288 |
+
| Age Group | $(18-30)$ | $(31-45)$ | $(46-85)$ | $(1-100)$ |
|
289 |
+
| :--- | :--- | :--- | :--- | :--- |
|
290 |
+
| Num utterances | 15956 | 14585 | 13349 | 43890 |
|
291 |
+
| % WER | 17.95 | 18.28 | 17.95 | 18.06 |
|
292 |
+
|
293 |
+
|
294 |
+
|
295 |
## NVIDIA Riva: Deployment
|
296 |
|
297 |
[NVIDIA Riva](https://developer.nvidia.com/riva), is an accelerated speech AI SDK deployable on-prem, in all clouds, multi-cloud, hybrid, on edge, and embedded.
|
|
|
317 |
|
318 |
[6] [HuggingFace ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard)
|
319 |
|
320 |
+
[7] [Towards Measuring Fairness in AI: the Casual Conversations Dataset](https://arxiv.org/abs/2104.02821)
|
321 |
+
|
322 |
|
323 |
## Licence
|
324 |
|