Spaces:
Sleeping
Sleeping
Report for bhadresh-savani/distilbert-base-uncased-emotion
#145
by
ZeroCommand
- opened
Hi Team,
This is a report from Giskard Bot Scan 🐢.
We have identified 2 potential vulnerabilities in your model based on an automated scan.
This automated analysis evaluated the model on the dataset dair-ai/emotion (subset split, split validation).
👉Performance issues (1)
For records in the dataset where text contains "know", the Precision is 5.25% lower than the global Precision.
| Level | Data slice | Metric | Deviation |
|---|---|---|---|
| medium 🟡 | text contains "know" |
Precision = 0.885 | -5.25% than global |
Taxonomy
avid-effect:performance:P0204Examples are too long to be displayed in this area.
👉Robustness issues (1)
When feature “text” is perturbed with the transformation “Add typos”, the model changes its prediction in 22.2% of the cases. We expected the predictions not to be affected by this transformation.
| Level | Metric | Transformation | Deviation |
|---|---|---|---|
| major 🔴 | Fail rate = 0.222 | Add typos | 222/1000 tested samples (22.2%) changed prediction after perturbation |
Taxonomy
avid-effect:performance:P0201Examples are too long to be displayed in this area.