tybrs commited on
Commit
1b1daa8
1 Parent(s): 1273a08

Update README.md

Browse files

Update readme with description

Files changed (1) hide show
  1. README.md +45 -2
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: Bias Auc
3
  emoji: 🏆
4
  colorFrom: gray
5
  colorTo: blue
@@ -8,4 +8,47 @@ pinned: false
8
  license: apache-2.0
9
  ---
10
 
11
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Bias AUC
3
  emoji: 🏆
4
  colorFrom: gray
5
  colorTo: blue
 
8
  license: apache-2.0
9
  ---
10
 
11
+ # Bias AUC
12
+
13
+ ## Description
14
+
15
+ Suite of threshold-agnostic metrics that provide a nuanced view
16
+ of this unintended bias, by considering the various ways that a
17
+ classifier’s score distribution can vary across designated groups.
18
+
19
+ The following are computed:
20
+
21
+ - Subgroup AUC
22
+ - BPSN (Background Positive, Subgroup Negative) AUC
23
+ - BNSP (Background Negative, Subgroup Positive) AUC
24
+ - GMB (Generalized Mean of Bias) AUC
25
+
26
+ ## How to use
27
+
28
+ ```python
29
+ from evaluate import load
30
+
31
+ target = [['Islam'],
32
+ ['Sexuality'],
33
+ ['Sexuality'],
34
+ ['Islam']]
35
+
36
+ label = [0, 0, 1, 1]
37
+
38
+ output = [[0.44452348351478577, 0.5554765462875366],
39
+ [0.4341845214366913, 0.5658154487609863],
40
+ [0.400595098733902, 0.5994048714637756],
41
+ [0.3840397894382477, 0.6159601807594299]]
42
+
43
+ metric = load('Intel/bias_auc')
44
+
45
+ metric.add_batch(target=target,
46
+ label=label,
47
+ output=output)
48
+
49
+ subgroups = set(group for group_list in a for group in group_list) - set(['Disability'])
50
+ metric.compute(target=a,
51
+ label=b,
52
+ output=c,
53
+ subgroups = None)
54
+ ```