VictorSanh
commited on
Commit
•
96b2b8c
1
Parent(s):
50966bf
image net a bit of formatting
Browse files
README.md
CHANGED
@@ -176,16 +176,15 @@ We note that since IDEFICS was trained on PMD (which contains COCO), the evaluat
|
|
176 |
| | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
|
177 |
| | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
|
178 |
|
179 |
-
For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance.
|
180 |
|
181 |
-
ImageNet-1k
|
182 |
-
|
183 |
-
|
184 |
-
|
|
185 |
-
| | 16, RICES 5k support set | 72.9 |
|
186 |
<br>
|
187 |
-
| IDEFIX 9B | 16
|
188 |
-
| | 16
|
189 |
|
190 |
Fairness Evaluations:
|
191 |
| Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |
|
|
|
176 |
| | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
|
177 |
| | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
|
178 |
|
179 |
+
For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance. This is the Retrieval-based In-Context Example Selection (RICES in short) approach introduced by [Yang et al. (2021)](https://arxiv.org/abs/2109.05014).
|
180 |
|
181 |
+
| Model | Shots | Support set size | Shots selection | ImageNet-1k<br>Top-1 acc. |
|
182 |
+
|:-----------|--------:|-----------------:|:----------------|--------------------------:|
|
183 |
+
| IDEFIX 80B | 16 | 1K | Random | 65.4 |
|
184 |
+
| | 16 | 5K | RICES | 72.9 |
|
|
|
185 |
<br>
|
186 |
+
| IDEFIX 9B | 16 | 1K | Random | 53.5 |
|
187 |
+
| | 16 | 5K | RICES | 64.5 |
|
188 |
|
189 |
Fairness Evaluations:
|
190 |
| Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |
|