VictorSanh commited on
Commit
96b2b8c
1 Parent(s): 50966bf

image net a bit of formatting

Browse files
Files changed (1) hide show
  1. README.md +7 -8
README.md CHANGED
@@ -176,16 +176,15 @@ We note that since IDEFICS was trained on PMD (which contains COCO), the evaluat
176
  | | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
177
  | | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
178
 
179
- For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance.
180
 
181
- ImageNet-1k Evaluation:
182
- | Model | Shots | ImageNet-1k |
183
- |:-----------|--------:|-----------:|
184
- | IDEFIX 80B | 16, 1k support set | 65.4 |
185
- | | 16, RICES 5k support set | 72.9 |
186
  <br>
187
- | IDEFIX 9B | 16, 1k support set | 53.5 |
188
- | | 16, RICES 5k support set | 64.5 |
189
 
190
  Fairness Evaluations:
191
  | Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |
 
176
  | | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
177
  | | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
178
 
179
+ For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance. This is the Retrieval-based In-Context Example Selection (RICES in short) approach introduced by [Yang et al. (2021)](https://arxiv.org/abs/2109.05014).
180
 
181
+ | Model | Shots | Support set size | Shots selection | ImageNet-1k<br>Top-1 acc. |
182
+ |:-----------|--------:|-----------------:|:----------------|--------------------------:|
183
+ | IDEFIX 80B | 16 | 1K | Random | 65.4 |
184
+ | | 16 | 5K | RICES | 72.9 |
 
185
  <br>
186
+ | IDEFIX 9B | 16 | 1K | Random | 53.5 |
187
+ | | 16 | 5K | RICES | 64.5 |
188
 
189
  Fairness Evaluations:
190
  | Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |