updating n_kv_heads in config.json
#2
by
fwtan
- opened
In config.json, it seems n_kv_heads was set to be the default value (32), which does not align with the actual setting (i.e. 24). The same case for other Sheared Llama models.
This may lead to an error when using transformers with a version > 4.28.
I just saw, shouldn't it be 16 like in the base shear?
I just saw, shouldn't it be 16 like in the base shear?
Yes, you're correct! It should be 16 for this model. Transformers <= 4.28 uses n_kv_heads == n_heads by default.