Arm optimized quants

#113
by SaisExperiments - opened

Is it possible to have the
Q4_0_4_4
Q4_0_4_8
Q4_0_8_8
formats added?

ggml.ai org
edited Aug 28

Thanks for the feature request; let's wait a bit to see if we get more requests for this.

reach-vb changed discussion status to closed
reach-vb changed discussion status to open

Sign up or log in to comment