|
--- |
|
license: llama3.1 |
|
--- |
|
**Disclaimer**: |
|
|
|
The model is reproduced based on the paper *VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models* [github](https://github.com/microsoft/vptq) |
|
|
|
The model itself is sourced from a community release. |
|
|
|
It is intended only for experimental purposes. |
|
|
|
Users are responsible for any consequences arising from the use of this model. |
|
|
|
**Note**: |
|
|
|
The PPL test results are for reference only and were collected using GPTQ testing script. |
|
|
|
```json |
|
{ |
|
"ctx_2048": { |
|
"wikitext2": 6.650720119476318 |
|
}, |
|
"ctx_4096": { |
|
"wikitext2": 6.269611358642578 |
|
}, |
|
"ctx_8192": { |
|
"wikitext2": 6.075358867645264 |
|
} |
|
} |
|
``` |
