sandwichdoge
/

Nous-Capybara-limarpv3-34B-4.65bpw-hb6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Nous-Capybara-limarpv3-34B-4.65bpw-hb6-exl2 / README.md

sandwichdoge's picture

Update README.md

afe4f22 12 months ago

|

204 Bytes

Original model: https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B

Using cleaned pippa.parquet as calibration dataset.

4.65bpw - Can run 21k context in ~23.3GB VRAM with 8bit-cache option.