sandwichdoge
/

Nous-Capybara-limarpv3-34B-4.65bpw-hb6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

sandwichdoge commited on Nov 25, 2023

Commit

afe4f22

•

1 Parent(s): 1799b90

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -2,4 +2,4 @@ Original model: https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B
 Using cleaned pippa.parquet as calibration dataset.
-5bpw - Can run 10k context in ~23.3GB VRAM with 8bit-cache option.


2
3	Using cleaned pippa.parquet as calibration dataset.
4
5	+ 4.65bpw - Can run 21k context in ~23.3GB VRAM with 8bit-cache option.