Commit
•
32fbafa
1
Parent(s):
467499e
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc-by-nc-4.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
---
|
| 4 |
+
c4ai-command-r-plus, quantized to GPTQ with these parameters:
|
| 5 |
+
|
| 6 |
+
python3 quant.py aCohereForAI/c4ai-command-r-plus /workspace/commandplus custom --bits 4 --group_size 128 --desc_act 1 --damp 0.1 --dtype float16 --seqlen 16384 --num_samples 256 --cache_examples 0 --trust_remote_code
|
| 7 |
+
|
| 8 |
+
The dataset used was openerotica/erotiquant2. I have included a script reconstitute.py to merge the files into one. Depending on the backend you might need to delete the index file after the files have been merged. I'll try to do this all in a better way once I work after I test out how marlin stacks up to exl2 for this model.
|