latimar commited on
Commit
1c6adcc
1 Parent(s): e793e71

Update README

Browse files
Files changed (1) hide show
  1. README.md +13 -8
README.md CHANGED
@@ -33,14 +33,19 @@ There are the following branches:
33
 
34
  * Calibration dataset used for conversion: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)
35
  * Evaluation dataset used to calculate perplexity: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/validation/0000.parquet)
 
 
36
  * PPL max seq. length used: 1792 (2048 with 5.0-bpw-h8 causes OOM on RTX 4090 when evaluating ppl, so had to go down a bit)
37
 
38
 
39
- | BPW | Perplexity | File Size (Gb) |
40
- | ----------- | ----------- | -------------- |
41
- | 2.55-h6 | 15.0901 | 10.56 |
42
- | 2.75-h6 | 13.6153 | 11.33 |
43
- | 3.8-h6 | 6.8803 | 15.37 |
44
- | 4.125-h6 | 6.8095 | 16.65 |
45
- | 4.625-h6 | 6.7992 | 18.58 |
46
- | 5.0-h8 | 6.7785 | 20.09 |
 
 
 
 
33
 
34
  * Calibration dataset used for conversion: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)
35
  * Evaluation dataset used to calculate perplexity: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/validation/0000.parquet)
36
+ * Calibration dataset used for conversion of `5_0-bpw-h8-ev`: [wizardLM-evol-instruct_70k](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k/blob/refs%2Fconvert%2Fparquet/default/train/0000.parquet)
37
+ * Evaluation dataset used to calculate ppl for `Evol-Ins`: : [nikrosh-evol-instruct](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/blob/refs%2Fconvert%2Fparquet/default/train/0000.parquet)
38
  * PPL max seq. length used: 1792 (2048 with 5.0-bpw-h8 causes OOM on RTX 4090 when evaluating ppl, so had to go down a bit)
39
 
40
 
41
+ | BPW | PPL on Wiki | PPL on Evol-Ins | File Size (Gb) |
42
+ | ----------- | ----------- | --------------- | -------------- |
43
+ | 2.55-h6 | 15.0901 | | 10.56 |
44
+ | 2.75-h6 | 13.6153 | | 11.33 |
45
+ | 3.8-h6 | 6.8803 | | 15.37 |
46
+ | 4.125-h6 | 6.8095 | | 16.65 |
47
+ | 4.625-h6 | 6.7992 | 2.0499 | 18.58 |
48
+ | 5.0-h8 | 6.7785 | 2.0448 | 20.09 |
49
+ | 5.0-h8-ev | 6.9376 | 2.0430 | 20.09 |
50
+
51
+