| **Testing** | |
| ===================== | |
| **Hyperparameters** | |
| --------------- | |
| * **Temperature**: 0.9 | |
| * **Penalize repeat sequence**: 1.05 | |
| * **Consider N tokens for penalize**: 256 | |
| * **Penalize repetition of newlines** | |
| * **Top-K sampling**: 40 | |
| * **Top-P sampling**: 0.95 | |
| * **Min-P sampling**: 0.05 | |
| **LLaMAcpp Version** | |
| ------------------ | |
| * b3527-2-g2d5dd7bb | |
| **File** | |
| ------------------ | |
| * Cathallama-70B.Q4_0.gguf | |
| **Test Cases** | |
| -------------- | |
| | Test Case | Result | | |
| | --- | --- | | |
| | Ball on cup | OK | | |
| | Door window combination | OK | | |
| | Big duck small horse | OK | | |
| | JSON | OK | | |
| | Killers | OK | | |
| | Dragon | OK | | |
| | Poem | OK | | |
| | Jane faster | OK | | |
| | Shirts | OK | | |
| | Sisters | OK | | |
| | Python snake game | OK* | | |
| | Story | OK | | |
| *best I ever saw on local LLMs including Qwen2 72b at 8bpw, Llama 3 70b 8bpw | |
| Note: See sample generations on the main folder of the repo. |