Update README.md
Browse files
README.md
CHANGED
@@ -110,13 +110,15 @@ Refer to the Provided Files table below to see what files use which methods, and
|
|
110 |
I use the following command line; adjust for your tastes and needs:
|
111 |
|
112 |
```
|
113 |
-
./main -t 10 -ngl 32 -m hermes-llongma-2-13b-8k.ggmlv3.q4_K_M.bin --color -c
|
114 |
```
|
115 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
116 |
|
117 |
Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
|
118 |
|
119 |
-
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins
|
|
|
|
|
120 |
|
121 |
## How to run in `text-generation-webui`
|
122 |
|
|
|
110 |
I use the following command line; adjust for your tastes and needs:
|
111 |
|
112 |
```
|
113 |
+
./main -t 10 -ngl 32 -m hermes-llongma-2-13b-8k.ggmlv3.q4_K_M.bin --color -c 8192 --rope-freq-base 10000 --rope-freq-scale 0.5 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
|
114 |
```
|
115 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
116 |
|
117 |
Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
|
118 |
|
119 |
+
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`.
|
120 |
+
|
121 |
+
The context length is set with `-c 8192`. Because RoPE scaling is required, you should also use `--rope-freq-base 10000 --rope-freq-scale 0.5` for 8K context.
|
122 |
|
123 |
## How to run in `text-generation-webui`
|
124 |
|