Update README.md
Browse files
README.md
CHANGED
@@ -54,12 +54,11 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
54 |
I use the following command line; adjust for your tastes and needs:
|
55 |
|
56 |
```
|
57 |
-
./main -t
|
58 |
-
### Instruction:
|
59 |
-
Write a story about llamas
|
60 |
-
### Response:"
|
61 |
```
|
62 |
-
Change `-t
|
|
|
|
|
63 |
|
64 |
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
|
65 |
|
|
|
54 |
I use the following command line; adjust for your tastes and needs:
|
55 |
|
56 |
```
|
57 |
+
./main -ngl 32 -t 10 -m Samantha-7B.v3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Hi Samantha, how are you today? ### Response:".
|
|
|
|
|
|
|
58 |
```
|
59 |
+
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
60 |
+
|
61 |
+
Remove `-ngl 32` if you don't have GPU acceleration support. `-ngl 32` loads 32 layers onto the GPU, requiring 3.5 (q4_0) - 6.5GB (q8_0) VRAM
|
62 |
|
63 |
If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
|
64 |
|