TheBloke commited on
Commit
9aa3d7e
1 Parent(s): 3fa9b5a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -5
README.md CHANGED
@@ -54,12 +54,11 @@ I have quantised the GGML files in this repo with the latest version. Therefore
54
  I use the following command line; adjust for your tastes and needs:
55
 
56
  ```
57
- ./main -t 12 -m Samantha-7B.v3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.
58
- ### Instruction:
59
- Write a story about llamas
60
- ### Response:"
61
  ```
62
- Change `-t 12` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
 
 
63
 
64
  If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
65
 
 
54
  I use the following command line; adjust for your tastes and needs:
55
 
56
  ```
57
+ ./main -ngl 32 -t 10 -m Samantha-7B.v3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Hi Samantha, how are you today? ### Response:".
 
 
 
58
  ```
59
+ Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
60
+
61
+ Remove `-ngl 32` if you don't have GPU acceleration support. `-ngl 32` loads 32 layers onto the GPU, requiring 3.5 (q4_0) - 6.5GB (q8_0) VRAM
62
 
63
  If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
64