starble-dev
commited on
Commit
•
f9b858a
1
Parent(s):
693dbfe
Update README.md
Browse files
README.md
CHANGED
@@ -34,11 +34,11 @@ Use:
|
|
34 |
llama-server.exe -m .\models\Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf -b 512 -ub 512 -c 4096 -ngl 100
|
35 |
```
|
36 |
|
37 |
-
Set `-b` to batch size
|
38 |
-
Set `-ub` to physical batch size
|
39 |
-
Set `-c` to context size
|
40 |
-
Set `-ngl` to amount of layers to load onto GPU
|
41 |
-
Change the path to where the model is actually stored.
|
42 |
If you need more clarification on parameters check out the [llama.cpp Server Docs](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md)
|
43 |
|
44 |
**License:**
|
|
|
34 |
llama-server.exe -m .\models\Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf -b 512 -ub 512 -c 4096 -ngl 100
|
35 |
```
|
36 |
|
37 |
+
Set `-b` to batch size<br>
|
38 |
+
Set `-ub` to physical batch size<br>
|
39 |
+
Set `-c` to context size<br>
|
40 |
+
Set `-ngl` to amount of layers to load onto GPU<br>
|
41 |
+
Change the path to where the model is actually stored. <br>
|
42 |
If you need more clarification on parameters check out the [llama.cpp Server Docs](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md)
|
43 |
|
44 |
**License:**
|