Update READMD.md Ollama section
#5
by
pacozaa
- opened
README.md
CHANGED
@@ -225,6 +225,20 @@ curl --location 'http://localhost:8000/completion' \
|
|
225 |
}'
|
226 |
```
|
227 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
228 |
### GPU Memory Requirements
|
229 |
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|
230 |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
|
|
|
225 |
}'
|
226 |
```
|
227 |
|
228 |
+
### Ollama
|
229 |
+
|
230 |
+
There are two ways to run on ollama
|
231 |
+
|
232 |
+
1. From this repo Modelfile and 4 bit quantized gguf
|
233 |
+
```bash
|
234 |
+
ollama create -f ./Modelfile
|
235 |
+
```
|
236 |
+
|
237 |
+
2. From Ollama CLI
|
238 |
+
```bash
|
239 |
+
ollama run pacozaa/openthaigpt
|
240 |
+
```
|
241 |
+
|
242 |
### GPU Memory Requirements
|
243 |
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|
244 |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
|