starble-dev
commited on
Commit
•
9af06e7
1
Parent(s):
7635710
Update README.md
Browse files
README.md
CHANGED
@@ -4,7 +4,7 @@ tags:
|
|
4 |
- mistral
|
5 |
- conversational
|
6 |
- text-generation-inference
|
7 |
-
base_model:
|
8 |
library_name: transformers
|
9 |
---
|
10 |
|
@@ -12,11 +12,8 @@ library_name: transformers
|
|
12 |
> **Sampling:**<br>
|
13 |
> Mistral-Nemo-12B-Instruct-2407 is very sensitive to the temperature sampler, try values near **0.3** or else you will get some weird results. This is mentioned by MistralAI at their [Transformers](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407#transformers) section
|
14 |
|
15 |
-
**Changes:**
|
16 |
-
This model is the original Mistral-Nemo-Instruct-2407 converted to GGUF and quantized using **llama.cpp**.
|
17 |
-
|
18 |
**How to Use:**
|
19 |
-
As of July
|
20 |
|
21 |
```
|
22 |
git clone -b mistral-nemo https://github.com/iamlemec/llama.cpp.git
|
@@ -45,22 +42,10 @@ If you need more clarification on parameters check out the [llama.cpp Server Doc
|
|
45 |
Apache 2.0
|
46 |
|
47 |
**Original Model:**
|
48 |
-
[
|
49 |
-
|
50 |
-
**Note:**
|
51 |
-
MistralAI does not have any affiliation with starble-dev.
|
52 |
|
53 |
# Quants
|
54 |
-
|
55 |
-
| ---- | ---- | ---- |
|
56 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q2_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q2_K.gguf) | Q2_K | 4.79 GB |
|
57 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q3_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q3_K.gguf) | Q3_K | 6.08 GB |
|
58 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q4_K_S.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q4_K_S.gguf) | Q4_K_S | 7.12 GB |
|
59 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q4_K_M.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q4_K_M.gguf) | Q4_K_M | 7.48 GB |
|
60 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q5_K_M.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q5_K_M.gguf) | Q5_K_M | 8.73 GB |
|
61 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q6_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q6_K.gguf) | Q6_K | 10.1 GB |
|
62 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf) | Q8_0 | 13 GB |
|
63 |
-
| [Mistral-Nemo-12B-Instruct-2407-Q8_0_L.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q8_0_L.gguf) | Q8_0 | 13.7 GB |
|
64 |
|
65 |
> [!NOTE]
|
66 |
> **Note: Q8_0_L**<br>
|
|
|
4 |
- mistral
|
5 |
- conversational
|
6 |
- text-generation-inference
|
7 |
+
base_model: BeaverAI/mistral-dory-12b
|
8 |
library_name: transformers
|
9 |
---
|
10 |
|
|
|
12 |
> **Sampling:**<br>
|
13 |
> Mistral-Nemo-12B-Instruct-2407 is very sensitive to the temperature sampler, try values near **0.3** or else you will get some weird results. This is mentioned by MistralAI at their [Transformers](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407#transformers) section
|
14 |
|
|
|
|
|
|
|
15 |
**How to Use:**
|
16 |
+
As of July 20, 2024, llama.cpp does not support Mistral-Nemo-Instruct-2407. However, you can still use it by building from iamlemec's branch **mistral-nemo** at [llama.cpp GitHub repository](https://github.com/iamlemec/llama.cpp/tree/mistral-nemo).
|
17 |
|
18 |
```
|
19 |
git clone -b mistral-nemo https://github.com/iamlemec/llama.cpp.git
|
|
|
42 |
Apache 2.0
|
43 |
|
44 |
**Original Model:**
|
45 |
+
[BeaverAI/mistral-dory-12b](https://huggingface.co/BeaverAI/mistral-dory-12b)
|
|
|
|
|
|
|
46 |
|
47 |
# Quants
|
48 |
+
Quantizing currently
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
49 |
|
50 |
> [!NOTE]
|
51 |
> **Note: Q8_0_L**<br>
|