starble-dev commited on
Commit
9af06e7
1 Parent(s): 7635710

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -19
README.md CHANGED
@@ -4,7 +4,7 @@ tags:
4
  - mistral
5
  - conversational
6
  - text-generation-inference
7
- base_model: mistralai/Mistral-Nemo-Instruct-2407
8
  library_name: transformers
9
  ---
10
 
@@ -12,11 +12,8 @@ library_name: transformers
12
  > **Sampling:**<br>
13
  > Mistral-Nemo-12B-Instruct-2407 is very sensitive to the temperature sampler, try values near **0.3** or else you will get some weird results. This is mentioned by MistralAI at their [Transformers](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407#transformers) section
14
 
15
- **Changes:**
16
- This model is the original Mistral-Nemo-Instruct-2407 converted to GGUF and quantized using **llama.cpp**.
17
-
18
  **How to Use:**
19
- As of July 19, 2024, llama.cpp does not support Mistral-Nemo-Instruct-2407. However, you can still use it by building from iamlemec's branch **mistral-nemo** at [llama.cpp GitHub repository](https://github.com/iamlemec/llama.cpp/tree/mistral-nemo).
20
 
21
  ```
22
  git clone -b mistral-nemo https://github.com/iamlemec/llama.cpp.git
@@ -45,22 +42,10 @@ If you need more clarification on parameters check out the [llama.cpp Server Doc
45
  Apache 2.0
46
 
47
  **Original Model:**
48
- [Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407)
49
-
50
- **Note:**
51
- MistralAI does not have any affiliation with starble-dev.
52
 
53
  # Quants
54
- | Name | Quant Type | Size |
55
- | ---- | ---- | ---- |
56
- | [Mistral-Nemo-12B-Instruct-2407-Q2_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q2_K.gguf) | Q2_K | 4.79 GB |
57
- | [Mistral-Nemo-12B-Instruct-2407-Q3_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q3_K.gguf) | Q3_K | 6.08 GB |
58
- | [Mistral-Nemo-12B-Instruct-2407-Q4_K_S.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q4_K_S.gguf) | Q4_K_S | 7.12 GB |
59
- | [Mistral-Nemo-12B-Instruct-2407-Q4_K_M.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q4_K_M.gguf) | Q4_K_M | 7.48 GB |
60
- | [Mistral-Nemo-12B-Instruct-2407-Q5_K_M.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q5_K_M.gguf) | Q5_K_M | 8.73 GB |
61
- | [Mistral-Nemo-12B-Instruct-2407-Q6_K.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q6_K.gguf) | Q6_K | 10.1 GB |
62
- | [Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q8_0.gguf) | Q8_0 | 13 GB |
63
- | [Mistral-Nemo-12B-Instruct-2407-Q8_0_L.gguf](https://huggingface.co/starble-dev/Mistral-Nemo-12B-Instruct-2407-GGUF/blob/main/Mistral-Nemo-12B-Instruct-2407-Q8_0_L.gguf) | Q8_0 | 13.7 GB |
64
 
65
  > [!NOTE]
66
  > **Note: Q8_0_L**<br>
 
4
  - mistral
5
  - conversational
6
  - text-generation-inference
7
+ base_model: BeaverAI/mistral-dory-12b
8
  library_name: transformers
9
  ---
10
 
 
12
  > **Sampling:**<br>
13
  > Mistral-Nemo-12B-Instruct-2407 is very sensitive to the temperature sampler, try values near **0.3** or else you will get some weird results. This is mentioned by MistralAI at their [Transformers](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407#transformers) section
14
 
 
 
 
15
  **How to Use:**
16
+ As of July 20, 2024, llama.cpp does not support Mistral-Nemo-Instruct-2407. However, you can still use it by building from iamlemec's branch **mistral-nemo** at [llama.cpp GitHub repository](https://github.com/iamlemec/llama.cpp/tree/mistral-nemo).
17
 
18
  ```
19
  git clone -b mistral-nemo https://github.com/iamlemec/llama.cpp.git
 
42
  Apache 2.0
43
 
44
  **Original Model:**
45
+ [BeaverAI/mistral-dory-12b](https://huggingface.co/BeaverAI/mistral-dory-12b)
 
 
 
46
 
47
  # Quants
48
+ Quantizing currently
 
 
 
 
 
 
 
 
 
49
 
50
  > [!NOTE]
51
  > **Note: Q8_0_L**<br>