Update README.md

README.md

@@ -1,13 +1,49 @@
 ---
 license: apache-2.0
+base_model: mistralai/Mixtral-8x22B-Instruct-v0.1
+inference: false
+model_creator: MaziyarPanahi
+model_name: Mixtral-8x22B-Instruct-v0.1-GGUF
+pipeline_tag: text-generation
+quantized_by: MaziyarPanahi
+tags:
+- quantized
+- 2-bit
+- 3-bit
+- 4-bit
+- 5-bit
+- 6-bit
+- 8-bit
+- 16-bit
+- GGUF
+- mixtral
+- moe
 language:
-- en
 - fr
+- en
 - es
-- de
 - it
+- de
 ---
 
+# Mixtral-8x22B-Instruct-v0.1-GGUF
+
+The GGUF and quantized models here are based on [mistralai/Mixtral-8x22B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1) model
+
+## How to download
+You can download only the quants you need instead of cloning the entire repository as follows:
+
+```
+huggingface-cli download MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF --local-dir . --include '*Q2_K*gguf'
+```
+
+## Load sharded model
+
+`llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files.
+
+```sh
+llama.cpp/main -m Mixtral-8x22B-Instruct-v0.1.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e
+```
 
 
 Original README