Quantization of Mixtral-8x7B (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) Using llama.cpp, PR add Mixtral support #4406 (https://github.com/ggerganov/llama.cpp/pull/4406) Instructions to run: ./main -m ./models/mixtral-8x7b-32k-q4_0.gguf \ -p "I believe the meaning of life is" \ -ngl 999 -s 1 -n 128 -t 8