unsloth

/

mistral-7b-v0.2-bnb-4bit

like 13

we get this error when using Inference API. In fact, this is with all unsloth models
ERROR ----No package metadata was found for bitsandbytes

Oh weird. Did you try pip install bitsandbytes
python -m bitsandbytes
Can you provide the exact installation instructions you used?

And what are you using for inference? Hugging Face? Google Colab?