Edit Models filters

Inference status

Misc

AutoTrain Compatible

Inference Endpoints

text-generation-inference

4-bit precision

8-bit precision

Misc with no match

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

130

Full-text search

Active filters: vllm

neuralmagic/gemma-2-9b-it-quantized.w8a8

Text Generation • Updated Oct 9 • 217 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated about 1 month ago • 896 • 2

mistral-community/pixtral-12b-240910

Image-Text-to-Text • Updated Oct 1 • 382

neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 18.7k • 10

neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.27k • 2

neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.79k • 2

neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 16.6k • 3

neuralmagic/Phi-3.5-mini-instruct-FP8-KV

Text Generation • Updated Oct 1 • 303 • 1

neuralmagic/pixtral-12b-FP8-dynamic

Text Generation • Updated 8 days ago • 2.54k • 1

mlx-community/Ministral-8B-Instruct-2410-bf16

Updated 23 days ago • 538 • 2

mlx-community/Ministral-8B-Instruct-2410-4bit

Updated 23 days ago • 243 • 4

mlx-community/Ministral-8B-Instruct-2410-8bit

Updated 23 days ago • 184 • 1

neuralmagic/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic

Text Generation • Updated 24 days ago • 11.1k • 8

aashish1904/Ministral-8B-Instruct-2410-HF-Q4_K_M-GGUF

Updated 20 days ago • 97 • 1

aashish1904/Ministral-8B-Instruct-2410-HF-Q2_K-GGUF

Updated 20 days ago • 31 • 2

Inferless/deciLM-7B-GPTQ

Text Generation • Updated Jan 4 • 6 • 1

Inferless/SOLAR-10.7B-Instruct-v1.0-GPTQ

Text Generation • Updated Jan 4 • 12 • 2

Inferless/Mixtral-8x7B-v0.1-int8-GPTQ

Text Generation • Updated Jan 25 • 52 • 2

neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV

Text Generation • Updated Jun 19 • 16.2k • 6

neuralmagic/Meta-Llama-3-70B-Instruct-FP8

Text Generation • Updated Jul 18 • 53.2k • 10

neuralmagic/Qwen2-0.5B-Instruct-FP8

Text Generation • Updated Jul 18 • 1.09k • 2

neuralmagic/Qwen2-1.5B-Instruct-FP8

Text Generation • Updated Jul 18 • 33

neuralmagic/Qwen2-7B-Instruct-FP8

Text Generation • Updated Jul 18 • 1.33k • 1

nm-testing/SparseLlama-3-8B-pruned_50.2of4-FP8

Text Generation • Updated Jun 25 • 18

FlorianJc/Hermes-2-Pro-Mistral-7B-vllm-fp8

Text Generation • Updated Jul 17 • 8

FlorianJc/openchat-3.6-8b-20240522-vllm-fp8

Text Generation • Updated Jul 17 • 18

FlorianJc/Llama3-ChatQA-1.5-8B-vllm-fp8

Text Generation • Updated Jul 17 • 12

Rallio67/magnum-72B-FP8

Text Generation • Updated Jun 26 • 11

neuralmagic/Meta-Llama-3-70B-Instruct-FP8-KV

Text Generation • Updated Jun 26 • 805 • 2

neuralmagic/Mistral-7B-Instruct-v0.3-FP8

Text Generation • Updated Jul 18 • 1.83k • 2