Edit Models filters

Inference status

Misc

compressed-tensors

AutoTrain Compatible

Inference Endpoints

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

258

Full-text search

Active filters: compressed-tensors

nm-testing/Mixtral-8x7B-Instruct-v0.1-W8A16-quantized

Updated 28 days ago • 162

nm-testing/Phi-3-mini-128k-instruct-FP8

Updated 28 days ago • 8.23k

nm-testing/Meta-Llama-3-8B-Instruct-fp8-compressed

Updated 28 days ago • 12

neuralmagic/gemma-2-9b-it-quantized.w8a16

Text Generation • Updated 28 days ago • 2.89k • 1

neuralmagic/gemma-2-2b-it-quantized.w8a16

Text Generation • Updated 28 days ago • 75 • 1

neuralmagic/gemma-2-2b-quantized.w8a16

Text Generation • Updated 28 days ago • 66

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-channel-quantized

Updated 28 days ago • 47

neuralmagic/Phi-3-small-128k-instruct-quantized.w8a16

Text Generation • Updated 28 days ago • 591

neuralmagic/gemma-2-2b-it-quantized.w4a16

Text Generation • Updated 28 days ago • 192

neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a16

Text Generation • Updated 28 days ago • 63

neuralmagic/Mistral-Nemo-Instruct-2407-quantized.w4a16

Text Generation • Updated 28 days ago • 1.06k • 2

neuralmagic/gemma-2-2b-it-quantized.w8a8

Text Generation • Updated 28 days ago • 468

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated 28 days ago • 470 • 2

nm-testing/Meta-Llama-3-8B-Instruct-fp8-hf_compat

Updated Aug 30 • 11

neuralmagic/SmolLM-360M-Instruct-quantized.w8a8

Text Generation • Updated 28 days ago • 21

neuralmagic/SmolLM-135M-Instruct-quantized.w8a8

Text Generation • Updated 28 days ago • 565

nm-testing/SmolLM-135M-Instruct-quantized.w4a16

Text Generation • Updated 28 days ago • 15

nm-testing/SmolLM-360M-Instruct-quantized.w4a16

Updated 28 days ago • 8

nm-testing/SmolLM-1.7B-Instruct-quantized.w4a16

Text Generation • Updated 28 days ago • 15

neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a8

Text Generation • Updated 28 days ago • 28

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16_channel-e2e

Text Generation • Updated about 7 hours ago • 871

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16-e2e

Text Generation • Updated about 9 hours ago • 547

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16_channel-e2e

Text Generation • Updated about 7 hours ago • 892

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16-e2e

Text Generation • Updated about 9 hours ago • 533

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_channel-e2e

Text Generation • Updated about 9 hours ago • 31

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_tensor-e2e

Text Generation • Updated about 9 hours ago • 30

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8-e2e

Text Generation • Updated about 7 hours ago • 606

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8_DYNAMIC-e2e

Text Generation • Updated about 9 hours ago • 36

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8-e2e

Text Generation • Updated about 8 hours ago • 37

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8_tensor_weight_static_per_tensor_act-e2e

Text Generation • Updated about 7 hours ago • 479