metadata
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
language:
- en
tags:
- transformers
- safetensors
- llama
- text-generation
- conversational
- en
- dataset:cerebras/SlimPajama-627B
- dataset:bigcode/starcoderdata
- dataset:HuggingFaceH4/ultrachat_200k
- dataset:HuggingFaceH4/ultrafeedback_binarized
- license:apache-2.0
- autotrain_compatible
- endpoints_compatible
- text-generation-inference
- region:us
license: apache-2.0
inference: false
datasets:
- cerebras/SlimPajama-627B
- bigcode/starcoderdata
- HuggingFaceH4/ultrachat_200k
- HuggingFaceH4/ultrafeedback_binarized
quantized_by: pbatra
TinyLlama-1.1B-Chat-v1.0
This repository contains quantized versions of the model from the original repository: TinyLlama/TinyLlama-1.1B-Chat-v1.0.
| Name | Quantization Method | Size (GB) |
|---|---|---|
| tinyllama-1.1b-chat-v1.0.fp16.gguf | FP16 | 2.05 |
| tinyllama-1.1b-chat-v1.0.fp32.gguf | FP32 | 4.10 |
| tinyllama-1.1b-chat-v1.0.Q2_K.gguf | q2_k | 0.40 |
| tinyllama-1.1b-chat-v1.0.Q3_K_S.gguf | q3_k_s | 0.47 |
| tinyllama-1.1b-chat-v1.0.Q3_K_M.gguf | q3_k_m | 0.51 |
| tinyllama-1.1b-chat-v1.0.Q3_K_L.gguf | q3_k_l | 0.55 |
| tinyllama-1.1b-chat-v1.0.Q4_0.gguf | q4_0 | 0.59 |
| tinyllama-1.1b-chat-v1.0.Q4_K_S.gguf | q4_k_s | 0.60 |
| tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf | q4_k_m | 0.62 |
| tinyllama-1.1b-chat-v1.0.Q5_0.gguf | q5_0 | 0.71 |
| tinyllama-1.1b-chat-v1.0.Q5_K_S.gguf | q5_k_s | 0.71 |
| tinyllama-1.1b-chat-v1.0.Q5_K_M.gguf | q5_k_m | 0.73 |
| tinyllama-1.1b-chat-v1.0.Q6_K.gguf | q6_k | 0.84 |
| tinyllama-1.1b-chat-v1.0.Q8_0.gguf | q8_0 | 1.09 |