kennylam/Taiwan-LLM-13B-v2.0-chat-exl2

Taiwan-LLM-13B-v2.0-chat with ExLlamaV2 Quantization

Original model 原始模型: https://huggingface.co/yentinglin/Taiwan-LLM-13B-v2.0-chat

This is a quantizated model from yentinglin/Taiwan-LLM-13B-v2.0-chat in exl2 format.

You are currently at the main branch, which provides only measurement.json used in the ExLlamaV2 quantization. Please take a look of your choices in following table of branches.

這裡是main branch, 只提供EvLlamaV2量化時所用到的measurement.json檔案。

8.0bpw-h8 8 bits per weight.

6.0bpw-h6 6 bits per weight.

4.0bpw-h6 4 bits per weight.

3.0bpw-h6 3 bits per weight.

2.0bpw-h6 2 bits per weight.

Citation

If you find Taiwan LLM is useful in your work, please cite it with:

@misc{lin2023taiwan,
      title={Taiwan LLM: Bridging the Linguistic Divide with a Culturally Aligned Language Model}, 
      author={Yen-Ting Lin and Yun-Nung Chen},
      year={2023},
      eprint={2311.17487},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Acknowledgement

Taiwan LLM v2 is conducted in collaboration with Ubitus K.K.. Ubitus provides valuable compute resources for the project.