| library_name: peft | |
| base_model: openlm-research/open_llama_3b | |
| # OpenLLaMa 3B PersonaChat | |
| This is a LoRA finetune of [OpenLLaMa 3B](https://huggingface.co/openlm-research/open_llama_3b) on the [personachat-truecased](https://huggingface.co/datasets/bavard/personachat_truecased) dataset with 3 epochs of 500 steps. | |
| ## Use | |
| Before using this model, you must first add these extra tokens: | |
| ```python | |
| tokenizer.add_special_tokens({"additional_special_tokens": ["<|human|>", "<|bot|>", "<|endoftext|>"]}) | |
| model.resize_token_embeddings(len(tokenizer)) | |
| ``` | |
| The model is finetuned with the format is as follows: | |
| ``` | |
| Personality: | |
| - [...] | |
| - [...] | |
| <|human|>Hi there!<|endoftext|><|bot|>Hello!<|endoftext|> | |
| ``` | |
| To use this model, you must first define the personalities. | |
| ```python | |
| personalities = """Personality: | |
| - [...] | |
| - [...] | |
| """ | |
| ``` | |
| Then, follow the format: | |
| ```python | |
| user = input(">>> ") | |
| prompt = f"{personalities}<|human|>{user}<|endoftext|><|bot|>" | |
| ``` | |
| ## Naming Format | |
| [model name]-finetuned-[dataset]-e[number of epochs]-s[number of steps] | |
| ## Training procedure | |
| The following `bitsandbytes` quantization config was used during training: | |
| - load_in_8bit: True | |
| - load_in_4bit: False | |
| - llm_int8_threshold: 6.0 | |
| - llm_int8_skip_modules: None | |
| - llm_int8_enable_fp32_cpu_offload: False | |
| - llm_int8_has_fp16_weight: False | |
| - bnb_4bit_quant_type: fp4 | |
| - bnb_4bit_use_double_quant: False | |
| - bnb_4bit_compute_dtype: float32 | |
| ### Framework versions | |
| - PEFT 0.4.0.dev0 | |