File size: 1,713 Bytes
badeea0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
---
### Overview:
description:
This is a llama2 7B HF chat model fine-tuned on 122k code instructions. In my early experiments it seems to be doing very well.
additional_info:
It's a bottom of the barrel model 😂 but after quantization it can be
valuable for sure. It definitely proves that a 7B can be useful for boilerplate
code stuff though.
### Plans:
next_steps: "I've a few things in mind and after that this will be more valuable."
tasks:
- name: "I'll quantize these"
timeline: "Possibly tonight or tomorrow in the day"
result: "Then it can be run locally with 4G ram."
- name: "I've used alpaca style instruction tuning"
improvement: |
I'll switch to llama2 style [INST]<<SYS>> style and see if
it improves anything.
- name: "HumanEval report and checking for any training data leaks"
- attempt: "I'll try 8k context via RoPE enhancement"
hypothesis: "Let's see if that degrades performance or not."
commercial_use: |
So far I think this can be used commercially but this is a adapter on Meta's llama2 with
some gating issues so that is there.
contact_info: "If you find any issues or want to just holler at me, you can reach out to me - https://twitter.com/4evaBehindSOTA"
### Library:
name: "peft"
### Training procedure:
quantization_config:
load_in_8bit: False
load_in_4bit: True
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: "nf4"
bnb_4bit_use_double_quant: False
bnb_4bit_compute_dtype: "float16"
### Framework versions:
PEFT: "0.5.0.dev0" |