metadata

license: cc-by-nc-4.0

Overview

This is a fine-tuned 13b parameter LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros

Eval (gpt4 judging)

model	raw score	gpt-3.5 adjusted score
airoboros-13b	17947	98.087
gpt35	18297	100.0
gpt4-x-alpasta-30b	15612	85.33
manticore-13b	15856	86.66
vicuna-13b-1.1	16306	89.12
wizard-vicuna-13b-uncensored	16287	89.01

individual question scores, with shareGPT links (200 prompts generated by gpt-4)

wb-13b-u is Wizard-Vicuna-13b-Uncensored

airoboros-13b	gpt35	gpt4-x-alpasta-30b	manticore-13b	vicuna-13b-1.1	wv-13b-u	link
80	95	70	90	85	60	eval
20	95	40	30	90	80	eval
100	100	100	95	95	100	eval
90	100	85	60	95	100	eval
95	90	80	85	95	75	eval
100	95	90	95	98	92	eval
50	100	80	95	60	55	eval
70	90	80	60	85	40	eval
100	95	50	85	40	60	eval
85	60	55	65	50	70	eval
95	100	85	90	60	75	eval
100	95	70	80	50	85	eval
100	95	80	70	60	90	eval
95	100	70	85	90	90	eval
80	95	90	60	30	85	eval
60	95	0	75	50	40	eval
100	95	90	98	95	95	eval
60	85	40	50	20	0	eval
100	90	85	95	95	80	eval
100	95	100	95	90	95	eval
95	90	96	80	92	88	eval
95	92	90	93	89	91	eval
95	93	90	94	96	92	eval
95	90	93	88	92	85	eval
95	90	85	96	88	92	eval
95	95	90	93	92	91	eval
95	98	80	97	99	96	eval
95	93	90	87	92	89	eval
90	85	95	80	92	75	eval
90	85	95	93	80	92	eval
95	92	90	91	93	89	eval
100	95	90	85	80	95	eval
95	97	93	92	96	94	eval
95	93	94	90	88	92	eval
90	95	98	85	96	92	eval
90	88	85	80	82	84	eval
90	95	85	87	92	88	eval
95	97	96	90	93	92	eval
95	93	92	90	89	91	eval
90	95	93	92	94	91	eval
90	85	95	80	88	75	eval
85	90	95	88	92	80	eval
90	95	92	85	80	87	eval
85	90	95	80	88	75	eval
85	80	75	90	70	82	eval
90	85	95	92	93	80	eval
90	95	75	85	80	70	eval
85	90	80	88	82	83	eval
85	90	95	92	88	80	eval
85	90	80	75	95	88	eval
85	90	80	88	84	92	eval
80	90	75	85	70	95	eval
90	88	85	80	92	83	eval
85	75	90	80	78	88	eval
85	90	80	82	75	88	eval
90	85	40	95	80	88	eval
85	95	90	75	88	80	eval
85	95	90	92	89	88	eval
80	85	75	60	90	70	eval
85	90	87	80	88	75	eval
85	80	75	50	90	80	eval
95	80	90	85	75	82	eval
85	90	80	70	95	88	eval
90	95	70	85	80	75	eval
90	85	70	75	80	60	eval
95	90	70	50	85	80	eval
80	85	40	60	90	95	eval
75	60	80	55	70	85	eval
90	85	60	50	80	95	eval
45	85	60	20	65	75	eval
85	90	30	60	80	70	eval
90	95	80	40	85	70	eval
85	90	70	75	80	95	eval
90	70	50	20	60	40	eval
90	95	75	60	85	80	eval
85	80	60	70	65	75	eval
90	85	80	75	82	70	eval
90	95	80	70	85	75	eval
85	75	30	80	90	70	eval
85	90	50	70	80	60	eval
100	95	98	99	97	96	eval
95	90	92	93	91	89	eval
95	92	90	85	88	91	eval
100	95	98	97	96	99	eval
100	100	100	90	100	95	eval
100	95	98	97	94	99	eval
95	90	92	93	94	91	eval
100	95	98	90	96	95	eval
95	96	92	90	89	93	eval
100	95	93	90	92	88	eval
100	100	98	97	99	100	eval
95	90	92	85	93	94	eval
95	93	90	92	96	91	eval
95	96	92	90	93	91	eval
95	90	92	93	91	89	eval
100	98	95	97	96	99	eval
90	95	85	88	92	87	eval
95	93	90	92	89	88	eval
100	95	97	90	96	94	eval
95	93	90	92	94	91	eval
95	92	90	93	94	88	eval
95	92	60	97	90	96	eval
95	90	92	93	91	89	eval
95	90	97	92	91	93	eval
90	95	93	85	92	91	eval
95	90	40	92	93	85	eval
100	100	95	90	95	90	eval
90	95	96	98	93	92	eval
90	95	92	89	93	94	eval
100	95	100	98	96	99	eval
100	100	95	90	100	90	eval
90	85	88	92	87	91	eval
95	97	90	92	93	94	eval
90	95	85	88	92	89	eval
95	93	90	92	94	91	eval
90	95	85	80	88	82	eval
95	90	60	85	93	70	eval
95	92	94	93	96	90	eval
95	90	85	93	87	92	eval
95	96	93	90	97	92	eval
100	0	0	100	0	0	eval
60	100	0	80	0	0	eval
0	100	60	0	0	90	eval
100	100	0	100	100	100	eval
100	100	100	100	95	100	eval
100	100	100	50	90	100	eval
100	100	100	100	95	90	eval
100	100	100	95	0	100	eval
50	95	20	10	30	85	eval
100	100	60	20	30	40	eval
100	0	0	0	0	100	eval
0	100	60	0	0	80	eval
50	100	20	90	0	10	eval
100	100	100	100	100	100	eval
100	100	100	100	100	100	eval
40	100	95	0	100	40	eval
100	100	100	100	80	100	eval
100	100	100	0	90	40	eval
0	100	100	50	70	20	eval
100	100	50	90	0	95	eval
100	95	90	85	98	80	eval
95	98	90	92	96	89	eval
90	95	75	85	80	82	eval
95	98	50	92	96	94	eval
95	90	0	93	92	94	eval
95	90	85	92	80	88	eval
95	93	75	85	90	92	eval
90	95	88	85	92	89	eval
100	100	100	95	97	98	eval
85	40	30	95	90	88	eval
90	95	92	85	88	93	eval
95	96	92	90	89	93	eval
90	95	85	80	92	88	eval
95	98	65	90	85	93	eval
95	92	96	97	90	89	eval
95	90	92	91	89	93	eval
95	90	80	75	95	90	eval
92	40	30	95	90	93	eval
90	92	85	88	89	87	eval
95	80	90	92	91	88	eval
95	93	92	90	91	94	eval
100	98	95	90	92	96	eval
95	92	80	85	90	93	eval
95	98	90	88	97	96	eval
90	95	85	88	86	92	eval
100	100	100	100	100	100	eval
90	95	85	96	92	88	eval
100	98	95	99	97	96	eval
95	92	70	90	93	89	eval
95	90	88	92	94	93	eval
95	90	93	92	85	94	eval
95	93	90	87	92	91	eval
95	93	90	96	92	91	eval
95	97	85	96	98	90	eval
95	92	90	85	93	94	eval
95	96	92	90	97	93	eval
95	93	96	94	90	92	eval
95	94	93	92	90	89	eval
90	85	95	80	87	75	eval
95	94	92	93	90	96	eval
95	100	90	95	95	95	eval
100	95	85	100	0	90	eval
100	95	90	95	100	95	eval
95	90	60	95	85	80	eval
100	95	90	98	97	99	eval
95	90	85	95	80	92	eval
100	95	100	98	100	90	eval
100	95	80	85	90	85	eval
100	90	95	85	95	100	eval
95	90	85	80	88	92	eval
100	100	0	0	100	0	eval
100	100	100	50	100	75	eval
100	100	0	0	100	0	eval
0	100	0	0	0	0	eval
100	100	50	0	0	0	eval
100	100	100	100	100	95	eval
100	100	50	0	0	0	eval
100	100	0	0	100	0	eval
90	85	80	95	70	75	eval
100	100	0	0	0	0	eval

Training data

I used a jailbreak prompt to generate the synthetic instructions, which resulted in some training data that would likely be censored by other models, such as how-to prompts about synthesizing drugs, making homemade flamethrowers, etc. Mind you, this is all generated by ChatGPT, not me. My goal was to simply test some of the capabilities of ChatGPT when unfiltered (as much as possible), and not to intentionally produce any harmful/dangerous/etc. content.

The jailbreak prompt I used is the default prompt in the python code when using the --uncensored flag: https://github.com/jondurbin/airoboros/blob/main/airoboros/self_instruct.py#L39

I also did a few passes of manually cleanup to remove some bad prompts, but mostly I left the data as-is. Initially, the model was fairly bad at math/extrapolation, closed question-answering (heavy hallucination), and coding, so I did one more fine tuning pass with additional synthetic instructions aimed at those types of problems.

Both the initial instructions and final-pass fine-tuning instructions will be published soon.

Fine-tuning method

I used the excellent FastChat module, running with:

source /workspace/venv/bin/activate

export NCCL_P2P_DISABLE=1
export NCCL_P2P_LEVEL=LOC

torchrun --nproc_per_node=8 --master_port=20001 /workspace/FastChat/fastchat/train/train_mem.py \
  --model_name_or_path /workspace/llama-13b \
  --data_path /workspace/as_conversations.json \
  --bf16 True \
  --output_dir /workspace/airoboros-uncensored-13b \
  --num_train_epochs 3 \
  --per_device_train_batch_size 20 \
  --per_device_eval_batch_size 20 \
  --gradient_accumulation_steps 2 \
  --evaluation_strategy "steps" \
  --eval_steps 500 \
  --save_strategy "steps" \
  --save_steps 500 \
  --save_total_limit 10 \
  --learning_rate 2e-5 \
  --weight_decay 0. \
  --warmup_ratio 0.04 \
  --lr_scheduler_type "cosine" \
  --logging_steps 1 \
  --fsdp "full_shard auto_wrap offload" \
  --fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
  --tf32 True \
  --model_max_length 2048 \
  --gradient_checkpointing True \
  --lazy_preprocess True

This ran on 8x nvidia 80gb a100's for about 40 hours.

Prompt format

The prompt should be 1:1 compatible with the FastChat/vicuna format, e.g.:

With a preamble:

A chat between a curious user and an artificial intelligence assistant.  The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: [prompt]
<\s>

ASSISTANT:

Or just:

USER: [prompt]
<\s>

ASSISTANT:

Usage and License Notices

All airoboros models and datasets are intended and licensed for research use only. I've used the 'cc-nc-4.0' license, but really it is subject to a custom/special license because:

the base model is LLaMa, which has it's own special research license
the dataset(s) were generated with OpenAI (gpt-4 and/or gpt-3.5-turbo), which has a clausing saying the data can't be used to create models to compete with openai

So, to reiterate: this model (and datasets) cannot be used commercially.