Could you share the training config?

by Lohse - opened Jun 18

Jun 18

I'm conducting a finetuning on alpaca-gpt4, could you share the training config (training script), like
batchsize
lr
lr_scheduler_type
seq_length
lora_rank
, and how to preprocess the input?
:)

Weyaxi

Owner Jun 23

Hi @Lohse ,

I lost the training script but here is the parameters:

lr: 0.0004
lr_scheduler_type: cosine
seq_length: 8192
lora_rank: 8

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment