xcomposer2d5 3rd party sft support
Hi, we are a team from ModelScope community, we have support the sft of internlm/internlm-xcomposer2d5-7b, please check this doc if interested:
https://github.com/tastelikefeet/swift/blob/main/docs/source_en/Multi-Modal/internlm-xcomposer2-best-practice.md
Or with this command:
# install
pip install git+https://github.com/modelscope/swift.git
# run
CUDA_VISIBLE_DEVICES=0 swift sft \
--model_type internlm-xcomposer2_5-7b-chat \
--dataset okvqa
Any idea why I get the following when I try to train it with Lora?
TypeError: Linear4bit.forward() takes 2 positional arguments but 4 were given
@tastelikefeet Thanks for the support! I have add a link to your support on github https://github.com/InternLM/InternLM-XComposer/tree/main?tab=readme-ov-file#finetune
I also get the TypeError: Linear4bit.forward() takes 2 positional arguments but 4 were given error when using --load-in-4bit , I get this error for all the xcomposer models now when using bitsandbytes (--load-in-4bit), but I didn't always.