简单测试，报错Can't load the configuration
报错：
OSError: Can't load the configuration of '.models/baichuan-inc_Baichuan2-13B-Chat-4bits/'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '.models/baichuan-inc_Baichuan2-13B-Chat-4bits/' is the correct path to a directory containing a config.json file
下载模型代码：
snapshot_download(repo_id="baichuan-inc/Baichuan2-13B-Chat-4bits",
local_dir="./baichuan-inc_Baichuan2-13B-Chat-4bits/",
resume_download=True,
)
测试模型代码：
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("./models/baichuan-inc_Baichuan2-13B-Chat-4bits/", use_fast=False, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(".models/baichuan-inc_Baichuan2-13B-Chat-4bits/", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
model.generation_config = GenerationConfig.from_pretrained(".models/baichuan-inc_Baichuan2-13B-Chat-4bits/")
messages = []
messages.append({"role": "user", "content": "解释一下“温故而知新”"})
response = model.chat(tokenizer, messages)
print(response)
完全按照model card操作，并且下载和测试其他模型不会报错。问题在哪？如何解决呢？
config文件可以贴一下
config文件可以贴一下
config.json没有改过
{
"_from_model_config": true,
"architectures": [
"BaichuanForCausalLM"
],
"auto_map": {
"AutoConfig": "configuration_baichuan.BaichuanConfig",
"AutoModelForCausalLM": "modeling_baichuan.BaichuanForCausalLM"
},
"tokenizer_class": "BaichuanTokenizer",
"bos_token_id": 1,
"eos_token_id": 2,
"gradient_checkpointing": false,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 13696,
"model_max_length": 4096,
"model_type": "baichuan",
"num_attention_heads": 40,
"num_hidden_layers": 40,
"pad_token_id": 0,
"quantization_config": {
"bnb_4bit_compute_dtype": "bfloat16",
"bnb_4bit_quant_type": "nf4",
"bnb_4bit_use_double_quant": true,
"llm_int8_enable_fp32_cpu_offload": false,
"llm_int8_has_fp16_weight": false,
"llm_int8_skip_modules": null,
"llm_int8_threshold": 6.0,
"load_in_4bit": true,
"load_in_8bit": false
},
"rms_norm_eps": 1e-06,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.29.2",
"use_cache": true,
"vocab_size": 125696
}
模型md5看一下？
模型md5看一下？
b3ac678cf5bd552f4f12843d542319cd pytorch_model.bin
也没问题，可能和你的环境有关，比如transformers库和Bitsandbytes库的版本。