Incompatibility with Embedding model
#16
by
NiloufarAb
- opened
I'm trying to load Qwen2-7b-Instruct using "HuggingFaceLLM" class. The embedding model is also loaded on multiple gpus using the below code:
from transformers import AutoModel
model = AutoModel.from_pretrained('Alibaba-NLP/gte-Qwen2-7B-instruct', device_map='auto')
model.eval()
However I get the error: raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'Qwen2Model' object has no attribute 'get_agg_embedding_from_queries', which shows the incompatibility between Qwen2 and Embedding model. Could someone help me with this?