Hardware requirements for using sentence-transformers/all-MiniLM-L6-v2
Hi,
Can someone please advise me upon the hardware requirements of using this model for a semantic similarity use-case. I had downloaded the model locally and am using it to generate embedding, and finally using util.pytorch_cos_sim to calculate similarity scores between 2 sentences. All was working good in my Mac Pro ( 2.4 GHz 8-Core Intel Core i9 processor and 32 GB memory); but after I moved the model to containers of 1 core CPU and 4 GB RAM, the code is taking at least 15-20 times more time to generate the cosine similarity score.
Did someone face a similar situation? Kindly advise.
Thank you in advance for the help!
N.B.: I have also shared the sample code for reference.
Hello,
I'm not sure but I think when you are using the transformer from a container, the SentenceTransformer is downloading the model each time you are running it. My advice is to log the loading time (the time to instantiate the class SentenceTransformer) in order to confirm/infirm the fact that you are spending more time in downloading than in executing the model.