evaluate-metric/bertscore · GPU issues with bert score

I am running into a GPU memory issue while using bertscore. If I call BertScore n number of times, the GPU memory keeps going up progressively till I get cuda out of memory. So each call for bertscore is predictions/references from one experiment out of n. But If I concatenate all the pairs of all experiments and then call bertscore over all examples, the memory used is constant. This leads me to believe every time bertscore is called a new instance of the model is initialized but not removed from GPU memory. Is that assumption correct? Even call torch.cuda.empty_cache after each call doesn't solve the issue. Is there any alternated way of doing this other than appending the outputs of all experiments in a single list ?