this model works bad
have you ever checked it? its not even working for your own example..
@galbendavids Thanks for your interest on tapex! Actually the model has been 2 years old since its first release - and yes I can confirm that the old transformers version can yield the expected output for this model on the "demonstration example". Do you mind sharing the generated results?
just ask simple question like "how many movies they are overall" or "how many movies are particpated by X and by Y" and youll get poor result.
Hello @galbendavids , I understand there may be some inaccuracies in the results when the model tries to do counting. However, when you mention that "it does not work for the provided example", could you please elaborate on the specific issues you've encountered?
Just tried this:
tokenizer = TapexTokenizer.from_pretrained("microsoft/tapex-large-finetuned-wtq")
model = BartForConditionalGeneration.from_pretrained("microsoft/tapex-large-finetuned-wtq")
data = {
"year": [1896, 1900, 1904, 2004, 2008, 2012],
"city": ["athens", "paris", "st. louis", "athens", "beijing", "london"]
}
table = pd.DataFrame.from_dict(data)
query = "How many cities are there?"
encoding = tokenizer(table=table, query=query, return_tensors="pt")
outputs = model.generate(**encoding)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
For the question "What is the average year of the cities?", it incorrectly answered 2004.
Then for the question "How many cities are there?", it incorrectly answered 5.
File "/home/miniconda3/envs/LLM/lib/python3.10/site-packages/torch/nn/functional.py", line 2237, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
unknown error