Simple query does not work
Hello! Why would a simple query like:
"Who will run for president in 2024?"
not work?
I copied the prompt and latest transformers version.
Hi @mber ,
Thanks for pointing this out. During training, we instructed the model to output an empty list if the query is not relevant to the provided tools, with relevance determined by the query and the API descriptions. In this case, the model might have been too conservative and decided that this query couldn't be directly answered by the provided APIs.
A quick fix is to revise format_instruction by removing the following content: If no function call is needed, please make tool_calls an empty list '[]'. After this adjustment, the 7b model should work:
We noticed that we hadn't fully considered this type of situation, particularly the various use scenarios of search_api, in our training pipeline. As a result, the model (especially the 1b model) might struggle to determine whether to use the search tool or refuse to answer. We will optimize our model in the next iteration.
For the current version of the model, we recommend providing more fine-grained APIs with detailed descriptions for the model to choose from. We also welcome and appreciate more user feedback, such as typical usage scenarios, queries, and APIs you'd like to use for real applications. This will help us further improve the model's applicability. Thanks!
Thank you for the quick response, it worked!
I did notice however that this prompt fix did not work for Salesforce/xLAM-1b-fc-r, it was still unable to produce a function call.
Is there a similar modification I can perform to the prompt, to make the 1B model to produce a function call instead of an empty list?