akhaliq
Abstract
We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://anonymous.4open.science/r/more_agent_is_all_you_need.
Community
This ought to be the way we create training data, iterate through existing dataset, create 15+ derivatives and use the voting results for DPO. Ideally each time the model improves, use it to generate the new data, rather than using a stale model. That way we sacrifice training speed but gain 15x inference time. I wonder how many times you can rinse and repeat before getting diminishing results
I'm not sure that more agents is "all you need" when there are diminishing returns with increasing the number agents (i.e., the performance gains are asymptotic). If you effectively hit a ceiling in performance at n=30 agents, and the performance is below the current SOTA, then how is it "all you need"? I feel like this paper should be called "crowd wisdom" or "agentic tree-of-thought," since this is essentially and agent-based implementation of ToT.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning (2024)
- Divide and Conquer for Large Language Models Reasoning (2024)
- Self-Discover: Large Language Models Self-Compose Reasoning Structures (2024)
- Understanding the planning of LLM agents: A survey (2024)
- LLM Harmony: Multi-Agent Communication for Problem Solving (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
I'm not sure that more agents is "all you need" when there are diminishing returns with increasing the number agents (i.e., the performance gains are asymptotic). If you effectively hit a ceiling in performance at n=30 agents, and the performance is below the current SOTA, then how is it "all you need"? I feel like this paper should be called "crowd wisdom" or "agentic tree-of-thought," since this is essentially and agent-based implementation of ToT.
... "is all you need" has become a bit of a joke. It's no different to the way that a lot of researchers "fudge" their charts and use unnecessarily complex terminology to hide the simplicity of the underlying method. Showmanship aside, the take away I get from this research, is that we haven't trained the models well enough. When we create 15+ derivatives of the same prompt, the votes should be uniform. Then we know that we have trained it sufficiently, because it is always outputting the best answer.
More Agents, Better Results: Boosting LLMs Performance with Ensembles
Links 🔗:
👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/
By Arxflix
Models citing this paper 2
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper