The Generative AI Paradox: "What It Can Create, It May Not Understand"
akhaliq
Abstract
The recent wave of generative AI has sparked unprecedented global attention, with both excitement and concern over potentially superhuman levels of artificial intelligence: models now take only seconds to produce outputs that would challenge or exceed the capabilities even of expert humans. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make? In this work, we posit that this tension reflects a divergence in the configuration of intelligence in today's generative models relative to intelligence in humans. Specifically, we propose and test the Generative AI Paradox hypothesis: generative models, having been trained directly to reproduce expert-like outputs, acquire generative capabilities that are not contingent upon -- and can therefore exceed -- their ability to understand those same types of outputs. This contrasts with humans, for whom basic understanding almost always precedes the ability to generate expert-level outputs. We test this hypothesis through controlled experiments analyzing generation vs. understanding in generative models, across both language and image modalities. Our results show that although models can outperform humans in generation, they consistently fall short of human capabilities in measures of understanding, as well as weaker correlation between generation and understanding performance, and more brittleness to adversarial inputs. Our findings support the hypothesis that models' generative capability may not be contingent upon understanding capability, and call for caution in interpreting artificial intelligence by analogy to human intelligence.
Community
No computer will ever "understand". It has been mathematically proven to be impossible (Godels incompleteness thereom + derivative works).
Understanding is rooted in "belief" which always escapes mathematics. I. E it is part of consciousness, and not part of computation.
Anyone who thinks a computer has the ability to understand, does not understand what it means to understand.
Such fools will inevitably call for algorithms to be given human rights.
Pardon my pedantic rant π the hypothesis this paper discusses is very insightful
@MichaelBarryUK
"It has been mathematically proven to be impossible"
"it is part of consciousness, and not part of computation"
Consciousness is unfalsifiable and has not yet been successfully defined in any testable fashion. You can't "mathematically prove" anything about it, because it's not a mathematical construct. See: the philosophical zombie thought experiment. Furthermore, due to this ambiguity, it's also rather silly to categorically assume that consciousness and computation are fundamentally separate things. It could just as well be argued that consciousness is computation.
Making absolute statements about the possibility for machine consciousness is silly unless you have a testable definition of consciousness in the first place. It could be impossible for machines to understand, or it could be totally possible. Within the limits of known cognitive science, it is impossible to know for sure.
If you don't believe me, here is a clip of Sir Roger Penrose (arguably the worlds greatest mathematician) explaining why consciousness is outside of computation. It really has been mathematically proven. And it's mind blowing.
https://m.youtube.com/watch?v=w11mI67R95I
He talks in great detail about this in many other videos, but that's the meat and bones right there.
Fun fact, the guy who did the proof was a philosopher who hated the idea of mathematics explaining the universe. The mathematicians of the time wouldn't listen to him. So he encoded his philosophical understanding as a mathematical proof. It was a trojan horse, using math to prove that math can only ever be a subset of the universe. The leading mathematicians of the day conceded defeat, and their lifes work was reduced to ruin (principica mathematica) and they never did math ever again. "Belief" is more fundamental than mathematics. Belief is the bedrock of mathematics. Without belief there can be no mathematics. That includes all forms of language, logic and intelligence too. It is all symbolic.
By belief I mean belief/understanding/truth etc. It's the old concept of the map is not the terrain. Truth/understanding/etc is ineffable. Truly unspeakable. And it belongs to and/or/is consciousness.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- How FaR Are Large Language Models From Agents with Theory-of-Mind? (2023)
- Critique Ability of Large Language Models (2023)
- From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning (2023)
- TADIS: Steering Models for Deep-Thinking about Demonstration Examples (2023)
- Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement (2023)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper