The influential criticism that language models stitch together patterns from their training data without any understanding of meaning, and that fluency misleads people into attributing comprehension. The paper it comes from also raised environmental cost, unfathomable training data and the risk of amplifying bias, and those arguments have aged well.
Technical view
From Bender and colleagues, arguing that form learned without grounding in communicative intent cannot constitute meaning. Counterarguments point to emergent capabilities not present in training data, evidence of structured world representations from probing, and success on tasks requiring composition. The disagreement is partly empirical and partly definitional, turning on what would count as understanding, and it remains unresolved rather than settled in either direction.
Related
Part of AI Terminology Index, section: Alignment and Safety Concepts.