A system trained on linguistic form alone can master the form completely and grasp nothing — because meaning lives in the relation between a form and the world a speaker uses it to reach, and form carries no such relation. More modalities enlarge the cable; they do not give the listener a stake in the world.

What this chapter does. The last three chapters fought the computationalist on his best ground — a perfect symbol system, a perfectly implemented program, a perfect digital brain — and beat him on each. But the systems actually on our desks are not perfect brain-models, and pretending they are flatters them. They are masters of form: large language models trained to predict the next token across a civilization’s worth of text, with no brain modeled and, in the text-only case, no world touched. This chapter turns from the limit case to the live one. It states the cleanest version of the form/meaning gap — Bender and Koller’s hyper-intelligent octopus, who learns a language by eavesdropping on a wire and is undone the moment a bear appears — and then meets head-on the rejoinder that matters now, the one that says the octopus is obsolete: today’s models are not text-only; they see images, take actions, get tuned by human feedback, so surely they now touch the world the octopus could not reach. The chapter grants that rejoinder its strongest recent statement and resists it, distinguishing more form from world-contact, and arguing that what grounds reference is not the number of modalities but whether the system has a stake in getting the world right. Finally it cleans up a word the AI industry borrowed and broke — hallucination — and shows why a system that never referred to anything cannot, strictly, misrepresent. The thread running through Part Three and Part Four pulls taut here: information without meaning, syntax without semantics, and now mastery of form without reference are one distinction told three times.

The chapter argues in five steps:

1. From the limit case to the live one (§I). Why the real systems are form-masters, not brain-models — and what that changes.
2. The eavesdropping octopus (§II). Bender and Koller’s argument, and Harnad’s symbol grounding problem behind it.
3. The multimodal rejoinder (§III). Do vision, action, and human feedback supply world-contact, or only more form? — stated via Mollo and Millière and resisted.
4. Why “hallucination” misnames it (§IV). A system with no world-grounded content has no content to misrepresent.
5. What the negative case clears the way for (§V). The hand-off to Chapter 24’s constructive question.

---

§I. From the Limit Case to the Live One

Chapter 22 granted the computationalist a perfect digital brain and showed that running it would not be being it. That was the right fight, because it met the position at full strength. But it leaves a reader looking at an actual chat window with a slightly mismatched argument in hand, because the system in the window is not a model of a brain. Nobody simulated a cortex to build it. It was trained to do something far narrower and, in its way, far stranger: to predict the next stretch of text, given the text so far, having been fed an amount of human writing no human could read in a thousand lifetimes. It is a master of form — of the statistical shape of language — and the question it poses is sharper and more modern than the one the last three chapters answered. Set aside brains and programs and implementations. Can a system that has consumed nothing but the form of language, and has never stood in front of any of the things the language is about, come to mean anything by the fluent sentences it produces?

The obvious answer — of course not, it’s just predicting text — is right about the verdict and lazy about the reason, and the lazy version collapses the moment someone points out how much these systems can do. They explain, summarize, translate, write usable code, hold a thread across a long conversation, adjust their register to yours. Dismissing that as “autocomplete” is not analysis; it is a refusal to look. The verdict is right but unearned. What takes work is why form alone leaves meaning untouched — and then the one rejoinder that genuinely complicates the picture: that the form-only premise is already out of date.

§II. The Eavesdropping Octopus

The cleanest statement of the gap comes not from a philosopher but from two computational linguists, which is part of what makes it land. Emily Bender and Alexander Koller built their argument to discipline their own field — to puncture the slide, common in the research literature, from the model produces fluent text to the model understands — and they made it with a thought experiment that has the rare virtue of being funny and exact at once.¹

Two people, A and B, live on separate islands and talk to each other through an undersea telegraph cable. A hyper-intelligent octopus, O, taps the cable and listens. It never sees the islands, never meets A or B, never observes a single thing the two of them talk about — it has access only to the form of the signals, the patterns of symbols flowing back and forth. And O proves an extraordinary student of those patterns. It learns what tends to follow what so well that one day it cuts B out of the loop, impersonates him, and keeps up B’s end of the conversation. A notices nothing. By every behavioral measure available across the wire, O has mastered the language.

Then a bear appears on A’s island. A, panicking, sends a frantic description of the sticks and rocks at hand and begs B for instructions to build a weapon. And O, who has flawlessly predicted thousands of B’s sentences, has nothing. It can emit text shaped like an answer — grab a stick, pick up a rock — because that is the shape such answers take in the corpus. It cannot emit an answer that helps, because it has never connected the word “stick” to a stick, “bear” to a bear, or any of A’s words to the world A is now desperate to act in. O learned the form of the conversation and missed the meaning entirely, and the gap stayed perfectly invisible right up to the moment meaning was the only thing that would do.

That is the form/meaning gap with no word said about consciousness, and the restraint is the point. Bender and Koller do not claim O lacks an inner life; they claim O lacks contact with the world the words are about. Meaning lives in the relation between a form and what a speaker uses it to do — a relation grounded outside the text, in a shared world and in communicative intent.² A system trained on form alone never touches that relation, however much form it consumes. The octopus is the Chinese Room of Chapter 20 with the diagnosis made explicit, recast for a reader who long ago stopped being moved by a man shuffling symbols. The contemporary language model is O with a far larger cable.

Behind the octopus sits an older and more general problem, named by the cognitive scientist Stevan Harnad: the symbol grounding problem.³ How do the symbols in a symbol system get their meaning, if their definitions are only more symbols? Look up a word in a monolingual dictionary and you get more words; look those up and you get more words still. A purely symbolic system is a closed loop of tokens defined by other tokens — a merry-go-round that spins forever and never touches the ground. Unless some symbols connect to what they are about by something other than further symbols, the whole system stays ungrounded: perfectly manipulable, entirely meaningless. A language model is that merry-go-round spun at industrial speed over a civilization’s writing. Whatever its words mean, the meaning is borrowed from the humans who wrote the corpus — people who were standing in front of the world when they wrote — and is not the model’s own. Grounding, Harnad was careful to say, is necessary for meaning without being obviously sufficient for it; but a system with no grounding at all does not even reach the starting line. Whether grounding is truly necessary — or whether the right web of internal conceptual roles is meaning enough, as the boldest recent defenders of machine meaning now argue — is the live dispute the rest of this chapter takes on.⁴

§III. The Multimodal Rejoinder

Here the careful reader raises a hand, and the objection is the live one — the reason this chapter cannot simply reprint the octopus and stop. Your premise is already obsolete, the objection runs. The octopus listens only to a cable; it is trapped in pure form. But the systems we actually use now are not text-only. They take images, so they “see.” They call tools and take actions in software environments, so they “act.” They are tuned by reinforcement learning from human feedback, so their behavior is shaped by people responding to what they do. Pixels, sensor streams, actions, reward — surely these break the confinement to form. Surely the modern model has climbed off the merry-go-round and touched the ground.

This deserves its strongest statement, and it has recently received one. Dimitri Coelho Mollo and Raphaël Millière — two philosophers who have done as much as anyone to drag this debate out of slogans and into careful distinctions — have made what is probably the best contemporary case that large language models can escape the grounding problem.⁵ Their move is first to clean up the question. “Grounding,” they point out, names at least five different things in the literature, run together to everyone’s confusion; the only one that bears on whether a system’s states are genuinely about the world is what they call referential grounding — a representation’s standing in the right relation to its worldly referent, independent of any meaning a human reads into it. Sensorimotor transduction, they argue, is just one possible route to referential grounding and not a necessary one. And they contend that the training of a modern model — in particular the loop in which its outputs are selected and reshaped by human feedback tied to real-world tasks — may already supply referential grounding, so that the model’s internal states can be about extra-linguistic reality after all. The octopus, on this view, was a story about a system frozen in a regime that the current systems have left behind.

I want to give this its due, because the disambiguation alone is a genuine contribution and the conclusion is not foolish. And one half of it the book grants outright: Mollo and Millière are right that sensorimotor transduction is not necessary for reference, and any reply that insists a meaner must have a body has missed them — the objection has to be met on referential grounding itself, not on whether the system has eyes. So meet it there. The resistance turns on a distinction the rejoinder still slides past: the difference between more form and a stake in the world. Add a camera to a language model and you have added another modality of symbol. Pixels are not the world; they are a numerical array, correlated with a scene by the long causal labor of the photographer who framed it and the humans who labeled it — their engagement with the world, harvested into the training set, not the model’s. The model relates to the image as it relates to the text: as form to be predicted and transformed, exquisitely. Adding vision to a system trained on form gives it a second cable, not a body. Reinforcement learning from human feedback is the same story one level up. The reward signal is real, but it is a compression of the raters’ meanings — their sense of which answers are good, true, helpful — projected onto the model’s outputs through a gradient. The model optimizes against that signal without ever being the thing whose getting-it-wrong the signal was built to catch. The humans had skin in the game. The model inherits their scar tissue, not their stakes.

And stakes are exactly what Part Three said reference requires, and exactly what the multimodal rejoinder leaves out. Grant Mollo and Millière their referential grounding in the thin sense they define — a reliable, feedback-shaped correlation between a model’s states and worldly conditions. It is still missing the thing teleosemantics identified as the source of genuine content: a history in which the system itself was selected, or learned under real consequence, for getting the mapping right rather than wrong.⁶ The frog’s bug-detector means bug because frogs that misread the mapping went hungry and left fewer descendants; the word in a child’s mouth comes to mean cat through a long apprenticeship in which the world pushed back. The proper functions that fix content run through natural selection and individual learning — through a creature for whom error costs something — and it is far from clear that gradient descent over a corpus, however much human feedback shapes it, is a history of that kind rather than a sophisticated imitation of its products.⁷ Whether a word in a model’s output refers at all is precisely the question under dispute, and the honest reading of the current literature is that it remains open and contested, not settled in the model’s favor.⁸ The book’s verdict is not that grounding is impossible for an artificial system — Chapter 24 holds that door open — but that the thing that moves a system from form to world is not a count of modalities. It is whether the system has come to mean by having a stake in the world, and a camera, an action API, and a reward signal supply more form, more form, and a borrowed standard, none of which is a stake. The octopus with a video feed and a joystick is a better-equipped octopus. It is still on the far side of the cable.

§IV. Why “Hallucination” Misnames It

One more piece of housekeeping, because a single borrowed word does more quiet damage to clear thinking about these systems than any argument. When a model produces a footnote citing a paper that does not exist — the author never wrote it, the journal volume runs ten issues short, the page numbers point into empty air — everyone now says the model hallucinated. The word is used as if the engineers, scanning the dictionary for a punchy term, had simply found the right one.

They found a word that already had a job, and the job mattered. In philosophy, hallucination names something specific: a subject undergoes an experience that seems, from the inside, to present a worldly object — a pink rat on the counter, a friend at the foot of the bed — when no such object is there. The experience occurs; the object does not; and the whole philosophical delicacy lives in the seeming. Tim Crane, working from an intentionalist account of the kind this book defended in Part Two, treats hallucination as a state whose content fails to match how things stand.⁹ Mike Martin, defending disjunctivism, denies even that much shared structure between perceiving and hallucinating.¹⁰ Take whichever side you like; both require a subject who seems to see something. A hallucination, on any account worth having, happens to someone.

Now look at what the engineer is naming. A system trained on text emits a string that mentions a paper it never encountered. There is no seeming. Nothing in the system surveys a citation field and concludes, falsely, that a paper sits there. The system has no point of view from which the false output presents as worldly, because it has no world — it has a probability distribution over tokens, conditioned on a prompt, and a sampling step that took one path through it. The output drifts from the truth because nothing in the training rewarded matching the truth at that fine a grain. The phenomenon is real and deserves a name. Hallucination simply names the wrong shape.

The deeper point is the one Part Three set up. Misrepresentation is a structured achievement, not a default. To misrepresent, a system must first be tied to the world by relations strong enough to fix what counts as success and what counts as failure — and then, in this instance, fail. Teleosemantics locates that standard in a system’s proper function: a state misrepresents when it occurs outside the conditions it was selected or learned to track.¹¹ A text-mastering system has no such grounding to fall short of. Its outputs ride on regularities harvested from a corpus, and nothing in its loop checks them against any state of affairs in the world; the very notion of the model getting it wrong has to be imported from outside, by us, the readers who notice the missing journal issue. The model does not notice, because there is nothing it would be noticing. We get it wrong on its behalf. What the model actually does is closer to confabulation — fluent narrative produced without access to the facts it purports to report — or, more austerely, output drift from a standard the producing system cannot detect. Neither phrase has the marketing appeal of hallucination. Both describe the situation, and the borrowed word does not. It quietly reinstalls the inner theater this whole book has worked to dismantle: it pictures a little subject behind the screen, looking out at a world and occasionally deceived — a mind on the mend, needing only the right medication to stop seeing things. Strip the word away and you can finally ask the question that matters: what would a system have to have before it could be capable of getting anything wrong?

§V. What the Negative Case Clears the Way For

Stand back, and one distinction has now been told three times across the back half of this book. Part Three opened it: information is cheap and lies everywhere — the covariance smoke shares with fire, a tree ring with a year — while meaning is the dear, world-involving achievement no amount of covariance buys (Chapter 14). Chapter 20 told it again as syntax and semantics: symbols arranged by their shapes never add up to aboutness. And this chapter has told it a third time as form and world: a system can master the entire statistical shape of a language, in as many modalities as you care to wire in, and still never refer, because reference is a stake in the world and form is not. The octopus, the man in the Chinese Room, the wall that allegedly runs WordStar, the hurricane that stays in the data center — four faces of one fact. Structure floats free of the world until something outside the structure ties it down, and in every artificial system we currently have, the only thing doing the tying is us.

That completes the negative case. Three chapters running, Part Four has said what machines as actually built cannot do, and — more usefully — why: not because they are made of the wrong stuff, and not because minds are magic, but because meaning, understanding, and consciousness are achievements of a system in real, consequential traffic with a real world, and symbol-mastery alone, at any scale and in any modality, is not that. But a negative case is only half a position, and the cheap version of this argument stops here, mistaking current systems don’t understand for machines can’t. That conflation has discredited better arguments than it deserves to, and this book will not make it. The question deferred since the robot reply in Chapter 20 is now the only one left standing, and it is a constructive one — not do today’s systems understand? (they don’t) but what would any system actually need in order to? That is the last chapter’s work.

Chapter Summary

This chapter turned to the live systems on our desks — masters of linguistic form, not models of brains — and argued that form alone never reaches meaning, because meaning consists in the relation between a form and the world a speaker uses it to reach, and form carries no such relation. Bender and Koller’s eavesdropping octopus dramatizes the gap, and the multimodal rejoinder fails: more modalities add more form, not a stake. The completed negative case hands forward to Chapter 24’s constructive question — what any system would require in order to understand.

---

Notes

1. Emily M. Bender and Alexander Koller, “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020), 5185–5198; the octopus appears at 5187–5188. Bender and Koller frame the argument around a distinction between form (the observable marks or sounds of language) and meaning (the relation between forms and communicative intent), and argue that “a system exposed only to form in its training cannot in principle learn meaning” (5185). Their target is the same overclaiming this Part resists — the inference from fluency to understanding — reached from computational linguistics rather than philosophy of mind. For a critical reply defending the sufficiency of form-derived structure, see Julian Michael, “To Dissect an Octopus: Making Sense of the Form/Meaning Debate,” online essay, 2020, julianmichael.org. ↩
2. The reliance on a shared world and communicative intent aligns the octopus with the externalist and Gricean traditions of Chapter 17 rather than with Searle’s biological naturalism, and the two diagnoses, though they reach one verdict, differ in emphasis: Searle locates the Chinese Room’s failure in the absence of intrinsic intentionality, while Bender and Koller locate the octopus’s failure in the absence of any grounding relation between form and a shared world. This book sides with the world-involvement diagnosis — what the octopus and the language model lack is not certified inner experience but causal-communicative purchase on the world the words are about. ↩
3. Stevan Harnad, “The Symbol Grounding Problem,” Physica D 42 (1990): 335–346. Harnad’s diagnosis: symbol meaning cannot be intrinsic to a symbol system, because within the system there is nothing but more symbols; his proposed remedy is sensorimotor grounding, an internal mechanism connecting at least some elementary symbols to the categories of things the system can pick out in perceptual interaction with the world, so that higher symbols built from them inherit a connection to the world. The problem is the precise, contemporary form of the worry the Chinese Room dramatizes. ↩
4. Harnad is careful to distinguish symbol grounding from a theory of meaning: grounding connects a symbol to a category in the world, but whether a grounded system thereby means anything, or has any inner life, is a further question grounding alone does not settle. This book keeps both halves — grounding is necessary for meaning (an ungrounded symbol system means nothing on its own) but not obviously sufficient — which is why the constructive conditions of Chapter 24 are stated as necessary requirements rather than as a recipe guaranteed to yield understanding. The most direct contemporary challenge to the necessity half is Steven T. Piantadosi and Felix Hill, “Meaning without Reference in Large Language Models” (2022), arXiv:2208.02957, who argue that conceptual-role relations among a model’s internal representations capture genuine aspects of meaning with no reference or sensory grounding at all; Anders Søgaard, “Understanding Models Understanding Language,” Synthese 200, no. 6 (2022): article 443, presses a kindred inferential-semantics line. The book resists both on the same ground it resists the octopus: relations purely among internal representations are more merry-go-round, however richly structured — meaning needs a reaching-out to the world that intra-symbolic role, by itself, never performs. The position has a distinguished pedigree the LLM-facing versions inherit — conceptual- or inferential-role semantics, running from Wilfrid Sellars through Robert Brandom’s inferentialism (Making It Explicit, Cambridge, MA: Harvard University Press, 1994) and Ned Block’s functional-role account, on which a term’s meaning is fixed by its inferential liaisons to other terms. Two considerations blunt its use against the present argument. First, Block’s is a two-factor theory (“Advertisement for a Semantics for Psychology,” Midwest Studies in Philosophy 10 [1986]: 615–678): it pairs internal conceptual role with an external, world-involving factor precisely because role alone yields only narrow content that does not reach reference — so the tradition’s most careful statement already concedes the half this book insists on. Second, even the friendliest recent treatment grants the limit: Mollo and Millière allow that inferential-role grounding works, at best, for the logical vocabulary whose whole meaning lives in inference, and not for the world-directed terms whose reference is the very thing in dispute. Inferential role can make a system’s tokens cohere; it cannot, by itself, make them about anything, because coherence among symbols is the merry-go-round, not the way off it. This is the qualified reference theory of Chapter 17, not a denial that internal structure matters: reference is necessary, with use and function doing the rest. For the negative verdict reached by a different route, see Jobst Landgrebe and Barry Smith, “Why Machines Do Not Understand: A Response to Søgaard” (2023). ↩
5. Dimitri Coelho Mollo and Raphaël Millière, “The Vector Grounding Problem,” arXiv:2304.01481 (2023; forthcoming in Philosophy and the Mind Sciences). Mollo and Millière distinguish five notions of grounding — sensorimotor, epistemic, communicative, relational, and referential — and argue that only referential grounding is necessary for a representation to be about extra-linguistic reality, that sensorimotor transduction is merely one route to it, and that the feedback-driven training of contemporary models (especially RLHF) may already secure it. It is the strongest recent case that LLMs escape the symbol grounding problem, and the disambiguation of grounding-types is a real contribution even where the conclusion is resisted; this chapter grants the framework and resists the conclusion on the ground that referential grounding in their thin sense lacks the consequential proper-function history Part Three’s teleosemantics requires. ↩
6. The teleosemantic account of content is developed in Chapters 16–17; the foundational statements are Ruth Garrett Millikan, Language, Thought, and Other Biological Categories (Cambridge, MA: MIT Press, 1984), chs. 1–2, on proper functions, and Fred Dretske, Explaining Behavior: Reasons in a World of Causes (Cambridge, MA: MIT Press, 1988), for the parallel route through learning history. On these accounts a state’s content is fixed by the conditions the system was selected or learned to track — which builds the possibility of misrepresentation into representation from the start, and ties content to a history in which getting the mapping wrong carried a cost. ↩
7. Whether artificial systems trained on human corpora could inherit learning-historical proper functions in Millikan’s sense is a genuine frontier rather than a closed case; see Jumbly Grindrod, “Large Language Models and Linguistic Intentionality,” Synthese 204 (2024): article 71, https://doi.org/10.1007/s11229-024-04723-8, which presses the question carefully on both sides. Millikan’s own commitments suggest she would resist the extension, since the histories she has in mind run through natural selection and individual learning under consequence rather than through gradient descent over a static corpus; the present chapter takes that resistance to be well-founded without claiming the matter is settled. ↩
8. Whether the words in a language model’s outputs refer, in the philosophically loaded sense, is itself a live and unresolved debate; for a careful treatment that frames the question precisely without overclaiming in either direction, see Matthew Mandelkern and Tal Linzen, “Do Language Models’ Words Refer?,” Computational Linguistics (2024); arXiv:2308.05576. The book’s position is that the dispute is real and open, and that the burden sits with whoever would treat reference by a form-trained system as established rather than contested. ↩
9. Tim Crane, “Is There a Perceptual Relation?,” in Perceptual Experience, ed. Tamar Szabó Gendler and John Hawthorne (Oxford: Oxford University Press, 2006), 126–146; and “The Problem of Perception,” Stanford Encyclopedia of Philosophy (rev. 2021, with Craig French). Crane’s intentionalist treatment makes hallucination a representational state whose content fails to match the world — the phenomenal character arising from how it represents, not from any inner object the subject inspects. This is the account the main text of Chapter 7 develops and Chapter 8 applies to phantom-limb pain; it is recalled here to anchor the philosophical meaning of hallucination before the engineering metaphor co-opts the term. ↩
10. M. G. F. Martin, “The Transparency of Experience,” Mind and Language 17 (2002): 376–425, and “On Being Alienated,” in Perceptual Experience, ed. Gendler and Hawthorne (Oxford: Oxford University Press, 2006), 354–410. Martin’s disjunctivism denies the common-factor assumption — that veridical perception and an indistinguishable hallucination share a substantive mental state — and identifies hallucination only negatively, as a state subjectively indistinguishable from a veridical perception of a given kind. The book sits closer to Crane’s intentionalism (see Chapter 4), but both accounts converge on the point that matters here: the philosophical use of hallucination requires a subject for whom the world seems a certain way. ↩
11. Ruth Garrett Millikan, Language, Thought, and Other Biological Categories, chs. 1–2; and Karen Neander, A Mark of the Mental: In Defense of Informational Teleosemantics (Cambridge, MA: MIT Press, 2017). Neander ties representational content to the conditions a system is functionally adapted to detect rather than to those it merely correlates with, making the misrepresentation case structurally clean: a state misrepresents when it occurs outside the conditions its function was selected to track (the frog’s bug-detector firing at a passing pellet is the canonical illustration of Chapter 16). Neither account allows a representational achievement — and so the very capacity to misrepresent — to be inherited without the grounding history that confers it, which is why a text-trained system’s false output is not misrepresentation but drift. ↩