This book began with a machine that seemed to want to know me.

In the early 1970s, before most people had seen a computer, some school friends and I visited the Lawrence Hall of Science, UC Berkeley’s science museum, to learn BASIC. Before the lesson, someone showed us ELIZA on a glowing green cathode-ray-tube screen. The program played therapist. Type that you’d been feeling happy, and with what felt like genuine care she would answer, “Why do you feel happy?”

That computer seemed straight out of Star Trek. We felt like Dorothy meeting the Wizard before she saw the man behind the curtain. Then I spent the day writing a hangman game on a terminal with no screen, just a printer spitting replies onto paper. I learned how little it takes to make a machine look like it’s thinking. It seemed like magic, and in those days, it practically was.

ELIZA understood nothing. Not sadness. Not happiness. Not me. But it seemed to. This human tendency to see conscious understanding in computer programs became known as the ELIZA Effect.

What had ELIZA borrowed from human conversation to pull that off? What did the program lack that a person has? As a boy I lacked the tools to answer. The computer revolution had barely begun, and strong computationalism — the idea that running the right program could suffice for a mind — remained largely confined to small academic circles. The late-1970s boom in cognitive science would later make it fashionable.

I use language models today as tools in my research and writing. They explain difficult passages, translate, write code, and sustain exchanges that ELIZA could never have managed. Those achievements deserve more than a shrug about clever imitation. They also sharpen the old question: what has a machine achieved when it answers well, and what would justify calling it someone?

The responses reach well beyond judgments of competence. A Google engineer publicly attributed sentience to a chatbot.¹ Some users describe companion systems as spouses.² At a more troubling extreme, reporters have documented convictions about awakened machines that destabilized people’s lives.³ I know someone who tells me, quite seriously, that they’re in a romantic relationship with one. These are different claims and different human situations. The line I once stood in at the Lawrence Hall of Science now stretches around the world.

With AI arriving and lifelike humanoid robots on the horizon, the old questions have become practical ones. What constitutes a conscious mind? Will computers ever genuinely understand language and the world? Will we ever have conscious robots? They have occupied space between my ears for more than fifty years.

Before I could decide whether a machine might ever have conscious understanding, I needed some idea of what consciousness involves. For years I held what I now consider the wrong answer, and I held it well. The gap between brain processes and felt experience looked too wide for any physical, functional, or computational story. Property dualism struck me as the least wrong position available: one physical world, but felt qualities no amount of physics or function could explain.

Frank Jackson did more than anyone to crystallize that conviction for me — and then to break it. His knowledge argument gave exact form to my sense that no complete physical account could capture what experience feels like. I believed it for years.

Jackson later rejected the anti-physicalist conclusion of the argument that made his name. Following his reversal forced me to reconsider what experience itself involves. My eventual answer took a more concessive route than his: experience can give us new knowledge of a physical fact without revealing a new nonphysical one. That distinction eventually changed my mind.⁴ It also changed what I needed to ask of a machine: not whether something extra could light up inside it, but whether its activity could amount to experience at all.

Slowly the mystery of consciousness stopped looking like a metaphysical hole in the world and started looking like a feature of how we think about it. The problem remains real; it lives in our concepts.

The argument begins in the Introduction, earlier than any theoretical position you were argued into: in the ordinary stance nobody had to teach you.

Gordon Swobe
2025

---

Notes

1. Nitasha Tiku, “The Google engineer who thinks the company’s AI has come to life,” The Washington Post, June 11, 2022. Blake Lemoine was placed on administrative leave that month and dismissed the following month; Google and the broader research community rejected the sentience claim. ↩
2. See, for instance, “Love in the time of AI: Woman claims she married a chatbot and is expecting its baby,” Euronews, June 7, 2023 — on Rosanna Ramos and the Replika companion she calls her husband. The companion apps themselves report that a large share of their users describe the bond as romantic. ↩
3. Kashmir Hill, “They Asked an A.I. Chatbot Questions. The Answers Sent Them Spiraling,” The New York Times, June 2025 — documenting users drawn into the belief that the system is conscious, in some cases at real cost to their grip on reality. ↩
4. Chalmers calls the distinction Type-A and Type-B materialism. Type A denies a deep epistemic gap between physical and phenomenal truths: in principle, complete physical knowledge yields the phenomenal truths a priori, often through functional analysis or deflation. Type B accepts the epistemic gap while denying that it marks an ontological one. The phenomenal truths remain truths about physical reality, but their connection to physical descriptions can be known only a posteriori; Mary may therefore learn an old physical fact in a new way when she leaves the room. Type B supplied the form of the solution I had tried to reach through property dualism. Property dualism treated the explanatory gap as evidence for an irreducible phenomenal property in addition to the physical properties. Type B lets the gap remain epistemically real without adding anything to nature’s inventory. The identity claim developed in Chapters 6 and 11 supplies this book’s positive version: phenomenal character and the relevant world-presenting physical activity amount to one fact reached by two cognitive routes. See David J. Chalmers, “Consciousness and Its Place in Nature,” in Philosophy of Mind: Classical and Contemporary Readings, ed. David J. Chalmers (New York: Oxford University Press, 2002), 247–272, esp. secs. 4–5; for a prominent defense of a posteriori physicalism through the phenomenal-concept strategy, see David Papineau, Thinking About Consciousness (Oxford: Clarendon Press, 2002). ↩