Mind, Matter, and Meaning

Claude Shannon

Claude Shannon (1916–2001) developed mathematical tools for switching circuits and communication. His information theory quantifies uncertainty, coding, and transmission reliability. It deliberately separates the engineering question of reproducing a message from questions about what that message means. This makes the theory widely applicable, while limiting what its quantities alone establish about representation or understanding.

Major Works and Dates

“A Symbolic Analysis of Relay and Switching Circuits” originated as Shannon’s MIT master’s thesis in 1937 and appeared as a journal article in 1938. “A Mathematical Theory of Communication” appeared in two parts in 1948. The 1949 book The Mathematical Theory of Communication combines Shannon’s paper with an introductory essay by Warren Weaver. The article’s “A” and the book’s “The” belong to different publication titles.

Logic and Physical Circuits

Shannon showed how Boolean algebra could be used to analyze and synthesize relay and switching circuits. Connections in series and parallel can implement logical conditions, allowing circuit behavior to be specified and simplified symbolically.

This is a bridge between an abstract logical description and an engineered physical mechanism. It does not depend on a particular interpretation of the circuit’s signals as English sentences. The contribution concerns implementation of logical operations, not a conclusion that every system implementing those operations thereby understands their semantic use.

Entropy and Information

For a discrete source with outcome probabilities pᵢ, Shannon entropy is:

H(X) = −Σᵢ pᵢ log₂ pᵢ.

Using base-two logarithms expresses entropy in bits. The quantity measures average uncertainty about which outcome will occur. The information associated with a particular outcome—the outcome’s surprisal—is −log₂ pᵢ. Entropy averages that quantity across possible outcomes; the two should not be conflated.

A predictable symbol carries little new information in this sense. A less probable symbol is more surprising. Neither fact makes the symbol more meaningful, more important, or more truthful. A source of random symbols can have high entropy without expressing an intelligible message.

Shannon credits John W. Tukey for the word “bit.” Information theory made the unit central; it did not require claiming that Shannon invented the word or every earlier idea about communication.

Channel Capacity and Coding

A communication system involves a source, transmitter, channel, receiver, and destination, with noise potentially altering the signal. The statistical properties of the source determine opportunities for compression. The channel’s properties limit the rate of reliable transmission.

The noisy-channel coding result establishes that, under specified assumptions and with sufficiently long suitable codes, transmission below capacity can make error probability arbitrarily small. It does not promise perfect communication at any finite block length or specify one universally practical code. The distinction between a mathematical existence result and an efficient engineering implementation is important.

Mutual information concerns the statistical dependence between source and received variables: how much knowing one reduces uncertainty about the other. Channel capacity optimizes the achievable informational rate over appropriate input distributions. These quantities depend on a specified probabilistic model, not on semantic interpretation alone.

Why Meaning Is Set Aside

Shannon explicitly says that semantic aspects are irrelevant to the engineering problem he defines. The system must reproduce a selection from possible messages, whether the selection expresses a true statement, a false statement, or no statement at all.

This is a scope restriction, not a denial that messages have meanings. Nor does it imply that statistics can never contribute to explanations of representation. Later theories attempt to connect informational relations with learning, biological function, or linguistic practice. Those additional proposals must explain reference and misrepresentation; the Shannon equations do not already contain them.

Applications and Limits

The framework can be applied to electronic communication, neural signals, and other physical processes when their states and distributions can be modeled appropriately. Its generality does not mean that all applications concern the same phenomenon or that information is a substance moving through every system.

For neural data, an informational dependence may identify a useful relation between activity and stimuli without establishing what the organism consciously experiences. For language, reducing prediction error may measure a statistical achievement without directly measuring truth or understanding. Conversely, a system can transmit a meaningful message using an informationally ordinary channel. Statistical information and semantic content are distinct questions, not mutually exclusive kinds of system.

Further Reading

  • Shannon, Claude E. “A Symbolic Analysis of Relay and Switching Circuits.” Transactions of the American Institute of Electrical Engineers 57, no. 12 (1938): 713–723. Based on the 1937 MIT master’s thesis.
  • Shannon, Claude E. “A Mathematical Theory of Communication.” Bell System Technical Journal 27 (1948): 379–423 and 623–656.
  • Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. University of Illinois Press, 1949.