A Bottom-Up Connectionist Computation

From Mentoring the Machines

Mentoring the Machines: Surviving the Deep Impact of an Artificially Intelligent Tomorrow is a new book project I've partnered on with esteemed cognitive scientist John Vervaeke. Visit www.mentoringthemachines.com to learn more or at Amazon.com

It’s widely recognized today that McCarthy’s Dartmouth summer research proposal was the first to formally present the term “Artificial Intelligence.” So, technically, he’s the visionary authorial source behind our contemporary pursuit and fascination with AI.

McCarthy coined the phrase.

Beyond its relative brevity and aggressive budget ($13,500, equivalent to $150,000 today), what’s striking about the proposal, which can be easily found on the internet, is the explicit recognition of two approaches to creating thinking machines (Kline 2011, 10; McCarthy et al. 1955, 2, 4). That is, McCarthy’s original hypothesis to John von Neumann back at Princeton in 1949 is deeply embedded in the Dartmouth summer research project proposal.

But the relationship between the two approaches is not addressed. It’s ignored. It’s as if the two ways McCarthy insists thinking “happens” are different, unrelated, separate kinds of thinking. That they do not work together in a relationship to form a whole greater than the sum of their parts. Instead, the assumption is that thinking can happen “this way” or “that way,” one or the other, both are valid, yet unrelated.

There’s the “brain” way and the “mind” way. Whatever floats your boat.

That characteristic, science’s assumption that the “connectionist” brain and the “symbolist” mind are distinct and separate entities had, and continues to have, a profound four-hundred-year-old legacy, stemming all the way back to René Descartes’s Meditations on First Philosophy (1641). But for now, let’s skip yet another explanation of how Descartes’s “I think therefore I am” declaration sliced the world into two different halves— objective matter and subjective mind—and focus our attention on the two ways McCarthy and his fellow coauthors modeled intelligent thought processes.

The Connectionist Computational Machine-As-Brain Model to Generate AI

This first approach relies upon, not surprisingly since Einstein and the quantum mechanics people made it the “cool” science, a “physicist’s” reductionist bottom-up tack. The connectionist models intelligent machines (computers) on the organizational structure inherent in the neurons and networks of neurons in the brain.

This is the “objective” computer-as-brain model.

This bottom-up approach holds firmly to fundamentally defining all of Reality as a place of objects. What distinguishes the real and true from the illusory and false is empirical measurement and numbers.

Quantities are primary. Essentially, if you can’t measure it, it’s not real.

Remember the binary of how we can know and understand Reality established in Part One: Orientation?

Here it is again.

Lens Number One conceives Reality as a place filled with existing objects or measurable facts—like rocks, bodies of water, trees, people, animals, plants, mist, etc.—in three primary state phases: solids, liquids, and gases.

Lens Number Two conceives Reality as an arena for action, where beings constantly judge the value of the objects around them so that they either increase or decrease their probability of solving three degrees of life experience problems. Those degrees are how to critically survive, proportionally thrive—mostly enjoying life’s ride more than enduring the suffering of life—and meaningfully derive better ways to survive and thrive.

The computer-as-brain model sees Reality through lens number one.

This framework hypothesizes that essential parts come together and aggregate to become wholes. In the case of living beings, these wholes can then display epiphenomenal qualities, meaning that a quantitative accumulation of firing neurons in particular patterns generates qualitative internal experiences, memory. An internal processor checks the memory and then directs (outputs) the being’s behavior. How beings contend with novelty (experiences not in their memory) is the $64,000 question, but the bottom line for the brain people (materialists) is that behavior programming comes from the environment.

Full stop.

In this view, beings are born blank slates, tabula rasa. Through feedback mechanisms, inputs into “being systems,” and outputs from those being systems, the environment selects the best “adapters.”

For fully committed bottom-uppers, beings don’t have “free will.” They just “epiphenomenally think” they do. Divinity for the bottom-uppers is an algorithm running on Pierre-Simon Laplace’s demonic mother machine. This deterministic formula can be hacked, thus the insistence that there is a single equation to rule them all, a theory of everything. With enough information, Laplace conjectured, all interactions—past, present, and future—can be predicted or retrodicted, which makes the arrow of time illusory. An influential thinker to this day, Laplace described this “demon” concept in his work “A Philosophical Essay on Probabilities” (1814).

Physics (random smashing atoms running on a single, elegant and knowable formula) is all in this worldview, and the rest (chemistry, biology, psychology, sociology, economics, history, art, religion) is nice and sometimes not nice, but it’s not “really” Real. In this view, technically, our emotions and experiences have no “real” causal effect. We’re just programmed by the underlying universal physics to believe they do. There is no self to take seriously. It’s an epiphenomenon, a fugazi, a fake and phony illusion.

The global concept for generating intelligence from the bottom up, then, is that if you mimic the organization of the brain’s neurons—think of them as a mathematician would, as nodes—and connect artificial neuron-nodes together into networks in a precise and logical way, the “thinking” qualities of the brain will at some point in time cross a threshold from nonthinking to thinking.

A physics “switch” turns on, and abracadabra, an insightful light bulb flashes, and “thinking” happens. Then you just train the thinking machine to do the things you wish it to do. You as creator can then play the part of the environment and program the behavior of the being under your selective command. Just like the universe and its various local environments do to us.

Robust theories—Kuhnian paradigms by 1955—that remain integral today and support the bottom-up approach are Norbert Wiener’s command-control Cybernetic theory ([1948] 2011) and Claude Shannon’s information theory (grounded in Shannon’s 1937 MIT master’s thesis, “A Symbolic Analysis of Relay and Switching Circuits,” and “A Mathematical Theory of Communication,” published in Bell System Technical Journal in July and October 1948 and then released as a book in July 1949).

Wiener tellingly named his theory by playing off of the Greek word “kybernetes,” which translates as “steersman” or “governor.” Cybernetics (pronounced with an “s” not a “k,” and the root word for “cyberspace”) is the intellectual underpinning of Psychologist B.F. Skinner’s influential categorical collapse explanation of behavior as learning through operant conditioning. That’s the carrot-and-stick approach to teaching. The controller rewards the trainee when they respond correctly to the controller’s stimulus commands and punishes them when they fail to.

Both Wiener and Skinner emphasized the role of the environment in thinking. The context that beings—and, by extension, machines—find themselves embedded within (another way of thinking about a specific locality in an environment, like “the office” or “the garage” or “at the pizza place”) is central to how they will behave. Change the contextual environment, and you’ll change the behavior of the content, the beings and the machines, in that context.

This view holds that the environment is the final “governor” of the objects, beings, and tools/machines—the artifacts created by beings. And thus, the survival of those same objects, beings, and tools/machines is a process of adapting to the inputs from the environment into those systems. Those that can adapt survive. Those that can’t die.

Nature dominates in this model. It impresses itself upon the being, and the being must align and remain submissive to nature’s domination.

As you can intuit, the physics as the only interaction model that matters is primary in this framework. In this view, life is downstream from physics. And everything can be reduced to how well living things can process environmental inputs and output correct behaviors—or input questions and output answers—that keep them alive. While consistent with Wiener and Skinner regarding the quantitative “math” as the bottom of everything, Shannon’s work concerned interpreting interaction as not “just math.” Instead, an interaction was communication, which he conceived as transfers of information—multi- packets of patterned bits of energy—input, some sort of internal processing, and then packeted informational output. Essentially, Shannon bridged the “pure math, all energy, physics” view with what he suspected words were...math patterns.

This is where his application of “Boolean algebra” took center stage. George Boole’s methodology reduced wordplay to “true” or “false” (Boole [1854] 1958). Shannon assigned the number “one” to “true” and “zero” to “false.”

Shannon studied Boolean algebra at Michigan as an undergraduate (1932–1936). He then worked as a graduate assistant at MIT while pursuing his math PhD (1940), contributing to the management of Vannevar Bush’s “Thinking Machine,” at the time, the most intelligent mechanical calculator in the world. Technically labeled the “Differential Analyzer,” it comprised steel wheels, gears, and rotating shafts that turned switches on and off. It weighed close to one hundred tons.

Programming the machine required assistants like Shannon to manually disassemble the monster and adjust its wheels, gears, and sprockets. That’s what Shannon was doing when he realized that Boole’s “trues” and his ones and Boole’s “falses” and his zeros paralleled the on-and-off switches the differential analyzer used to solve problems.

The ramifications were that if one used Boolean algebra, one could reduce a word—later on “true” and “false” representations, too—into a series of zeros and ones, which could, in turn, serve as on and off switches for a machine.

Shannon’s understanding of how energy could represent the signals of “on” and “off” that could be converted into zeros and ones, which in turn could be translated into words (patterns of signals), i.e., energy to number to word, and the reversal of that process, word to number to energy ratcheted the idea of cybernetics up to a whole new level.

Norbert Wiener, cybernetics’ didactic founder, respected Shannon’s work and his mastery of his own theory so much that he recommended that Shannon write the description of cybernetics for the Encyclopedia Britannica. Keep in mind that Wiener was famous for dissing John von Neumann. He feigned sleep when he attended von Neumann’s lectures and maintained an intellectual superiority to the polymath’s polymath. He obviously subscribed to the notion that a good offense is a good defense.

What separates Shannon from Wiener is his consideration of symbolic representation, a fancy phrase for language communication in addition to mathematical computation. That is, Shannon does not hand wave at “mind-generated” words as epiphenomenal from quantitative firings of neurons. Instead, he translated words into mathematical representations using George Boole’s logical linguistic principles, called Boolean algebra. This technique enabled him to boil down complicated language processing into binary “True/On or False/Off” logic gates. In other words, complex logical processes could be reduced to zeros or ones. Nos or yeses. Ons or offs. Trues and falses. This-es or thats.

These ideas, if not fully fleshed out, were embedded within his 1937 master’s thesis, “A Symbolic Analysis of Relay and Switching Circuits,” often cited as the most significant thesis ever written. It resonates to this day.

The gist is that Shannon postulated that words can be translated to the “bottom” as numbers.

While not spelled out directly, his work suggests that in the case of verbal communication, when words are received, the first “thinking” step is that the word inputs are somehow translated into numbers. The numbers are then crunched at the brain’s neuron/network level and then somehow up-regulated/translated back into words. Those words are then manipulated in another realm of experience called the mind. Once a worded response is formulated in the mind, those words are down-regulated— converted into numbers—to the brain, facilitating motor- action verbalization.

Wiener used relatively unambiguous and technical terms like input, processor, and output/feedback to describe the physical interactive loop. However, Shannon used broader phraseology like transmitter, which referred to the source of input, channel—the medium through which the message would travel—and receiver, which referred to the system or being picking up the signaled transmission.

But both cybernetics and information theory took a computational machine process stance when it came to the behavior of life. The “universe as a machine” approach was consistent with the legacy of the four-century-old scientific revolution. In short, per revolutionary figures like Galileo, Descartes, and Newton, the world is a mechanical process, and living things—with human beings endowed with language being the exception—are machines. Wiener and Shannon conceived behavior as a binary/digital mechanism that could be reduced to mathematics as the model for physics as the explanation for the deterministic mechanical universe. Cybernetics was at the bottom, and information theory was “next level.”

B.F. Skinner took those ideas and tested them with living beings. His carrot and stick operant conditioning framework—a key component in a mosaic of ways beings learn—is empirically robust, well-replicated, and one of the foundations of cognitive science.

But is that all there really is?