Many approaches to AGI today can be roughly characterized as building ever larger and more powerful large language models until one of them is so intelligent that it can do anything the average human can. There is another route. Marvin Minsky described it in 1986, and I believe it is both faster and safer than the other.
Minsky was among the researchers who gathered at Dartmouth College in the summer of 1956 for the workshop widely considered the founding event of the field of Artificial Intelligence.
Claude Shannon, Allen Newell, and Herbert Simon were there as well.
Each of these intellectual giants helped lay the foundation of the field, and each left behind an idea that matters more now than when he first published it.
About the same time that David Rumelhart, Geoffrey Hinton, and Ronald Williams published their landmark work on backpropagation, which helped lay the foundation for modern deep learning, Minsky published a highly readable book called The Society of Mind. Its first line proclaims that the book tries to explain how minds work. He answers that you can build a mind from many little parts, each mindless by itself. He called those parts agents. Each agent by itself can only do something simple that requires no thought at all. When we join these agents in societies, in certain very special ways, this leads to true intelligence.
The idea of AI agents has since become wildly popular. By the first nine months of 2023, Google Scholar was already finding more than 16,000 articles mentioning them. The idea of AI agents has since become wildly popular. By the first nine months of 2023, Google Scholar was already finding more than 16,000 articles mentioning them. Since then, the open-source community has built an entire layer of tooling for combining agents into systems, and open standards have emerged that let agents call external tools and communicate directly with one another, enabling agents built by different groups on different frameworks to work together in a single system. The pace is beyond any one person's ability to follow.
Minsky’s idea went further than building a series of agents.
Joining them in particular ways would yield true intelligence, or what we would call Artificial General Intelligence, or AGI, today.
Minsky was an early proponent of the idea that AGI emerges from the collective intelligence of many agents with lower levels of intelligence. Extrapolating from his view, achieving AGI will require a group of agents.
He never specified that the agents must be artificial. His stated goal was to explain how minds work, which I read as explaining how all types of minds work.
Minsky’s big idea was that combining the lesser cognitive capabilities of agents results in a more intelligent entity.
Couldn’t the agents being combined include both human and artificial agents?
The answer is yes.
I suggest that a Minsky-inspired system that harnesses the collective intelligence of human and AI agents is both the fastest and the safest path to AGI.
It is the fastest because human agents can handle tasks that artificial agents are not equipped to handle on Day One. It is the safest for two reasons. First, with humans in the loop, the system maximizes the opportunity for humans to align the system’s values with human values. Second, once AI agents learn from humans and begin to perform most cognitive tasks faster than humans do, we end up with a system composed of multiple AI agents rather than one. I have argued elsewhere that if each agent reflects the values of a unique human owner, the system’s collective values will be more stable than those of a single model trained on a small subset of values during the reinforcement learning from human feedback process prevalent today.
Take that one level further. A society of AGI minds can comprise a SuperIntelligence many times more powerful than the individual AGIs that make it up. If each AGI has a value system, the collective values of that SuperIntelligence are likely to be more stable than the values of any one AGI on its own.
Every one of these ideas is decades old.
Minsky published in 1986.
Shannon published in 1948.
Newell and Simon published in 1972.
When these pioneers developed most of their ideas, the dominant approach to AI was symbolic, and it was widely believed that the only realistic way to get intelligent behavior from machines was to program it into them as rules. Neural network approaches to machine learning were only explored in earnest in the 1980s, and they met intense skepticism from many of AI’s founders.
Three of these four scientists, Minsky being the exception, never lived to see deep learning begin to realize its potential.
Can we really learn anything new or relevant from scientists who never lived to see GPT?
I have two answers to this question.
On a personal level, I remember being a young graduate student in the 1980s, interested in AI and problem-solving. I had come to Carnegie Mellon to learn from Herbert Simon, who won the Nobel Prize in Economics in 1978 and who had co-authored Human Problem Solving, the definitive work on the subject, with Allen Newell in 1972.
In one of our first meetings, this great man recommended that I begin by looking at Wolfgang Köhler’s work from 1925 and Karl Duncker’s from 1945.
“Really?” I protested. “I came here to learn about modern problem solving, not to study the work of researchers who lived long ago.”
He shot back, “Surely, you don’t mean to imply that modern scientists have a monopoly on good ideas? There were also plenty of smart scientists back then, you know.”
Of course, he was right.
I discovered that both Köhler and Duncker were brilliant. Applying modern thinking and new experimental work to some of their fundamental ideas ultimately led to research that Simon and I published in Cognitive Psychology in 1990.
More importantly, I learned that an idea must be judged on its merits and not by the source, or even the period, from which it sprang. If the idea is powerful, it can drive innovation even if it was first expressed many years ago by thinkers now long gone. Given the opportunities and dangers that AI presents today, we need all the powerful ideas we can find!
My second answer is that the ideas are relevant if we can apply them productively to current and future problems of AI research. The proof is in the pudding.
Minsky’s gift is not sufficient on its own. Newell and Simon provided a rigorous framework that enables human and AI agents to communicate about any problem. Simon also argued that no amount of intelligence can derive right from wrong, which means an AGI must get its values from a source outside itself.
Shannon’s gift is the one this white paper turns on. His work implies that a system cannot get smarter without information that adds something to what it already knows. Large language models have gotten quite far by scooping up vast quantities of data available on the internet, cleaning and filtering it, and training on it. But as more of what is useful online has already been learned, taking in more of the same returns less and less. The problem shifts from gathering information to finding the information that is not already redundant.
The next post explains Shannon’s insight using an ice cream shop that sells two flavors, and what an AI does once it has learned the ice cream preferences of every human on the planet.
This series draws on White Paper 6: Catalysts for Growth of SuperIntelligence.
If this made you think, subscribe to Superintelligence at read.superintelligence.com so you don’t miss what comes next. And if someone in your life needs to understand where AI is heading, send this to them.