Many approaches to AGI today can be roughly characterized as building larger and more powerful large language models until one of them is so intelligent that it can do anything the average human can do. Marvin Minsky's collective intelligence approach is very different.
Minsky (1927 to 2016), Claude Shannon (1916 to 2001), Allen Newell (1927 to 1992), and Herbert Simon (1916 to 2001) were among the researchers who gathered at Dartmouth College in the summer of 1956. Each of these intellectual giants helped lay the foundation for the field of AI.
About the same time that David Rumelhart, Geoffrey Hinton, and Ronald Williams published their landmark work on backpropagation, which helped lay the foundation for modern deep learning, Minsky published a highly readable book called The Society of Mind. The first line of Minsky’s book boldly proclaims that it tries to explain how minds work. He lays out his big idea succinctly. You can build a mind from many little parts, each mindless by itself. He calls these parts agents. Each mental agent by itself can only do a simple thing that needs no mind or thought at all. Yet when we join these agents in societies, in certain very special ways, this leads to true intelligence.
What are the implications for modern AI researchers? Well, first, the idea of AI agents has become wildly popular, with Google Scholar finding more than 16,000 articles mentioning them in the first nine months of 2023 alone.
However, Minsky’s idea was not just that we could build a series of AI agents, but also that joining the agents in particular ways would yield true intelligence, or what we would probably call Artificial General Intelligence, or AGI, today. Using more modern terminology, we could say that Minsky was an early proponent of the idea that AGI emerges from the collective intelligence of many agents with lower levels of intelligence. Extrapolating from Minsky’s view, achieving AGI will require a group of agents.
What are these agents?
Well, many of them are AI agents, certainly. Since the release of ChatGPT in November 2022, there has been an explosion of AI agents populating sites like GitHub and Hugging Face. Using technologies such as LangChain, the open-source community is combining multiple agents into systems at a pace that is beyond any one person’s ability to understand fully.
Yet Minsky does not specify that agents must be artificial. Remember, his overall goal was to explain how minds work, which I read as explaining how all types of minds work.
Minsky’s big idea was that combining the lesser cognitive capabilities of agents results in a more intelligent entity. Couldn’t the agents that are being combined include human as well as artificial agents?
The answer, of course, is yes. I suggest that a Minsky-inspired system, harnessing the collective intelligence of human and AI agents, represents both the fastest and safest path to AGI.
Such a system would be on the fastest path because human agents would be able to handle any tasks that artificial agents are not equipped to deal with on Day One.
The system is also the safest path for two reasons. First, with humans in the loop, the system could maximize the opportunity for humans to align the AGI system’s values with human values. Second, once AI agents learn from humans and begin to perform most cognitive tasks faster than humans do, we end up with a system composed of multiple AI agents rather than one. I have argued elsewhere that if each AI agent reflects the values of a unique human owner, the collective values of the AGI system will be more stable than those of a single large language model trained on a small subset of values during the typical reinforcement learning from human feedback process prevalent today.
Finally, if we take Minsky’s ideas to the next level, we could imagine a society of AGI minds that comprise a SuperIntelligence, many times more powerful than the individual AGIs that make up the society. Similarly, if each AGI has a value system, the collective values of the SuperIntelligence that is comprised of the society of AGIs are likely to be more stable than the values of any one AGI on its own.
Thus, from both a practical standpoint, where the goal is to reach AGI or SuperIntelligence as quickly as possible, and a safety standpoint, where the goal is to have a stable, human-aligned value system, a Minsky-inspired collective intelligence approach seems promising.
However, given the rapid pace of AI development, one may question the relevance of AI research that is several decades out of date. After all, when these pioneers developed most of their ideas, the dominant approach to AI was symbolic. It was widely believed at the time that the only realistic way to get intelligent behavior from machines was to program it into them as rules. Knowledge engineering was in fashion. Neural network, or connectionist, approaches to machine learning were only explored in earnest in the 1980s, and at that time they met intense skepticism from many of AI’s founders. Further, three of these four great scientists, Minsky being the exception, never lived to see deep learning begin to realize its potential. Can we really learn anything new or relevant from scientists who never lived to see GPT?
I have two answers to this question.
On a personal level, I remember being a young graduate student in the 1980s, interested in AI and problem-solving. I had come to Carnegie Mellon to learn from Herbert Simon, a Nobel Laureate who had co-authored Human Problem Solving, the definitive work on the subject, with Allen Newell in 1972.
In one of our first meetings, this great man recommended that I begin by looking at Wolfgang Köhler’s work from 1925 and Karl Duncker’s from 1945.
“Really?” I protested. “I came here to learn about modern problem solving, not to study the work of researchers who lived long ago.” He shot back, “Surely, you don’t mean to imply that modern scientists have a monopoly on good ideas? There were also plenty of smart scientists back then, you know.”
Of course, he was right.
I discovered that both Köhler and Duncker were brilliant. Applying modern thinking and new experimental work to some of their fundamental ideas ultimately led to research that Simon and I published in Cognitive Psychology in 1990.
More importantly, I learned that an idea must be judged on its merits and not by the source, or even the period, from which it sprang. If the idea is powerful, it can drive innovation even if it was first expressed many years ago by thinkers now long gone. Given the opportunities and dangers that AI presents today, we need all the powerful ideas we can find.
So, my second answer to the question of whether ideas from these four deceased AI founders are relevant is this: The proof is in the pudding. The ideas are relevant if we can apply them productively to current and future problems of AI research. So, let’s find out. On to the pudding!
Minsky’s gift from the past might prove critical to the design of safe AGI and, therefore, to the future survival and prosperity of humans. However, we will need additional intellectual contributions from some of Minsky’s fellow AI co-founders to flesh out a vision of safe AGI. Newell and Simon provided a rigorous problem-solving theory that enables human and AI agents to communicate about any problem, with real-time safety checks as each goal and subgoal is set. Simon’s theory of bounded rationality explains why a vastly more intelligent system still needs to get its values from a non-rational source, which, for the human species, should be humans.
Shannon’s gift is the one this white paper turns on. His work implies that a system cannot get smarter without new information to ingest. Large language models have gotten quite far by scooping up vast quantities of data available on the internet, cleaning and filtering it, and training on it. But as more of what is useful online has already been learned, taking in more of the same returns less and less.
The next post explains Shannon’s insight using an ice cream shop that sells two flavors, and what an AI does once it has learned the ice cream preferences of every human on the planet.
This series draws on White Paper 6: Catalysts for Growth of SuperIntelligence.
If this made you think, subscribe to Superintelligence at read.superintelligence.com so you don’t miss what comes next. And if someone in your life needs to understand where AI is heading, send this to them.