It is almost axiomatic that AI, or at least modern machine learning, rests on three pillars: data, computing power, and algorithms. To make progress, one must innovate on at least one of these pillars. The pillar that may run short first is data, and the reason was worked out before the field of AI had a name.
Claude Shannon’s seminal paper, A Mathematical Theory of Communication, was published in 1948 and predates the founding of the field of AI by eight years. Still, his big idea, first elucidated in that paper, continues to have major implications for AI researchers today and in the future. While almost every page of Shannon’s paper is filled with mathematical formulae and notation, his essential insight can be described without math at all.
Here is how I typically explain the essence of Information Theory to my non-researcher friends.
Imagine that an ice cream shop has only two types of ice cream, strawberry and chocolate. Suppose you know that I am allergic to strawberries and love chocolate. If you see me walking out of the ice cream shop with a chocolate ice cream cone, does that event give you very much information?
No. That is because you already knew I loved chocolate and was allergic to strawberries, so you already expected me to come out with a chocolate ice cream. Seeing me with chocolate ice cream added little information, since it just told you what you already knew. Chocolate was the expected, and highly probable, flavor.
On the other hand, if you see me walking out with a strawberry ice cream, well, that is surprising. It is unexpected. It is a low-probability event and conveys much information.
Suddenly, you are learning a lot of information you did not already know, and your brain starts processing it.
Maybe I have overcome my allergy, but how?
I may be throwing caution to the wind and trying strawberry ice cream for the first time in years anyway, but why?
Maybe I am buying the ice cream for someone else, but for whom?
What Shannon said in his famous paper was that unusual or surprising events convey more information than expected ones. More specifically, he said that the amount of information conveyed by an event increases as its probability decreases. Simply put, the rarer or more unusual an event is, the more information it contains. Brilliant, and useful!
The concept of cross-entropy loss, used to evaluate the performance of many modern machine learning models, is an elaboration of Shannon’s big idea, as are almost all compression algorithms.
Of the three pillars, computing power is the simplest to increase. Throw more of it at the problem, using the same datasets and algorithms. But physics imposes limits on how many circuits can fit on a chip, how fast communication bandwidth can be, and how much power can be consumed before everything melts. So we must also work on new, better algorithms.
The Transformer architecture, as described by Ashish Vaswani and his co-authors in their 2017 paper Attention Is All You Need, illustrates the kind of performance improvement that is possible with new and better algorithms. However, algorithmic breakthroughs are difficult to predict, and even if we could predict the next breakthrough, there are limits to how efficient even the best algorithm can be. In machine learning, the limits ultimately depend on the amount of new information in the datasets used to train the model.
So we come full circle to Shannon. His work, together with the work of others building on his ideas, fundamentally implies that AI cannot get smarter unless it has new information to ingest.
Large language models have gotten quite far by scooping up vast quantities of data available on the internet, cleaning and filtering it, and then using it to train. But as more of the useful information available online has already been learned, taking in more of the same returns less and less. AI will have learned the ice cream preferences of every human on the planet, so to speak, and observing new human behavior will yield very little additional information.
What will AI do then? How will AI meet its insatiable demand for new information so that it increases its intelligence?
One possible scenario is that AI will begin generating new information itself by simulating trillions of new types of behaviors and scenarios much faster than human thought can. In this case, we might imagine millions of agents, mostly artificial but including some human, each processing existing information to create new information patterns and seeking the patterns that carry the most surprise. These new information patterns might then feed into a SuperIntelligence powered by all the agents in a Minsky-like community.
In Classical Information Theory, when every possible event is equally likely, uncertainty is at its maximum. Shannon’s measure of that uncertainty is entropy, and an equal-probability distribution has maximum entropy. Random noise is an everyday example.
Intelligence can be viewed as an anti-entropic force. Intelligence strives for order rather than the chaos of randomness. Intelligence is the signal on your television set, contrasted to the white noise, the snow of randomness. The snow is also extraordinarily unpredictable. Frame after frame carries very high entropy while telling the viewer nothing worth knowing. An intelligence that pursued whatever surprised it most would sit and watch the static.
So surprise alone is not enough. An intelligent system that wants to get smarter needs information that bears on what it is trying to do, and Shannon’s measure cannot tell it which information does. Shannon built his theory to measure communication. He did not build it to measure meaning. It tells us how unexpected an event is under a probability model. It does not tell us whether that event is worth anything to a mind pursuing a goal.
Information is better understood as difference, and surprise is only one kind of difference among many. The next post makes that case, and takes up Blaise Pascal’s remark in 1657 that he had made a letter longer only because he had not had the time to make it shorter.
This series draws on White Paper 6: Catalysts for Growth of SuperIntelligence. Read it in full to see how every piece fits together!
If this made you think, subscribe to Superintelligence at read.superintelligence.com so you don’t miss what comes next. And if someone in your life needs to understand where AI is heading, send this to them.