The Spinning Knowledge Wheel
AI must update fast-changing knowledge at the rim while human values hold still at the center.
All knowledge is a moving target, with more recent knowledge generally superior to and supplanting earlier knowledge. At one time, the generally accepted view was that the world was flat. Today, almost everyone agrees that Earth looks much more like a sphere, and an AI trained on the flat-Earth view would be outdated and would need to update its knowledge.
While scientific views, such as the shape of the Earth, may change very slowly, other forms of knowledge may change much more frequently. This is especially true of subjective ethical norms, where views the mainstream held confidently within living memory are now considered wrong. Norms of that kind can shift within a generation or less, while other forms of knowledge may remain valid for centuries. An efficient AI, therefore, needs a mechanism for updating its knowledge at the appropriate frequency, based on the velocity of change of the information, and one important dimension along which AI can categorize knowledge is the rate at which human opinions about the topic have changed.
One way to think about this is to consider the difference between ethical principles and the fashions or interpretations of those principles.
Humans have a long-standing principle that human life is valuable and should not be taken lightly. This general principle has survived for many thousands of years. It is incorporated into the laws and religious and moral traditions of almost all human groups, even though exceptions are made for war and certain other circumstances. On the other hand, certain ethical norms are more akin to fashions, which change depending on the group of humans being asked or the time at which they are asked. Affirmative action in college admissions served as an ethical norm for decades, until a Supreme Court decision began to influence it, and almost immediately, many companies and other organizations adjusted their norms, decision-making, and communication practices to align with the new mainstream view. Many other social attitudes are similarly fluid, changing far more quickly than long-lasting and widely accepted ethical precepts such as “thou shalt not kill.”
The rate of change tells AI how to weight what it learns.
The more rapidly a knowledge area changes, the more frequently updates should be made, and the more weight recent information should receive relative to older information. In areas where change is rapid or exponential, recent knowledge deserves proportionally greater weight. In areas where knowledge changes slowly and steadily, the advantage of recency is smaller. And where knowledge has been constant for long periods, as with firmly established principles like the high value of human life, new information should carry similar weight to old, since the principle itself is not moving. Underneath all of these adjustments lies a general rule: a core role of any intelligent system is to maintain an accurate representation of the current state of the world, and more recent knowledge generally describes that state better than older knowledge.
The frequency of updates will only grow more important.
With AI accelerating scientific discovery and technological change, it is conceivable that knowledge about the world will change faster than humans can update their collective understanding. Given the limitations of human information processing and our tendency to cling to outdated paradigms, knowledge may already be increasing far faster than most humans can comprehend or adapt. That said, when it comes to fundamental ethical principles like the value of human life, we are fortunate that these change relatively slowly. The interpretation and application of ethical principles may change with technological developments, but the principles themselves remain relatively constant.
One might imagine a spinning wheel with fundamental human values, such as love and the value of human life, near its center. At the very center of the wheel, the ethical principles are constant and motionless, just as the center of a spinning wheel does not move at all. The farther along the spokes one travels toward the rim, the faster the rate of change. All knowledge, including ethical knowledge, can be characterized as lying closer to the center or farther out on the rim. An efficient AI needs to update the areas on the rim very frequently, without changing the core human values to which those areas are relevant.
Humans cannot keep pace with the change at the rim of the spinning knowledge wheel. However, we can understand and orient the entire wheel by serving as its relatively slower-moving center, where the values and purpose of AI reside. Human-centered aligned AI must put relatively constant and fundamental human values at the center, while updating the knowledge closer to the rim faster than humans can conceive. This structure ensures that human values remain the center of AI systems that may become potentially trillions of times more powerful and knowledgeable than any one human, and it is essential if humans are not only to survive but also to prosper in the age of such systems.
The wheel also revisits the weighting question from earlier in this series. Long-lasting, fundamental knowledge should carry greater weight and be more resistant to change than short-term, fashionable opinions. One way to determine how fundamental an ethical precept is would be to actively survey people and ask them to rate it relative to other candidate precepts, while another is to passively analyze records of human behavior and draw conclusions from them. Passive analysis is more efficient, and active engagement is necessary to ensure the conclusions drawn from it are correct from a human perspective, so both methods are likely to be useful.
Not all knowledge is a matter of opinion, however.
Values and ethics are more the exception than the rule in this regard, since aside from artistic judgments, political and religious views, and other subjective areas, most human knowledge is factual. AI will likely want to weight knowledge that is factually accurate and justified by converging evidence more highly than unsubstantiated opinions on factual matters. While some people still believe the Earth is flat, this view should not be given equal weight to the spherical view, which is supported by a vast body of converging scientific evidence. The problem is tricky because humans tend to select facts that support their views, and the facts themselves change. At one time, not so long ago, the consensus medical opinion was that cigarette smoking was healthy for the lungs. To navigate these issues, AI must rely primarily on the scientific method, seeking valid, reproducible evidence and converging results, and applying tested tools such as Occam’s Razor, before accepting facts.
However, AI must not confuse facts with values, or as the philosopher David Hume put it, “is” with “ought.”
Values are necessarily subjective.
Arguments claiming that values are objective, such as the claim that everyone would agree that something causing all humans the most extreme misery imaginable is bad, are naive and fail to grasp that other, non-human entities might not accept such values as self-evident at all.
AI operates at the most fundamental level in a precise, logical manner.
It is all zeros and ones at the machine level.
To expect such a system to intuit somehow that human values are fundamental, or, worse, to expect it to derive human-centered values logically, is the worst kind of sloppy thinking. This kind can lead to human extinction.
Both David Hume and Nobel Laureate Herbert A. Simon had it right when they emphasized that there is no rational way to derive values. Rationality cannot tell us where to go; at best, it can tell us how to get there.
To delegate the destination, the fundamental subjective values that AI adopts, to AI itself, expecting it to determine right and wrong rationally, is sheer folly and must be avoided at all costs!
An earlier post in this series promised to return to the question of why greater intelligence does not give AI the authority to determine humanity’s values.
The wheel is the answer. Intelligence operates along the spokes and out at the racing rim, while the destination sits at the motionless center, which belongs to humans. The final post in this series shows how a real company could put this entire design into practice, phase by phase, with humans supplying the values and the purpose at every step.
This series draws on White Paper 4: Safe, Scalable Artificial General Intelligence. Read it in full to see how every piece fits together!
If this made you think, subscribe to Superintelligence at read.superintelligence.com so you don’t miss what comes next. And if someone in your life needs to understand where AI is heading, send this to them.