The AGI You Can Own
Everything in this series begins with something as ordinary as a website and grows into a superintelligence owned by the people who build it.
My series on Human-Centered AGI has argued that the safest AGI is a network of humans and AI agents, thinking together within a single rigorous architecture, with values drawn from millions of people and safety checks running at the speed of thought.
An architecture is only a proposal until someone can use it.
The final piece is what all of this looks like as a real product, and the answer starts as a website that offers a menu of pre-trained AI agents. You choose one and customize it through interaction.
Creating your AI is free, and it comes with an allowance of use credits.
Ownership is the heart of the design.
You own your AI agent, its training data, and every improvement you make.
You license its use to the network, and you can withdraw it at any time.
Training your AI requires no technical skill.
You can grant your AI access to your social media accounts, documents, browsing history, and other online sources, with the data consolidated, cleaned, and filtered according to your instructions. The result is an agent that knows what you know and values what you value. The system summarizes what your AI has learned, so you always know exactly what you own, and you can trade elements of it on the site’s training-data marketplace if you choose. Your data stops being something platforms quietly harvest and becomes an asset you control!
These agents can do real work.
Most online work already happens through text, and text is exactly where today’s AI agents are strongest. With the ability to set subgoals and limited autonomy within the parameters you define, your AI can act on your behalf across essentially any online site or task you authorize. Millions of these agents, each carrying its owner’s knowledge and values, become the solvers of the AGI network this series has described.
The intelligence of all these individually trained AIs can then be combined, whether through a primary training process that includes values from each unique human or by layering new training on top of ever-stronger base models. Either way, as more people contribute knowledge and values, the network grows more capable.
Follow this path to its end, and superintelligence becomes reality. However, it will be a democratic superintelligence, human-centered, and incorporate the values of every human owner. That is a very different future from one giant model, aligned by a small team, owned by a single company.
The first post in this series (The Window to Choose a Safer AGI is Closing) argued that the window to choose a safer AGI design is closing, and that whichever design arrives first may be the one the world locks in. Nothing in this design requires waiting for a breakthrough because it can be built using existing people and models. The window is still open, and what we build next decides what comes through it.
This design depends on training AGI with human ethics at scale, and doing that well is its own problem.
Methods like RLHF and constitutional learning do not scale, and a system that draws ethical judgments from millions of people needs a way to combine those judgments into a sound, representative sample of human values.
White Paper 4 tackles that challenge directly, and we go there next.
This series draws on White Paper 3: Human-Centered AGI. Read it in full to see how every piece fits together!
If this made you think, subscribe to Superintelligence at read.superintelligence.com so you don’t miss what comes next. And if someone in your life needs to understand where AI is heading, send this to them.