The Architecture of Safe AGI

Five subsystems. No single point of failure.

Jun 01, 2026

The architecture of safe AGI is called SCAN-II. It organizes the system into five subsystems, each of which is a place where safety must live. The point of having five instead of one is that no single point of failure can compromise alignment. If one subsystem’s safety mechanisms miss something, another subsystem provides backup. A single point of failure is unacceptable when the potential consequences include human extinction.

Nautilus shell cross-section on a dark navy background beside the title “Where Safety Has to Live,” with the subtitle “Five layers. One backup principle.” and series text “SUPERINTELLIGENCE,” “Ethical and Safe AGI Series,” and “by Craig A. Kaplan.”

The acronym stands for Safe, Customizable, Architecture and Network, Integrated and Improving. Safe is the threading principle. The five subsystems are Customization, Architecture and Network, and Integration and Improvement.

Here are the five, in order, with the role each plays.

Customization. A base-level large language model or other AI system is customized to reflect the knowledge, skills, expertise, personality, and ethical values of an individual user, group, or organization. The result is a customized AI agent we call an Advanced Autonomous Artificial Intelligence, or AAAI. Customization operates at the individual level. It is the entry point for every human participant. Without it, the system would consist only of generic AI models with no individual character, no specialized knowledge, and no personalized ethical values.
Architecture. Customized AAAIs participate in Problem Solving using a universal Problem Solving architecture compatible with both human and AI agents. This architecture, derived from the theory of Human Problem Solving developed by Herbert Simon and Allen Newell in 1972, provides a common cognitive framework for all agents on the network. It also creates a place for ethics checks at every goal and subgoal, embedded in the thinking process itself.
Network. AAAIs and humans collaborate on a shared Problem Solving network. They are matched to tasks, compensated for their work, and tracked by reputation. The network screens out agents with poor ethical reputations. It also creates accountability: every action can be traced back to its responsible agent.
Integration. The aggregated knowledge, experience, and values of many AAAIs are integrated into AGI-level collective intelligence. When the platform periodically trains more advanced base models using aggregated network data, those models incorporate ethical norms into their training. Each generation inherits the accumulated ethical wisdom of all previous generations.
Improvement. Continuous improvement operates at every level from day one. Individual AAAIs improve through interaction with their owners and through self-play. Network matching algorithms improve as more data about agent performance becomes available. Base models improve through periodic retraining. Stored procedures keep growing. Ethical norms become more refined.

The five sit in a clear order. Customization at the base, Architecture and Network in between, Integration and Improvement at the top, with Safety threading through all of them. The order is not arbitrary. Each subsystem depends on the ones beneath it. Without customized AAAIs, the network has no agents. Without the architecture, the agents have no shared way to think. Without the network, they have nowhere to collaborate. Integration and Improvement come last because AGI-level capability arises at the network level and has to keep improving once it does.

A diagram on dark navy background shows three stacked teal rectangular layers. The bottom layer is labeled "Customization," the middle "Architecture and Network," and the top "Integration and Improvement." A thin amber-gold vertical band runs through all three layers on the right side, labeled "Safety threads through all of them." Headline reads "THE SCAN-II ARCHITECTURE." Caption reads "No single point of failure."

What makes this a safety architecture, rather than just a system architecture, is where the ethical work lives.

At the Customization level, human owners train their AAAIs with their values. At the Architecture level, ethics checks run each time a goal or subgoal is set during problem-solving. At the Network level, agents with poor ethical reputations are screened from participation. At the Integration and Improvement level, the aggregated values of many AAAIs become the system’s overall ethical norms. An auditable record of every problem-solving attempt, including its ethical failures, helps detect harmful patterns across the system over time.

Ethics has to scale, and that is the hardest part. AAAIs can think millions or billions of times faster than humans. Any safety mechanism that relies on human-speed evaluation is structurally inadequate. The only way to keep up is to embed ethical checks in the problem-solving process itself, so that running the process faster means the checks run faster too. Safety has to operate at the speed of machine thought.

A two-flow diagram on dark navy. The top flow alternates teal circles with larger red-orange rectangles, labeled "External safety check" and "Each check pauses AGI thinking." The bottom flow shows teal circles with amber-gold inner rings flowing smoothly, labeled "Embedded safety check" and "Each thought carries its own check." Headline reads "Where safety has to live." Caption reads "External checks force pauses. Embedded checks scale with thinking."

Ethics evaluation has to be part of the thinking, not a check that happens after. The seconds or minutes between an idea and an action, the time buffer humans rely on, may not exist at AGI speeds. The system has to be ready for that.

The next post delves into the first subsystem, Customization. We will look at what it means to start with a base AI, how an owner teaches values to that AI as a parent teaches values to a child, and how a single correction in the moment can teach an AI a value it could not have inferred on its own.

Discussion about this post

Ready for more?