AGI and Localization Risks in Sight Beyond the Scaling Race (Theory of AGI, AGI and Society, and Japan’s Choices: Part 2)
Introduction
Current AI progress is largely explained by “scaling laws.” That is, as model parameter counts, the amount of training data, and compute (FLOPs) expand exponentially, performance improves, and tasks previously thought impossible have become achievable. Whether “AGI (artificial general intelligence)” lies as an extension of this path has attracted widespread interest.
In the previous Part 1, we introduced the current situation where theoretical studies suggesting Transformer-based models can asymptotically approach Solomonoff induction (SI) have recently been published across multiple venues at once, bringing the contours of AGI into view along the scaling race’s extension. We also discussed that, considering the immense societal impact should AGI be realized, relying on the strong assumption that “AGI does not exist as an extension of the current technological race” entails significant risk.
Therefore, society needs to fully recognize the scenario in which “AGI does not arrive through a sudden leap, but could be reached continuously.”
In this three-part series, we discuss the possibility that the modern AI development race is continuous with AGI and examine its implications from the following three perspectives:
- Part 1: Introduced recent learning theory research suggesting Transformer-based models may asymptotically approach Solomonoff induction, illustrating the possibility that AI and AGI are more closely continuous than previously thought.
- Part 2 (this article): Building on the possibility of continuity from AI to AGI, examines the meaning of the scaling race led by mega-corporations in the US and China, as well as strategic issues for achieving a desirable future.
- Part 3: Explores the research and policy course Japan should take and presents several concrete proposals.
Continuity Between Scaling and AGI
—We Should Consider the Possibility of Reaching AGI Without a Fundamental Leap—
For a long time, there was a viewpoint that realizing AGI would require “fundamentally new algorithms” or “unknown paradigms.” Advances in theoretical research, however, are putting this view into perspective. If current Transformer-based learning algorithms have the potential to approach Solomonoff induction as data and compute increase, then the path toward AGI can be described not as a “leap,” but as an “asymptote.”
To be academically rigorous, this is naturally not a definitive assertion that “AGI will inevitably be realized.” All we can say is that “there is theoretical continuity.” Yet this distinction carries great significance for society. Because in the current situation, where it remains unknown how much more scaling will be required to realize AGI, neglecting preparations for a future where AGI becomes reality will cause irreversible delays should AGI actually materialize.
Anchoring on the “Reaching AGI” Scenario
The foundation of risk assessment is:
Priority for Consideration = Probability of Occurrence × Impact if Realized
Considering the extraordinarily broad and profound impact on society if AGI technology is realized, treating “not reaching AGI” as the primary scenario would be far too risky, even if there is a considerable degree of uncertainty regarding its probability.
This line of thinking resembles discussions around climate change countermeasures. Even if global warming does not progress as fast as predicted, investments in preparation retain a certain value. But if warming advances rapidly without preparations in place, society will face a grave crisis. Similarly for AGI, anchoring “reaching AGI” as one of the primary scenarios in policy and institutional design is a rational premise. Even if AGI is ultimately not realized, that simply means one anticipated type of catastrophe did not materialize; it does not necessarily mean the preparations made for it were wasted.
Localization Risk and Alternative Scenarios
Various commentators have long sounded the alarm regarding the grave risks of advanced AI technologies, including AGI, being monopolized or oligopolized by a handful of nations or entities.
Singleton Scenario
A concept that often appears when discussing AGI and the superintelligence (ASI) beyond it is the “singleton.” This was proposed by Nick Bostrom, formerly of the University of Oxford. AGI refers to technology capable of substituting for all intellectual tasks humans can perform. Since tasks humans can perform include improving AI, AGI would also be capable of improving itself. In other words, once an AI reaches a certain level of reasoning capability, it will be able to run a self-improvement loop—re-improving itself from that point onward without human assistance. A singleton refers to a state in which a single entity (an AI or an organization possessing it) monopolizes overwhelming intelligence through exponential performance gains driven by such a self-improvement loop. Bostrom argues that once this state is reached, the exponential widening of the initial capability gap makes it difficult to overturn the monopoly (the acquisition of a “decisive strategic advantage”). If a singleton emerges, its actions would unilaterally dictate the future of the entire planet, which is considered directly linked to “existential risk” for humanity.
Multipolar Scenario
While the plausibility of the singleton scenario has been discussed from various angles by Bostrom and others, it can also be seen as an extreme assumption that disregards physical and social constraints. If a singleton does not emerge, the next conceivable scenario is the “multipolar scenario.” In a multipolar scenario, multiple entities (such as nation-states or corporations) hold an oligopoly over AGI/ASI technology and exist in mutual competition. Even with multiple entities, if they are limited to a small number, a massive power disparity will arise between those who possess AGI/ASI technology and those who do not. Moreover, because the AGI/ASI entities would remain in constant competition with one another, this is considered an unstable scenario in which accidents or catastrophes originating from AI technology are prone to occur.
Ecosystem Scenario
In contrast to these not necessarily desirable scenarios, a more preferable alternative is the “ecosystem scenario.” Here, we envision not merely a multipolar state, but one in which an interdependent AI network is formed. Various types of AI agents—including AGI—and the individuals, corporations, and other entities that make up human society are interconnected. By complexly and spontaneously forming relationships of mutual dependence, mutual verification, mutual deterrence, mutual delegation, mutual auditing, and mutual complementarity, the system is expected to exhibit robustness akin to a natural ecosystem. This is also an original idea developed by Japanese researchers grounded in an Eastern sensibility.
An ecosystem-network approach offers diverse advantages. If a certain AI attempts to run rogue, other nodes can handle verification and control, enabling the stable operation of the entire combined human–AI system. Furthermore, the presence of AI models based on diverse designs, data, and cultural backgrounds reduces the risk of a single flaw cascading across the whole. It provides mechanisms for ensuring transparency through mutual auditing, as well as verifying the validity of decisions and outputs from multiple perspectives. In short, incorporating AI into human society as an ecosystem-like network can be regarded as the desirable path from the perspectives of safety and reliability.
Note: However, merely configuring a complex network does not automatically guarantee that it will be desirable for human society. Clarifying theoretically and mathematically what conditions must be met for a combined human–AI system to function stably and soundly is one of the primary research themes of our AI Alignment Network.
Detailed explanations of these scenarios and their conditions for establishment are elaborated in the following paper:
The Need for International Multipolar Competition
At present, cutting-edge AI development, which requires enormous capital and infrastructure, is concentrated in a few mega-corporations in the United States and China. Backed by massive compute and vast training datasets, these companies are driving the scaling race. While their technical achievements are significant, they are extremely unevenly distributed internationally. This structure threatens to materialize future localization risks and is a matter of concern.
To enhance the stability of human society as a whole after AGI technology is realized, it is necessary to foster as many third, fourth, and subsequent poles as possible beyond the US and China. If multiple nations and corporations exist as major players, the likelihood of transforming a multipolar structure into an ecosystem-network structure increases. Furthermore, to ensure robustness and antifragility across society as a whole, it is crucial that these players possess diverse cultural and technical backgrounds, thereby ensuring a diversity of values.
Candidates for third, fourth, and subsequent poles are expected to include Japan, Europe, India, and Middle Eastern nations. Among these, Japan in particular possesses the prerequisites in terms of technical foundations, research talent, and international trust, making it possible for the country to assume that role through large-scale investment.
Conclusion
The insight shown by theoretical research—that “AGI is positioned as a continuous extension of the scaling race”—brings about a major shift in premises for society. If we understand that AGI has the potential to be reached not through a “leap” but through an “asymptote,” the priorities for preparation change significantly.
Namely, while confronting the dangers of AGI technology being monopolized by a small number of players, we must advance the construction of multipolar, interdependent network-based AI, and more nations must invest proactively to foster forces beyond the US and China.
For Japan, this direction aligns with both enhancing the stability of human society and maintaining its competitiveness as a developed nation. In the next installment, Part 3, we will present concrete proposals regarding the actions Japan should take and the policies it should implement amid this technological and international landscape.
This article is reprinted from the ALIGN Blog (October 10, 2025). Original article: https://www.aialign.net/blog/transformer-agi-part2
Koichi Takahashi