AGI Through Scaling — LLM Architecture as the Path to General Intelligence

Brown et al. (2020) show that increasing language-model scale can substantially improve task-agnostic few-shot and zero-shot performance without gradient updates, with GPT-3 evaluated across translation, question answering, cloze tasks, reasoning-style benchmarks, and other domains. This is supportive evidence that scaling a Transformer language model can broaden usable capability. The paper does not claim or establish that sufficiently scaling the existing architecture will produce AGI, and it does not establish the stronger legacy attribution that this was already the dominant AGI path hypothesis across OpenAI, Anthropic founders, and DeepMind researchers. It therefore supports the capability premise underlying the path claim, not the AGI-path prediction itself.

OpenAI's GPT-4 Technical Report (2023) documents a major capability advance over earlier models, including human-level performance on a range of professional and academic benchmarks, while explicitly stating that GPT-4 remains less capable than humans in many real-world scenarios. The report also shows that some aspects of GPT-4 performance could be predicted from much smaller training runs. This supports continued capability gains from large-scale language-model development while preserving a substantial gap between benchmark performance and general real-world competence. The report does not establish the legacy claims that GPT-4 delivered a smaller relative gain than GPT-3 over GPT-2, that MMLU/HellaSwag saturation was accelerating, or that scaling was already producing diminishing returns on the capabilities most relevant to AGI.

Ilya Sutskever left OpenAI in 2024 and co-founded Safe Superintelligence Inc. Reuters' reporting of his December 2024 NeurIPS remarks records his view that pre-training as practised through ever more data is approaching a limit because the stock of human-generated data is finite, and that future systems will rely on different strategies including reasoning. This is institutional and conceptual evidence that one influential architect of the modern scaling era expects the development formula to change. It does not establish that Sutskever has rejected scaling itself, that he believes current LLMs cannot contribute to AGI, or that his departure in 2024 was evidence of a 2023 internal rejection of the scaling path.

OpenAI's September 2024 o1 release introduces a reasoning-focused model trained with reinforcement learning to produce extended internal chains of thought. OpenAI reports that o1 performance improves with both additional reinforcement-learning compute during training and additional time spent thinking at test time, alongside strong results on reasoning-heavy evaluations. This is evidence that capability progress is being obtained through a broader compute-and-post-training recipe than simply enlarging a base pretrained model. It does not by itself establish that o1 is a fundamentally different architecture, that process reward models are the operative mechanism, or that continued progress toward AGI requires departure from next-token-prediction language models. It therefore pressures the meaning of 'scaling current LLM architectures' without settling it.

Villalobos et al. estimate the stock of public human-generated text and project that, if then-current LLM development trends continue, training datasets would become comparable in size to the available public human-text stock between roughly 2026 and 2032. They discuss synthetic data, transfer learning, and improved data efficiency as possible ways to extend progress. This is a real structural constraint on one input to continued pre-training scale, but not a demonstrated hard ceiling on LLM scaling. The source does not establish the legacy bundle that compute costs are growing faster than performance, that energy regulation will impose a binding infrastructure limit, or that the 2020–23 compute trajectory cannot continue through 2030.

OpenAI's public Charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work. This wording predates the late-2024 period attributed by the legacy instance and therefore does not constitute evidence that OpenAI migrated the AGI target in 2024–25. The source is relevant because it demonstrates one influential operational definition of AGI and illustrates the broader definitional problem surrounding the term, but it cannot support the record's earlier claim that institutional pressure caused a late-stage goalpost shift. The specific target-migration event previously attributed to IN-006 is withdrawn as unsupported by the cited evidence.

Snell et al. (2024) systematically study inference-time computation as a scaling axis. They evaluate search with process-based verifier reward models and adaptive modification of a model's response distribution, finding that the effectiveness of additional test-time compute depends strongly on prompt difficulty. Their compute-optimal strategy improves test-time scaling efficiency by more than fourfold relative to a best-of-N baseline, and in FLOPs-matched evaluations a smaller model using additional test-time compute can outperform a model 14 times larger on problems where the smaller model already has non-trivial success. This establishes that capability can be scaled through inference-time computation as well as pre-training scale. It does not establish the legacy 2025–26 field-wide claims about a consensus shift, a general latent saturation trend in training scaling, or specific frontier-lab capital commitments. For this record it is partial evidence that the operative path is broadening beyond the narrowest reading of 'scaling current LLM architectures,' not evidence that scaling has been abandoned.