AGI, from First Principles

First Principles Analysis

First Principles Thinking (FPT) is a valuable tool to break out of wrong or limiting assumptions. This usually involves reducing an issue to its most basic fundamentals, challenging common assumptions, and perhaps taking a multi-disciplinary approach.

Here are some well-known examples:

• Apple’s iPhone — Steve Jobs broke out of the mold to re-imagined what a modern phone could do

• The Theory of Relativity — Einstein thought through the fundamental implication of a fixed speed of light

• SpaceX’s Reusable Rockets — Elon Musk deeply questioned what made space flight so expensive. He is known for seeking to apply FPT to all aspects of his various businesses (though he doesn’t seem to have done so with respect to AGI)

Now what are the core principles for creating AGI?

We need to understand intelligence in order to build it. The key question to ask is what makes human intelligence so powerful, so special? What is its essence?

Insights from epistemology (theory of knowledge) highlight the crucial role of concept formation — in particular our ability to autonomously form hierarchies of abstract concepts. Child development tells us about the need to have layered learning, that knowledge and skills need to be acquired in a certain order. Psychometric testing teases out the importance of metacognitive control, while cognitive psychology informs us about conscious vs subconscious thinking (Kahneman’s System 1 & 2) (ref)

Combining these, other insights and common sense allows us to get at the key characteristics of human intelligence:

• Conceptual and contextual learning and reasoning

• Real-time adaptive with limited knowledge, resources, and time

• Incremental learning, updating, and unlearning

• Autonomous metacognitive control and error correction

“We need to understand intelligence in order to build it.”

Here are two common objections to applying FPT to AGI:

1. “Nobody can define intelligence. I’ll know it when I see it. Intelligence is what intelligence does. We need to build AGI to understand intelligence.”

2. “We don’t need a theory of intelligence, we’re doing fine as we are. We’ll just continue focusing on benchmarks/ applications/ beating competitors”

As to the first, we clearly can describe the essential cognitive requirements for human-level AGI. We don’t need an overarching definition of all kinds of intelligence, however to believe that we can build intelligent systems without understanding what we’re trying to build reduces us to stumbling blindly in design space. You need theory.

As to the second, yes, with current theory-free approaches there’s money to be made, applications to be build, and competitors to be beaten. And money to be raised. However, there are strong indications that scaling LLMs is yielding diminishing returns and there’s a growing consensus (76% survey) that generative AI is not the path to AGI.

The road to AGI

With a first principles understanding of human intelligence one can clearly see that big data, statistical systems cannot get us to AGI – they lack autonomy (self-monitoring), and real-time incremental learning and adaptability. These limitations are not fixable.

On the other hand, first principles point us in the right direction — Cognitive AI, or what DARPA calls the ‘Third Wave’ of AI. Here the starting point is not how much training data or compute we have, but whether and to what degree the system is autonomous and adaptive. After all, our brains work with 20 watts, not 20 gigawatts, and children can learn language and reasoning interactively with less than 2 million words, not 20 trillion.