“AI Will Soon Become AGI”
In the last few years, AI has been improving exponentially. Some think that trend will continue forever and even accelerate until life on earth becomes unrecognizably futuristic. That AI will soon exceed human abilities in every way. (Artificial General Intelligence, or AGI) Certainly, the investors and CEOs need us to believe this so they can make fat stacks of cash to pile on top of their existing stacks of cash. But seeing as how I’m stackless, I can try for a more realistic view. While it’s true AI can scale in ways that human intelligence cannot, can transfer data much faster than humans can, and replicate itself in ways humans can’t, AI still has real limits to overcome that often get glossed over. We will do some unglossing here.
Here are a litany of challenges standing in the way of AGI:
Infrastructure
Large Language Models (or LLMs) consume gobs and gobs of electricity. According to MIT Technology Review, just to make a 5 second video with Sora requires 3,400,000 joules–enough power to run a microwave for over an hour. https://www.
We can’t produce enough power to meet current AI demand, let alone future demand. That’s why tech giants are building dedicated power plants for their data centers.
Companies like Google and Facebook can afford this, but most IT companies cannot. So they ask existing power companies to expand current plants or build new ones or delay the retirement of old ones. In any case, expanding power production is notoriously slow and expensive for reasons I won’t list here. That would fill its own article.
No matter the type, it takes years to build a new power plant. Nuclear often takes a decade or more. Between planning and permits, building and testing, environment studies and grid interconnection, the effort and red tape are monumental. And much of that is justified. If someone builds a reservoir for hydroelectric, and you live in its floodplain, you want to know the wall of that dam will hold. They don’t make snorkels long enough for floods like that. 😉 Given the complexity of these projects, keeping within budgets and schedules almost never happens, and paying to run and maintain these plants once they’re built isn’t cheap, either.
This long timeframe is unavoidable, no matter how much money you throw at the problem. And getting cities, counties, and states to agree on where to let you install new transmission lines takes 4 to 10 years, on average.
Training
Current AI models go through 2 phases: Training and Inference.
Training requires massive computational resources and the aforementioned power guzzling. LLMs have to scrape the entire public internet for data and process it, baking in numerical assumptions that teach the AI how to solve problems.
How much power does this need? MIT Technology Review says, “it’s estimated that training OpenAI’s GPT-4 took over $100 million and consumed 50 gigawatt-hours of energy, enough to power San Francisco for three days.”
After months of this training, the LLM gets shrunk down and released to smaller data centers that host AI prompts for end users. This is the Inference phase, when AI uses what it learned to answer questions. Problem is, at this point, the model is has learned all it will ever know. (With some small exceptions.)
But if AI is to match human thinking, it must be learning all the time, long after it graduates from AI school. But because its training is too computationally expensive to perform on small devices or even midsize data centers, it cannot learn on-the-fly. To enable this, we’ll have to make training fundamentally faster, cheaper, and more efficient so it can run on smaller hardware. It’s quite possible that LLMs must be replaced with a superior algorithm–that our current approach will never scale adequately. Or we’ll have to pack massive computational resources into smaller and cheaper places. Or both.
Solving problems this big will not be quick or easy.
Data
AI companies are running out of new data to train on. LLMs have already scraped all publicly available data on the internet. Content stored behind password prompts has been spared, for now. But big tech is searching for workarounds. Oracle helps users connect their private data stored in databases to an AI model to supplement its original training knowledge. They call this RAG (Retrieval-Augmented Generation.)
This sounds great in theory, but how many people would willingly feed their financial and medical data into AI and trust it not to leak it to the whole world? Or do something crazy with that info, like buy a thousand salt lamps with your credit card? Granted, we trust companies to store our credit card info today. That’s nothing new. But the software those companies use are not subject to hallucinations or its own deceptive agenda like AI agents are known for.
Don’t companies trust procurement experts to buy expensive equipment on their behalf? How is AI-empowered purchasing any different? Well, presidents know they can fire fraudulent employees and even sue them. But if AI embezzles from them, how will they fire and sue?
AI has a LOT to prove before users will trust it with their own money and private data. But perhaps that decision won’t be ours for long.
The CEO of Oracle said, “…we took all of our customer data and we vectorized it and used RAG to make it available to the models.” Here is real footage of him admitting this:
But even if AI companies somehow stole all private data and shoveled it into training models, that still wouldn’t be enough data to train AI to a human degree. Why? Because you can’t find much taste, touch, and smell data on the internet. (Some exists, but it’s a pittance compared to text, video, and audio.) There are entire domains of the physical world AI cannot train on without this data. We humans can feel the weight and texture of a chicken egg. We can smell omelets and know how they taste. ChatGPT has only the fuzziest idea of these things based on watching Food Network.
These senses may seem trivial compared to sight and hearing, but would you trust a robot chef that couldn’t taste or smell and had little sense of texture? I wonder if it could distinguish between fresh ingredients and spoiled ones. And would you hire a robot investigator that confused ketchup with blood or that smelled rotten cabbage and went looking for a dead body?
Even something as basic as word meanings can be hard for AI due to lack of data. Not all human languages are on the internet–especially ones spoken by primitive people groups that don’t even have a writing system. Those may seem like exceptional cases, but AI even struggles with major languages like Arabic. For one thing, it’s diglossic, meaning that the vocab used in writing is often quite different from the vocab used in daily speech. On top of that, Arabic has many dialects. Which one a given video uses is usually not labeled. People simply have to know the differences well enough to tell dialects apart, and to sense when a given speaker is mixing dialects–sometimes mid-sentence.
For all these reasons, finding enough data is a real limit for AI. Like infrastructure and training problems, this is not easy or quick to solve, and there may be little financial incentive to capture and upload all this missing data to the internet. Think of the vast array of smells in the world. Which company is going to pay someone to fart into a gas chromatography-mass spectrometer or a chemical sensor array?
-Biswarup Ganguly, CC BY-SA 4.0, via Wikimedia Commons
I imagine this wouldn’t be a one-and-done event, either. Such data providers would have to squeeze out hundreds of samples after eating a wide variety of foods to provide enough data for electronic noses to rely on.
If AI takes my programming job and writing career, this might become my backup plan. 
Testing
As LLMs improve, they will run longer before making mistakes. This sounds great, but consider the ramifications: AI developers will have to test a model longer in order to find and fix latent bugs. If it takes only 10 minutes for a self-driving car to make a navigational mistake now, testers only need do 10-minute test runs. But if it takes an hour for the car to take a wrong turn, the same test runs will take a full hour.
And because long-running tasks are the real challenge for AI, this is not a theoretical problem. Real world scenarios can be long and complicated and can’t always be decomposed into many short tasks. We could make a simulation and speed that up to the expected point of failure, but there are limits to this, too. And testing coordination b/n several agents will get harder as those agents become more numerous and powerful, because the testing factors and use cases can balloon into extremely high numbers of combinations.
Evaluating Success
The real world is chaotic. Deciding how well AI scored on a standardized test is easy–just compare its responses to an answer key. And testing the performance of a self-driving car is fairly simple, too. Either it reached the destination or got lost. Either it hit pedestrians and traffic signs, or it didn’t.
However, evaluating a video game built by AI isn’t so clear. If it creates a boring or confusing mess that nobody wants to play, how do you help AI avoid this next time? If you complain of monotonous gameplay, will it know how to add variety without throwing together random, incohesive elements? And if AI writes a novel with characters who make illogical decisions, how do you fix this? Current AI is notorious for messing up cause-and-effect because it relies on correlation without understanding causation. You can’t just say “Your character motivations are silly. Fix it.” AI would need more specific guidance. But what would such guidance look like? You could amend your prompt to explain how each character would truly react, but this will likely create new inconsistencies.
AI lacks nuance. It cannot consider the context and decide accordingly. Self-driving cars stop when they see a picture of a stop sign on a bulletin board. Alexa once dared a child to plug a phone charger partway into a wall socket and touch a penny across the exposed prongs! Toddlers can tell when someone is waving and when someone is swatting away a fly, but AI just sees two different hand movements.
Exponents
Even if current technology could get us 80% of the way to AGI, achieving the final 20% would prove MUCH harder. Diehard AI apologists claim this is no problem because AI has been making exponential progress, and they see no reason for that trend to stop. But many things in life that seem to be exponential functions turn out to be S curves when you reach unforeseen limits:
- Biological populations. without predators, the population of a new species can boom. But when food and space run out, that growth slows down. Males compete for mates and start killing each other, stunting growth further.
- Disease transmission. At the start, each new victim transmits a virus to new carriers who infect others. This runaway effect seems boundless until people recover and gain immunity. Many others die off, reducing the number of possible hosts. When the virus runs out of vulnerable people to infect, the exponential curve flattens.
- Tech adoption. When the iPhone 3G released, its lower price triggered rapid growth. But the market became saturated, and the time for each new adopter stretched out further and further as only the more skeptical users remained.
Many of AI’s early leaps are likely due to solving the easier problems. Just look at the “Will Smith Eating Spaghetti” benchmark to see how far AI videos have come:
With dramatic improvements like that, no wonder people are hyped about AI. But once you solve all the easy problems, only hard ones remain. Often, those problems take far longer to solve, if they can be solved at all.
It’s tempting to extrapolate past results forever into the future, but history has proven that easy wins eventually run out. But… as some experts have observed, AI isn’t a single curve of progress. It only appears that way with casual observation. When you zoom in on it, you’ll find it’s made of many smaller S curves, each representing one small algorithm improvement. As long as these improvements keep happening, the overall progress curve may continue to be exponential, which may look something like this:
 |
But this assumes we’re able to keep this up. See how the S curves must become more steeply sloped to keep the main curve exponential? That could be mitigated by adding multiple new improvements at the same time, but that assumes so many improvements are possible. Which could happen if AI produces a self-improvement feedback loop where each innovation spawns two more, like a chain reaction.
But that sounds like a nuclear explosion, and I don’t even mean that in a ominous way. I merely point out a simple fact: once all the weapons grade plutonium or uranium in a warhead has exploded, the chain reaction stops. There are simply no more unstable atoms to undergo fission. Put another way: AI might have hard limits we don’t know about because we’re not there yet. It may run out of fissile material, so to speak.
To those who insist that AI will keep improving itself forever, I ask the following:
-What if further development needs more power, and we someday build so many power plants on earth that the waste heat is killing us for lack of cooling?
-Fine, we just build data centers in space, like SpaceX and Google want. But who will launch all the raw materials up there when it costs thousands of dollars per kilo? Even if AI designed a plausible space elevator, would we have the exotic materials and all the technology to build such a thing?
-Maybe that’s the wrong approach. Say we send AI-powered mining bots to nearby asteroids to mine raw materials in space. Then we don’t need to send up material in chemical rockets. But what if AI can’t figure out how to keep mining bots attached to the asteroids? Drilling creates a recoil that would push bots away. They can’t just clamp on, because the surface of these asteroids are often not solid rock but loose collections of rubble that easily break apart.
-Assuming AI solves that problem, what if these asteroids are so far apart that mining bots can’t collaborate in real time, slowing progress? Now they need faster-than-light signal transmission, but not even AI can break the laws of physics.
-There might a way to break the light speed barrier that we haven’t discovered it yet. Maybe AI could. But what if it lacks the lateral thinking needed to pioneer scientific breakthroughs? Especially when such breakthroughs rely on guesswork and hunches that lie well beyond its training data.
-What if further progress eventually requires us to solve problems that are truly unsolvable? (such problems are proven to exist in math, for example)
At some point, we may reach the point of diminishing returns, where further discoveries become increasingly harder to make. That’s not to say humanity will be stuck. It just means people might actually get time to adjust to current tech before being thrust into the next big thing. And give the poor courts time to figure out how to rewrite laws around all this change.
Political Pushback
People don’t want to lose their jobs. Or have their secrets stolen and plastered all over Facebook. Or data centers built in their neighborhood where they can’t sleep for all the loud humming. Or be deprived of drinking water because those data centers are guzzling it up. Or be forced to pay increasingly higher power bills to bankroll new data centers and power plants. Or sift through more and more AI slop on the internet. Or have their creative works strip mined by LLMs, destroying demand for their own art. Or pay higher and higher prices for a basic laptop because the price of IC chips is perpetually hiked up by AI’s never-ending demand.
Many people resent the way AI companies are imposing themselves on the rest of the world, and some are fighting back. Hacktivists are poisoning training data, hijacking AI agents, and tricking models to ignore their internal rules so that they generate malicious code and leak secrets. Coalitions are stopping data centers from being built in their hometowns. Unions like the Writers Guild of America are using collective bargaining in an attempt to stop their work from being fed into training models. Record labels have sued AI companies for infringing on their copyrights. And I expect this resistance to grow as more people lose their livelihoods. Such resistance is already slowing AI’s rise in noticeable ways.
Financial Bubble
If all the other speedbumps weren’t enough, AI is in a precarious funding situation that comes in 2 parts: circular funding and real customer demand.
In 2026, Nvidia invested $30 billion into OpenAI, who used that money to buy Nvidia’s GPUs. This makes Nvidia’s revenue look awesome and makes OpenAI look well-funded, even before a real customer pays for the product.
Then Microsoft invested $13 billion into OpenAI, though much of that was not paid in cash, but in Azure cloud credits. OpenAI could record this as capital raised, but they can’t pay employees with those credits. Microsoft’s “investment” amounted to a gift card that could only be spent at their own store. To repay Microsoft, OpenAI must pay back 75% of its earnings in real dollars. Many other companies have entangled themselves in this circular funding, and here’s a video to break that down in more detail:
Where will OpenAI get enough money to repay all these investors? The obvious answer is AI subscribers. Problem is, even their highest tier ChatGPT subscription of $200/month makes OpenAI lose money. The company pays more for compute resources than they earn back through the subscription.
So why not charge users more? ChatGPT is the leading AI tool right now, and OpenAI wants to keep it that way. If it raises prices, it may lose market share to competitors like Anthropic. Also, these losses are being absorbed by the exorbitant investment money that keeps flowing in. OpenAI is betting that a few years of massive losses will be more than made up for by big profits later on.
But what if they’re wrong?
What if creditors suddenly demanded their money back or investors sold off their stocks? That’s not unlikely, especially if they consider these roadblocks and conclude that AI may never turn a profit for anyone but Nvidia. And what happens if prices rise to meet their real costs? Would enough users keep their subscriptions? If not, much of the existing revenue vanishes, compounding the problem.
The ensuing financial disaster would be far worse than the dot com bubble, but it would take a full article to explain why. Maybe I’ll elaborate on that next month. Or give you a break from AI because this topic is getting heavy.
The Wrong Takeaway
I’m not saying that AI is a dead end. Far from it, I think it’ll make brow-raising progress in the coming years. Some of its biggest impacts may not be fully grokked for decades when we look back through a historian’s monocle. All I’m saying is this: before AI can surpass human intelligence in every way, there are some very hard problems to solve. And as with all engineering projects, Murphy’s law will always have its say. For once, that may prove to be a blessing. For if AGI ever does come into being, there won’t be any going back.
Not until God renovates the universe with fire, at least. (See 2 Peter 3:10 and Revelation 21:1)
References:
–https://www.newsweek.com/map-
–https://en.wikipedia.org/
–https://finance.yahoo.com/
–https://www.reddit.com/r/
Writing update: I’m almost done with the first draft of Emolecipation episode 3. I have only a chapter or two left, then I’ll start mapping out episode 4.
Dylan West
