In the real world, token costs seem to be going up, as early stage pricing at a loss gives way to pricing that generates revenue.
Compute costs might go down a little over the next five years, but there's nothing coming along in hardware that leads to huge reductions in price. NVidia says don't expect better price/performance before 2030.
The models keep getting bigger, and people put loops around them which iterate, burning tokens.
Where is this cost reduction coming from?
I suspect it might be a question of conversational loop vs agentic dev - the former uses much less tokens than letting an autonomous agent churn away on your codebase.
Edit: A glaring omission on my part there is that growth of aggregate industry demand for tokens has the potential to outpace increases in supply provided by new datacenters buildouts. So tokens certainly could go up depending on how things play out.
You are right - tokens are going up currently.
From last month: https://peinsights.substack.com/p/apollo-and-blackstone-clos...
The fact that we aren't seeing an app explosion (I think) is evidence that building applications people will pay for is significantly more complex than just prompting claude/codex/etc
I talked with a friend last week, who has never coded before in his life, who built an absolutely incredible fit-for-purpose app for his own job. He gave me a demo and it blew my mind. It will never go beyond his walls, and he will never buy SaaS that only kinda fits what he needs.
I see things like this happening. The proliferation isn't public because why sell it? Just build the thing to make your domain job easier and save thousands per month cancelling SaaS subs.
The ROI of AI is starting to show, but it isn't in terms of growth or selling new things - it's reducing spend across the board on software and tools.
I also have repeatedly experienced the phenomenon of nontechnical people having built custom software to run their businesses. A lawyer friend was first, sending me a link to his GitHub(!), where he has built a custom client intake/practice-management application to work as the firm works. He's not the only non-technical lawyer I know who has shared vibe coded apps with me.
I personally build many, many single-use apps than I ever would have before. Gnarly debugging sessions can be greatly simplified by inserting a custom piece of disposable tooling/etc. I am not a Mac programmer, but I now have custom Mac apps to solve problems that only I want solved. Do these count?
Honestly, I would be a little surprised if anyone posting on HN did not have some personal exposure to the explosion of apps.
I personally have been building a bunch of little personal apps for my home that aren't worth the effort of sharing - like a customized dashboard of the Trimet buses closest to my house. The cost to build the initial good-enough version was literally 5 minutes plus another 10 to test and deploy.
A good friend of mine helped his mom keep track of Meals On Wheels (or a similar volunteer org) orders, deliveries, cancellations, etc. They were managing all of this via paper before.
I compiled a list of online recipes. Then I had an LLM typeset them for me into a printable PDF and build a companion website with links to the original recipes and complete ingredient lists for shipping. had the LLM encode links for the companion site into QR codes so the printed copy of the cookbook would bring me immediately to a shopping list, making trying a new recipe soooo much less daunting.
There are so many little things like this that you can make that just take too much effort to justify otherwise. I have other ideas for personal projects that I'll probably get to some day.
Some of them have half baked financial models, but nobody will invest dollars backing a SaaS offering that could easily be replicated, or that could be made redundant tomorrow.
On a commercial support forum I moderate we had to ban software announcements there were so many.
Hypothetical. Assume you can in fact point agents at a tool and say "replicate it. Make no mistakes". You then have software being instantly copy-able.
Assume these agents can then be pointed to a customer feedback board in perpetuity and they autonomously upgrade the software over time. They analyze usage patterns and behave like PMs figuring out what to prune and what to build. Then the maintenance part of the stack also goes to zero.
Over time, the highest margin competitiveness will go to the distributor of the tokens. Aka the AI model makers.
In a world like that (which the frontier labs claim is within a year or two of happening) it feels like it's only a matter of time before they opt to own the entire stack down to the consumer apps. Kind of like Amazon deciding they want to knock off products doing well and then favour their own product over the original seller.
My guess is that if the capability arrives the only reason the frontier labs don't move to own the entire stack immediately is because of optics. Boil the frog instead.
The distinction is that the games being made are garbage, and I mean worse than shovelware garbage. It's actively made things much harder as someone that fancies himself an indie game curator because you gotta dig through more and more games to find stuff with actual people behind it.
Now, on the first order point, I agree that non-tech companies seem to be taking longer to see results from AI, even if the argument was bad.
I work on SaaS for the logistics space, and I feel like prior to the end of 2025, almost all the discussion about AI for logistics was vaporware, starting this year, companies are actually trying to deploy agents, and we'll start finding out what the ROI is later this year or next.
But then if this happens - all of the stock market has risen in the promise of AI. If AI eats profits instead of grows them, then the economy shrinks right? So maybe that’s worse? That there is no productivity increase?
And I don't think this is unusual. It took decades for previous technologies to be fully integrated into existing businesses. In the 80s you could see the IT revolution everywhere... except the productivity statistics, which didn't catch up until the 90s.
LLMs are still very new and have significant limitations (like prompt injection and high token costs) that are very likely solveable but will take time.
Also, adoption isn’t lacking because of lack of awareness. Adoption isn’t happening because the math doesn’t add up and the ROI isn’t there. Consulting pixie dust can’t fix that.
"The first chart below shows that so far there are no signs of profit margins rising outside the tech sector. This is ultimately what we are waiting for, because the value of AI companies today rests entirely on the promise that margins in the S&P 493 will eventually climb."
This is absolutely not necessary. The bull case is that AI will bring great efficiencies. The surplus profits from those efficiencies could easily be competed away by firms who have adopted AI. Those firms who do not adopt AI will have their margis crushed.
… usurped by the tech companies?
Thats at odds with current inflation trends to say the least.
If your employees can suddenly magically do more work with the same pay, that's free money (for you). You can pay fewer employees, or pay them less by threatening to replace them with the magic robot.
The magical thinking version of this is that your productivity gains magically translate into more customers and more sales for the same input cost and labor. The free money is really free because you're a magical special snowflake company and every consumer will want your brand of magic machine outputs and not the other guy's. Where does all this money come from? Do those extra customers even exist? Who cares!
Pepsi starts using AI in some magical way that allows them to increase their margins. This allows them to reduce prices while increasing profits. Price-sensitive customers switch from Coca Cola products to Pepsi products. Coca Cola loses some market share, reducing economies of scale, and reducing margins, thus reducing profits. As the cycle repeats, Pepsi moves to dominate the market, and Coca Cola is slowly squeezed down.
Pepsi starts collecting the extra profits with zero price reductions.
The market has clearly spoken. Knowing what you're doing is much more valuable than just the doing. That still requires humans. This AI winter has already begun.
Well that's just wrong. Reasoning models are new and very powerful. LLMs can complete open-ended tasks that require many complex steps.
We're just beginning. The bubble will pop and investors will lose a lot of money, but we're not going back into a winter. It actually works this time.