I assume until LLMs are 100% better than humans in all cases, as long as I have to be in the loop there will be a pretty hard upper bound on what I can do and it seems like we’ve roughly hit that limit.
Funny enough, I get this feeling with a lot of modern technology. iPhones, all the modern messaging apps, etc make it much too easy to fragment your attention across a million different things. It’s draining. Much more draining than the old days
I do as well, so totally know what you're talking about. There's part of me that thinks it will become less exhausting with time and practice.
In high school and college I worked at this Italian place that did dine in, togo, and delivery orders. I got hired as a delivery driver and loved it. A couple years in there was a spell where they had really high turnover so the owners asked me to be a waiter for a little while. The first couple months I found the small talk and the need to always be "on" absolutely exhausting, but overtime I found my routine and it became less exhausting. I definitely loved being a delivery driver far more, but eventually I did hit a point where I didn't feel completely drained after every shift of waiting tables.
I can't help but think coding with LLMs will follow a similar pattern. I don't think I'll ever like it more than writing the code myself, but I have to believe at some point I'll have done it enough that it doesn't feel completely draining.
With the rise of open source, there started to be more black-box compositing, you grabbed some big libraries like Django or NumPy and honestly just hoped there weren't any bugs, but if there were, you could plausibly step through the debugger and figure out what was going wrong and file a bug report.
Now, the LLMs are generating so many orders of magnitude more code than any human could ever have the chance to debug, you're basically just firing this stuff out like a firehose on a house fire, giving it as much control as you can muster but really just trusting the raw power of the thing to get the job done. And, bafflingly, it works pretty well, except in those cases where it doesn't, so you can't stop using the tool but you can't really ever get comfortable with it either.
so as a human, you would make the judgement that the cases where it works well enough is more than make up for the mistakes. Comfort is a mental state, and can be easily defeated by separating your own identity and ego with the output you create.
The code part is trivial and a waste of time in some ways compared to time spent making decisions about what to build. And sometimes even a procrastination to avoid thinking about what to build, like how people who polish their game engine (easy) to avoid putting in the work to plan a fun game (hard).
The more clarity you have about what you’re building, then the larger blocks of work you can delegate / outsource.
So I think one overwhelming part of LLMs is that you don’t get the downtime of working on implementation since that’s now trivial; you are stuck doing the hard part of steering and planning. But that’s also a good thing.
I've written it up here, including the transcript of an actual real session:
https://www.stavros.io/posts/how-i-write-software-with-llms/
I just woke up recently myself and found out these tools were actually becoming really, really good. I use a similar prompt system, but not as much focus on review - I've found the review bots to be really good already but it is more efficient to work locally.
One question I have since you mention using lots of different models - is do you ever have to tweak prompts for a specific model, or are these things pretty universal?
And when you make the decisions it is you who is responsible for them. Whereas if you just do the coding the decisions about the code are left largely to you nobody much sees them, only how they affect the outcome. Whereas now the LLM is in that role, responsible only for what the code does not how it does it.
LLMs will do pretty much exactly what you tell them, and if you don't tell them something they'll make up something based on what they've been trained to do. If you have rules for what good code looks like, and those are a higher bar than 'just what's in the training data' then you need to build a clear context and write an unambiguous prompt that gets you what you want. That's a lot of work once to build a good agent or skill, but then the output will be much better.
The result is that I could say that it was code that I myself approved of. I can't imagine a time when I wouldn't read all of it, when you just let them go the results are so awful. If you're letting them go and reviewing at the end, like a post-programming review phase, I don't even know if that's a skill that can be mastered while the LLMs are still this bad. Can you really master Where's Waldo? Everything's a mess, but you're just looking for the part of the mess that has the bug?
I'm not reviewing after I ask it to write some entire thing. I'm getting it to accomplish a minimal function, then layering features on top. If I don't understand where something is happening, or I see it's happening in too many places, I have to read the code in order to tell it how to refactor the code. I might have to write stubs in order to show it what I want to happen. The reading happens as the programming is happening.
For context, I started an experiment to rebuild a previous project entirely with LLMs back in June '25 ("fully vibecoded" - not even reading the source).
After iterating and finally settling on a design/plan/debug loop that works relatively well, I'm now experiencing an old problem like new: doing too much!
As a junior engineer, it's common to underestimate the scope of some task, and to pile on extra features/edge cases/etc. until you miss your deadline. A valuable lesson any new programmer/software engineer necessarily goes though.
With "agentic engineering," it's like I'm right back at square one. Code is so cheap/fast to write, I find myself doing it the "right way" from the get go, adding more features even though I know I shouldn't, and ballooning projects until they reach a state of never launching.
I feel like a kid again (:
If I give it anything resembling anything that I'm not an expert on, it will make a mess of things.
Is that why it's in quotes because it's the opposite of the right way?
If there's one thing I learned in a decade+ of professional programming, it's that we can't predict the future. That's it, that simple. YANGNI. (also: model the data, but I'm trying to make a point here)
We got into coding because we like to code; we invent reasons and justifications to code more, ship more, all the world's problems can be solved if only developers shipped more code.
Nirvana is reached when they that love and care about the shipping of the code know also that it's not the shipping of the code that matters.
The most important thing is shipping/getting feedback, everything else is theatre at best, or a project-killing distraction at worst.
As a concrete example, I wanted to update my personal website to show some of these fully-vibecoded projects off. That seemed too simple, so instead I created a Rotten Tomatoes-inspired web app where I could list the projects. Cool, should be an afternoon or two.
A few yak shaves later, and I'm adding automatic repo import[0] from Github...
Totally unnecessary, because I don't actually expect anyone to use the site other than me!
I JUST WANT TO CODE!
It gets us all. And it makes us better I think, to care about the craft. LLM people seem split on that. But it's both to me: gotta care about the craft, also as a professional, it's not the code, it's business outcomes. All good. hold two truths.
I can finally do my preferred workflow: Research, (design, critique), (plan, critique, design), implement.
Design and planning has a quick enough turnaround cycle to not get annoying. By the time the agent is writing code, I have no involvement anymore. Just set it and forget it, come back in half an hour or so to see if it's done yet. Meanwhile, I look at the bigger picture and plan out my next prompt cycles as it churns out code.
For example, this project was entirely written by LLM:
https://github.com/kstenerud/yoloai
I never wrote a single line of this code (I do review it, of course, but even then the heavy lifting for that can be offloaded to an LLM so that I can focus on wider issues, which most often are architectural).
In particular, take a look at the docs/dev subdir to see the planning and design. Once the agent has that, it's MUCH harder for it to screw things up.
Is it as tight as it could be? Nope, but it has a solid architecture, does its job well, and has good debugging infrastructure so fixes are fast. I wouldn't use this approach for embedded or projects requiring maximum performance, but for regular code it's great!
But I absolutely loathe reviewing these generated PRs - more so when I know the submitter themselves has barely looked at the code. Now corporate has mandated AI usage and is asking people to do 10k LOC PRs every day. Reviewing this junk has become exhausting.
I don’t want to read your code if you haven’t bothered to read it yourselves. My stance is: reviewing this junk is far more exhausting. Coding is actually the fun part.
That's a big red flag if I ever saw one. Corporate should be empowering the engineering team to use AI tooling to improve their own process organically. Is this true or exaggeration? If it's true I'd start looking for a more balanced position at more disciplined org.
I’ve certainly seen my share of what I call slot driven development where a developer just throws things at the wall until something mostly works. And plenty if cut and paste development.
But it’s far from the majority. It’s usually the same few developers at a company doing it, while the people who know what they’re doing furiously work to keep things from falling apart.
If the majority of devs were doing this nothing would work. My worry is that AI lets the bad devs produce this kind of work on a massive scale that overwhelms the good devs ability to fight back or to even comprehend the system.
The resilience of the system has taken a massive hit, and we were told that it doesn’t matter. Managers, designers, and product folks are being asked to make PRs. When things cause Sev0 or Sev1 incidents, engineers are being held responsible. It’s a huge clown show.
"Look, if the AI fairy worked like that our company would be me and the investors."
I should make t-shirts. They'll be worth a fortune in ironic street cred once the AI fairy works like that.
Coldtea's law: "Never attribute to context rot that which is adequately explained by cost-cutting".
You might think that the "constant" task switching is draining, but I don't switch that frequently. Often I keep the main focus on one task and use the waiting time to draft some related ideas/thoughts/next prompt. Or browse through the code for light review/understanding. It also helps to have one big/complex task and a few simpler things concurrently. And since the number of details required to keep "loaded" in your head per task is fewer, switching has less cost I think. You can also "reload" much quicker by simply chatting with the agent for a minute or two, if some detail have faded.
I think a key thing is to NOT chase after keeping the agents running at max efficiency. It's ok to let them be idle while you finish up what your doing. (perhaps bad of KV cache efficiency though - I'm not sure how long they keep the cache)
(And obviously you should run the agent in a sandbox to limit how many approvals you need to consider)
[1] I use the urgent-window hint to get a subtle hint of which workspace contain an agent ready for input.
EDIT: disclaimer - I'm relative new to using them, and have so far not used them for super complex tasks.
As somebody who has been coding for just shy of 40 years and has gone through the actual pain on learning to run a high level and productive dev team, your experience does not match mine. Even great devs will forget some of the basics and make mistakes and I wish every junior (hell even seniors) were as effective as the LLMs are turning out to be. Put the LLM in the hands of a seasoned engineer who also has the skills to manage projects and mentor junior devs and you have a powerful accelerator. I'm seeing the outcome of that every day on my team. The velocity is up AND the quality is up.
This is not my experience on a team of experienced SWEs working on a product worth 100m/year.
Agents are a great search engine for a codebase and really nice for debugging but anytime we have it write feature code it makes too many mistakes. We end up spending more time tuning the process than it takes to just write the code AND you are trading human context with agent context that gets wiped.
We've spent years reducing old debt and modernizing our application and processes. The places where we've made that investment are where we are currently seeing the additional acceleration. The places where we haven't are still stuck in the mud, but per your "search engine for a codebase" comment our engineers are starting to engage with systems they would not have previously touched.
There are areas for sure where LLMs would fall down. That's where we need the experts to guide them and restructure the project so that it is LLM friendly (which also just happens to be the same things that make the app better for human engineers).
And I'm serious about the quality comment. Maybe there's a difference in how your team is using the tools, but I have individuals on my team who are learning to leverage the tools to create better outputs, not just pump out features faster.
I'm not saying LLMs solve everything, FAR from it. But it's giving a master weapon to an experienced warrior.
You said that you're restructuring the project to be LLM friendly, which also makes the app better for humans. I 100% agree with this. Code that is unreadable and unmaintainable for humans is much more difficult for AI to understand. I think companies that practiced or prioritized code hygiene will be ahead of the game when it comes to getting good results with agentic AI.
I would be curious to see if I'm just imaging this or it really is a trend.
The cheaper models can't be taught or improved due to their inherit limitations, which makes it a huge pain to even try with even the simplest of tasks. Perpetually, no matter your instruction file(s).
Do you mean that? It's clearly false, but I don't want to waste time gathering famous-person counterexamples if you already know it's a huge exaggeration at best.
I find LLMs are great for building ideas, improving understanding and basic prototyping. This is more useful at the start of the project lifecycle, however when getting toward release it's much more about refactoring and dealing with large numbers of files and resources, making very specific changes e.g. from user feedback.
For those of us with decades of muscle memory who can fix a bug in 30 seconds with a few Vim commands, LLMs are very likely to be slower in most coding tasks, excepting prototyping and obscure bug spotting.
Maybe the right answer is to sometimes slow down, explore and think a little more instead of just letting it try something until it (eventually, sort of) works.
Another trick I learnt is you can ask Claude to ask you comprehensive questions for clarification. Usually, it will then offer you a choice of 3 options per question that it might have and you can steer it towards the right implementation.
You know you can leave abusive relationships. Ditch the clanker and free your mind.
It's like with regular non-llm assisted coding. Sometimes you gotta sleep on it and make a new /plan with a fresh direction.
I mostly use YOLO mode which means I'm not constantly watching them and approving things they want to do... but also means I'm much more likely to have 2-3 agent sessions running in parallel, resulting in constant switching which is very mentally taxing.
What impacts cognition for me, and IMO for a lot of folks, is how well we end up defining our outcomes. Agents are tremendous at working towards the outcome (hence by TDD red-green works wonderfully), but if you point them to a goal slightly off, then you'll have to do the work of getting them on track, demanding cognition.
So the better you're at your initial research/plan phase, where you document all of your direction and constraints, the lesser effort is needed in the review.
The other thing impacting cognition is how many parallel threads you're running. I have defaulted to major/minor system - at any time I have 1 major project (higher cognition) and 1 minor agent (lower cognition) going. It's where managing this is comfortable.
Many top labs [1] [2] already have heavily automated code review already and it's not slowing down. That doesn't mean I'm trusting everything blindly, but yes, over time, it should handle less and less "lower level" tasks and it's a good thing if it can.
[1] https://openai.com/index/harness-engineering/ [2] https://claude.com/blog/code-review
Further I want to vent about two things:
- Things can be improved.
- You are allowed to complain about anything, while not improving things yourself.
I think the mid 2010s really popularized self improvement in a way that you can't really argue with (if you disagree with "put in more effort and be more focused", you're obviously just lazy!). It's funny because the point of engineering is to find better solutions, but technically yes, an always valid solution is just "suck it up".
But moreover, if you do not allow these two premises, what ends up happening in practice for a lot of people, is that basically you can just interpret any slightly pushback as "oh they're just a whiner", and if they're not doing something to fix their problem this instant, that "obviously" validates your claim (and even if they are, it doesn't count, they should still not be a "debbie downer", etc.).
Sometimes a premise can sound extreme, but people forget that premises are not in a complete logical vaccuum, you actually live out and believe said premises, and by taking on a certain position, it's often more about what follows downstream from the behavior than the actual words themselves.
My limits are now many of the same things that are have always been core to software dev, but are now even more obvious:
- what is the thing we are building? What is the core product or bug fix or feature?
- what are we _not_ building? What do we not care about?
- do I understand the code enough to guide design and architecture?
- can I guide dev and make good choices when it’s far outside my expertise but I know enough to “smell” when things are going off the rails
It’s a weird time
I think the exhausting part is more probably more tied to the evaluation of the work the agent is doing, understanding its thought process and catching the hang up can be tedious in the current state of AI reasoning.
same thing happened with crypto - the underlying technology is cool but the community is what makes it so hated
Working with an agent coding all day can be exhilarating but also exhausting - maybe it’s because consequential decisions are packed more tightly together. And yes cognition still matters for now.
so the bottleneck shifts. before: generating code is slow, integration is easy (you built it). after: generating code is instant, integration requires the same mental load as before because the codebase complexity didn't decrease -- it just grew faster.
I learned years ago that I when I write code after 10 PM, I'm go backward instead of forward. It was easy to see, because the test just wouldn't pass, or I'd introduce several bugs that each took 30 minutes to fix.
I'm learning now that it's no different, working with agents.
I imagine I will greatly reduce my job prospects as a hold out, but honestly, from what I've read I think I'd rather take a hefty pay hit and not go there. It sounds like a mental heath disaster and fast track to serious burnout.
YMMV, I realize I'm in the minority, this is unproductive ranting, yada yada yada
There's probably a Codex equivalent, but I don't know what it is.
Its amazing how right and wrong LLMs can be in the output produced. Personally the variance for me is too much... I cant stand when it gets things wrong on the most basic of stuff. I much prefer doing things without output from an LLM.
Another way you can read this is as a new cult member that his chiding himself whenever he might have an intrusive thought that Dear Leader may not be perfect, after all.
My pet theory is we haven't figured out what the best way to use these tools are, or even seen all the options yet. But that's a bigger topic for another day.
The only time I've felt something akin to this with a compiler is when I was learning Rust. But that went away after a week or two.