I'm pushing the need for basic engineering principles across whole organisations.

You wouldn't give an engineer 1000 lines of code to review without the original spec of what you're trying to achieve for context (at a minimum, ideally the reviewer was in the room when the work was introduced, and has full context).

So, these docs, they're given as an all or nothing.

Do you push back on the 39th metric that is defined to the utmost detail? Or just resign yourself to the fact that it is what it is?

A one (6 is the goto if we're talking Amazon?!) pager.. "this is what I am proposing" at least gives the skeleton of the idea to push back at the general shape of the idea, refine it, before all the emotional investment of your precious report being complete.

Y'know.. the traditional product running through the spec in a SCRUM* environment.. the engineers doing proper code reviews..

* Yes SCRUM is dead, but that's another thing.

You mean the people they fired and demoralized?

One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts."

Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore.

This is an interesting topic. We treat vetting output the same as doing the work ourselves, but that is not the case.

Doing the work is not the same as reviewing work done by others.

I have heard reports of software engineering companies that have gone full agentic. Their seniors only review stuff written by LLMs and it burns them out, because they have to switch context constantly.

I find this interesting because part of being a senior developer is that you are experienced enough that you won‘t make grave mistakes anymore. This is the case in many professions: you are relied upon to not make grave mistakes.

But those same people are now swamped with stuff that they are not able to review, so they will let a grave mistake slip through at some point.

So they really can‘t trust themselves anymore?

With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.

The problem is that output sometimes take longer to verify than to create in the first place.

That turns AI into a deeply negative ROI system for many applications.

Yeah probably not for the same reason I left VFX rather than have a lifetime of completely disregarding my own generative creativity and cleaning up LLM-generated bullshit. Fuck that. Double-fuck creating ‘content’ to train the models.

In code, LLMs automate away a lot of the drudgery. I wasn’t sad to avoid spending a couple hours looking up the usage patterns and idioms for some ported library, or do some rote task that didn’t make the project significantly better. In most other jobs, they automate away the only fun part and leave humans with all of the drudgery.

The tech industry has always been arrogant to some extent, but assuming the world of talented professional knowledge workers and creatives would be content to professionally proofread, apply lipstick to pigs, and polish turds is a whole new level of out-of-touch. I’d rather live out of my car and dig through the garbage for bottles with deposits.

Why?

I think a lot of the time it's just pure laziness. AI gives people a magical "do all the work for me" button and it can bring out the worst in them.

I’ll have to try again later on desktop. The content looks interesting but it’s literally impossible to read. I cannot get past the section that introduces Ernst and Young.

Some people should not be allowed to make a website.

There may be a lot of demand for do-nothing services.

A lot of corporate work is just do-nothing box-ticking.

Boss: get me a report about X, so I can give that report to my boss who won't read it.

You: E&Y, please get me a report. Here's $200k.

I think their audit work is in a downwards spiral. Audit has become so competitive that they are struggling to find ways to make it cheaper. They have become slaves to reducing the hours booked, and the rate of those hours. To do this they substitute less experienced people all the time. You used to be able to chat with your partner about an issue you have coming up, now you get their assistant if you are lucky. By chasing 'efficiency' they have lost their value-add. Now the first time the partner has looked at your file is right before the clearance meeting, and they spot issues that should have been picked up earlier and tested on the day you should be signing. So you end up doing it all again. I'm trying to coin a term for the inneficiency caused by chasing efficiency.

Yes, this technique and its variations[1][2] "work" but it's still not 100% perfect. And it's not as widely used it might be because, among other reason:

a. it takes longer to implement

b. it costs more (more tokens spread across multiple llm calls)

c. higher latency (getting an answer takes longer due to multiple llm calls involved)

d. the final answer is probabilistically more likely to be correct, but is still not guaranteed to be error free, so you can never fully escape the need for Human in the Loop.

[1]: https://en.wikipedia.org/wiki/LLM-as-a-Judge

[2]: https://github.com/karpathy/llm-council

I think this may be part of the problem. The actual humans creating the report don't have the expertise to know which one to trust. At least that was what consulting was like in my experience at a similar firm.

Even people who like their jobs work because they need money to live.