I’ve done some reading on the topic and am just not getting it.
The benefits of doing your own harness here is that you get to explicitly program around those specific things to optimize cost. And they both heavily benefit you in the way that these sorts of jobs work - at least for the hunt side of things.
TLDR - context management is pretty important for cost management when you're doing something repetitive in bulk.
I've toying with pi.dev to do something like that.
It's nearly as if, in the end, we'll still be coding after all.
That's ridiculous.
And yet, presumably, the vulns are real.
What are businesses supposed to actually do with this?
What happens to the whole AI value proposition when instead of it being a way to pump out lines of code for crazy cheap it becomes a way for each line of code to become vastly more expensive than it was before?
I can't help but notice that in the section headed "What it costs" the $ symbol is conspicuously absent. I would definitely like to hear a much more concrete number on some sort of per-100k-line basis or something. I know that per-line isn't great but at least it would be something.
Maybe if the models get a lot better we wouldn't need all this cross-checking. And maybe they'd write fewer vulnerabilities for the other models to laboriously and expensively figure out in the first place.
But good gracious does this sound like the AI industry just asking for you to hand the a blank check, because it sure would be a shame if something happened to your code base, wouldn't it?
I don't have a solution to this. This is stricly a cri de coeur.
Except maybe to say that if this is going to be the way in the future that it becomes vastly, vastly more important to work out how to write more secure code from the very beginning. We've certainly been trending this way for the last few decades, hand-wringing aside security has gotten a lot better than it used to be, it's just the world has gotten more complicated and harsh as well so it doesn't always look like it. But it takes over a decade for things like "use parameters in your SQL query instead of concatenation" to go from some crazy guy's idea in some obscure open source package somewhere to common practice that can almost be counted on. That loop is going to have to close much faster and there's a lot of things that are barely registering on people's radars, like, the *at APIs for files (openat, etc.) need to be the default much, much sooner than their current trajectory has them on.
I would consider that a good thing. Businesses should not able to make money by writing terrible, insecure code.
Anybody who prevents them from doing so surely deserves to capture the additional value created.
A responsible business should increase an attacker's cost above the likely benefit. If your current threat model accepts the risk of a particular attack because "it would be too costly" and this model changes that, then you need to consider mitigating rather than accepting that risk.