Significant Raise of Reports
77 points
4 hours ago
| 10 comments
| lwn.net
| HN
0x3f
15 minutes ago
[-]
Why don't we just pagerank github contributors? Merged PRs approved by other quality contributors improves rank. New PRs tagged by a bot with the rank of the submitter. Add more scoring features (account age? employer?) as desired.
reply
SoftTalker
4 minutes ago
[-]
It will be gamed, just as pagerank was.
reply
cafebabbe
1 minute ago
[-]
Not by everyone, so that would be better than nothing.
reply
glimshe
2 hours ago
[-]
The last paragraph is interesting: "Overall I think we're going to see a much higher quality of software, ironically around the same level than before 2000 when the net became usable by everyone to download fixes. When the software had to be pressed to CDs or written to millions of floppies, it had to survive an amazing quantity of tests that are mostly neglected nowadays since updates are easy to distribute."

Was software made before 2000 better? And, if so, was it because of better testing or lower complexity?

reply
bombcar
4 minutes ago
[-]
It was the best of times, it was the worst of times.

Best/better because yes, QA actually existed and was important for many companies - QA could "stop ship" before the final master was pressed if they found something (hehe as it was usually games) "game breaking". If you search around on folklore or other historical sites you can find examples of this - and programmers working all night with the shipping manager hovering over them ready to grab the disk/disc and run to the warehouse.

HOWEVER, updates did exist - both because of bugs and features, and because programmers weren't perfect (or weren't spending space-shuttle levels of effort making "perfect code" - and even voyager can get updates iirc). Look at DooM for an example - released on BBS and there are various versions even then, and that's 1994 or so?

But it was the "worst" in that the frameworks and code were simply not as advanced as today - you had to know quite a bit about how everything worked, even as a simple CRUD developer. Lots of protections we take for granted (even in "lower level" languages like C) simply didn't exist. Security issues abounded, but people didn't care much because everything was local (who cares if you can r00t your own box) - and 2000 was where the Internet was really starting to take off and everything was beginning to be "online" and so issues were being found left and right.

reply
LatencyKills
1 hour ago
[-]
I was a developer at Microsoft in the 90s (Visual Studio (Boston) and Windows teams). I won't claim that software back then was "better," but what is definitely true is that we had to think about everything at a much lower level.

For example, you had to know which Win32 functions caused ring-3 -> ring-0 transitions because those transitions could be incredibly costly. You couldn't just "find the right function" and move on. You had to find the right function that wouldn't bring your app (and entire system) to its knees.

I specifically remember hating my life whenever we ran into a KiUserExceptionDispatcher [0] issue, because even something as simple as an exception could kill your app's performance.

Additionally, we didn't get to just patch flaws as they arose. We either had to send out patches on floppy disks, post them to BBSs, or even send them to PC Magazine.

[0]: https://doar-e.github.io/blog/2013/10/12/having-a-look-at-th...

reply
adrian_b
9 minutes ago
[-]
It is hard to say which of the 2 is the reason, more likely both, i.e. lower complexity enabled more exhaustive testing.

In any case some of the software from before 2000 was definitely better than today, i.e. it behaved like being absolutely foolproof, i.e. nothing that you could do could cause any crash or corrupted data or any other kind of unpredictable behavior.

However, the computers to which most people had access at that time had only single-threaded CPUs. Even if you used a preemptive multitasking operating system and a heavily multi-threaded application, executing it on a single-threaded CPU was unlikely to expose subtle bugs due to race conditions, that might have been exposed on a multi-core CPU.

While nowadays there exists no standard operating system that I fully trust to never fail in any circumstance, unlike before 2003, I wonder whether this is caused by a better quality of the older programs or by the fact that it is much harder to implement software concurrency correctly on systems with hardware parallelism.

reply
psyklic
1 hour ago
[-]
> Was software made before 2000 better?

At the time of release, yes. They had to ensure the software worked before printing CDs and floppies. Nowadays they release buggy versions that users essentially test for them.

reply
dspillett
1 hour ago
[-]
Also in terms of security, there was generally a much smaller potential attack surface and those surfaces were harder to reach because we were much less constantly connected.
reply
tptacek
13 minutes ago
[-]
Software has gotten drastically more secure than it was in 2000. It's hard to comprehend how bad the security picture was in 2000. This very much, extremely includes Linux.
reply
PaulHoule
22 minutes ago
[-]
But there was much less awareness of buffer overflows and none of the countermeasures that are widespread today. It was almost defining of the Win95 era that applications (eg. Word) frequently crashed because of improper and unsafe memory management.
reply
armchairhacker
18 minutes ago
[-]
I remember opening a webpage and being hacked seemed more likely. Adobe Flash and Java had more vulnerabilities and weaker (if any) sandboxes than JavaScript.
reply
ModernMech
6 minutes ago
[-]
Depends what you mean by better. It crashed more and there was a lot of data loss, but it wasn't explicitly evil so maybe on measure it was better.
reply
Xenoamorphous
55 minutes ago
[-]
Just think of 8 and 16 bit video console games. Those cartridges were expensive so just how sure they had to be they were bug free before making millions of them?
reply
philipstorry
20 minutes ago
[-]
Yes and no.

Yes. The incentives for writing reliable, robust code were much higher. The internet existed so you could, in theory, get a patch out for people to download - but a sizeable part of any user base might have limited access, so would require something physical shipped to them (a floppy or CD). Making sure that your code worked and worked well at time of shipping was important. Large corporate customers were not going to appreciate having to distribute an update across their tens of thousands of machines,

No. The world wasn't as connected as it is today, which meant that the attack surface to reasonably consider was much smaller. A lot of the issues that we had back then were due to designs and implementations that assumed a closed system overall - but often allowed very open interoperability between components (programs or machines) within the system. For example, Outlook was automatable, so that it could be part of larger systems and send mail in an automated way. This makes sense within an individual organisation's "system", but isn't wise at a global level. Email worms ran rampant until Microsoft was forced to reduce that functionality via patches, which were costly for their customers to apply. It damaged their reputation considerably.

An extreme version of this was openness was SQL Slammer - a worm which attacked SQL Servers and development machines. Imagine that - enough organisations had their SQL Servers or developer machines directly accessible that an actual worm could thrive on a relational database system. Which is mindboggling to think about these days, but it really happened - see https://en.wikipedia.org/wiki/SQL_Slammer for details.

I wouldn't say that the evidence points to software being better in the way that we would think of "better" today. I'd say that the environment it had to exist in was simpler, and that the costs of shipping & updating were higher - so it made more sense to spend time creating robust software. Also nobody was thinking about the possible misuse or abuse of their software except in very limited ways. These days we have to protect against much more ingenious use & abuse of programs.

Furthermore today patching is quick and easy (by historical comparison), and a company might even be offering its own hosted solution, which makes the cost of patching very low for them. In such an environment it can seem more reasonable to focus on shipping features quickly over shipping robust code slowly. I'd argue that's a mistake, but a lot of software development managers disagree with me, and their pay packet often depends on that view, so they're not going to change their minds any time soon.

In a way this is best viewed as the third age of computing. The first was the mainframe age - centralised computer usage, with controlled access and oversight, so mistakes were costly but could be quickly recovered from. The second was the desktop PC age - distributed computer usage, with less access control, so mistakes were often less costly but recovering from them was potentially very expensive. The third is the cloud & device age, with a mix of centralised and distributed computer use, a mix of access control, and potentially much lower costs of recovery. In this third age if you make the wrong decisions on what to prioritise (robustness vs speed of shipping), it can be the worst of both the previous ages. But it doesn't have to be.

I hope that makes sense, and is a useful perspective for you.

reply
1970-01-01
48 minutes ago
[-]
It was a simpler time. Not better. Not worse. Programs still had bugs, but they weren't sloppy UI bugs, they were logic bugs and memory leaks. If software was better back then, we'd still be using it!
reply
empath75
1 hour ago
[-]
> Was software made before 2000 better?

Literally the moment everyone got on the internet, pretty much every computer program and operating system in the world was besieged by viruses and security flaws, so no.

reply
hsbauauvhabzb
21 minutes ago
[-]
I think there was a period where things got better but I don’t think it was pre-internet.

There was a point in time where both windows wasn’t constantly bsoding and Microsoft’s primary objectives weren't telemetry and slop coding.

reply
Shank
1 hour ago
[-]
Important to note that this is a comment on this article: https://lwn.net/Articles/1065586/.
reply
JumpCrisscross
39 minutes ago
[-]
“Reversing was already mostly a speed-bump even for entry-level teams, who lift binaries into IR or decompile them all the way back to source. Agents can do this too, but they can also reason directly from assembly. If you want a problem better suited to LLMs than bug hunting, program translation is a good place to start.”

Huh. Direct debugging, in assembly. At that point, why not jump down to machine code?

reply
bombcar
3 minutes ago
[-]
Decompiled assembly is basically machine code; without recreating the macros that make assembly "high level" you're as close to machine code as you're going to get unless you're trying to exploit the CPU itself.
reply
HAMSHAMA
1 hour ago
[-]
Probably related to this (genuinely interesting) talk given by an entropic researcher https://youtu.be/1sd26pWhfmg?si=j2AWyCfbNbOxU4MF
reply
adverbly
1 hour ago
[-]
Anecdotally, I've been seeing a higher rate of CVEs tracked by a few dependabot projects.

Seems supported by this as well: https://www.first.org/blog/20260211-vulnerability-forecast-2...

Interesting that it's been higher than forecast since 2023. Personally I'd expect that trend to continue given that LLMs both increase bugs written as well as bugs discovered.

reply
nayroclade
2 hours ago
[-]
> I don't know how long this pace will last. I suspect that bugs are reported faster than they are written, so we could in fact be purging a long backlog

Hopefully these same tools will also help catch security bugs at the point they're written. Maybe one day we'll reach a point where the discovery of new, live vulnerabilities is extremely rare?

reply
Sharlin
1 hour ago
[-]
Around 70% of security vulnerabilities are about memory safety and only exist because software is written in C and C++. Because most vulnerabilities are in newly written code, Google has found that simply starting writing new code in Rust (rather than trying to rewrite existing codebases) quickly brings the number of found vulnerabilities down drastically.
reply
noosphr
34 minutes ago
[-]
And to a good approximation all real world Rust uses unsafe everywhere.

So we now have a new code base in an undefined language which still has memory bugs.

This is progress.

reply
panstromek
5 minutes ago
[-]
The parent comments references real world data from Google: https://security.googleblog.com/2024/09/eliminating-memory-s...
reply
kibwen
18 minutes ago
[-]
No, this is false. For Rust codebases that aren't doing high-peformance data structures, C interop, or bare-metal stuff, it's typical to write no unsafe code at all. I'm not sure who told you otherwise, but they have no idea what they're talking about.
reply
siruwastaken
1 hour ago
[-]
It's interesting to hear from people directly in the thick of it that these bug reports are apparently gaining value and are no longer just slop. Maybe there is hope for a world where AI helps create bug free software and doesn't just overload maintainers.
reply
themafia
2 hours ago
[-]
An AI enthusiast having a breathless and predictive position on the future of the technology? No way! It's almost like Wall Street is about to sour on the whole stack and there is a concerted effort to artificially push these views into the conversation to get people on board.

Then again, I'm a known crank and aggressive cynic, but you never really see any gathered data backing these points up.

reply
dieulot
1 hour ago
[-]
Could you back up your assertion that Willy Tarreau — who used to maintain the Linux kernel — is “an AI enthusiast”? I can’t find anything about it.
reply
cdavid
33 minutes ago
[-]
Also one of the initial creator of haproxy, a well known reverse proxy. To imply somebody like as a simple "AI shill" is just ignorant.
reply
logicprog
45 minutes ago
[-]
Anyone who says anything good about AI must be an AI shill from the start, not someone who is genuinely observing reality or had their mind changed, don't you know?
reply
logicprog
45 minutes ago
[-]
> but you never really see any gathered data backing these points up.

https://www.anthropic.com/news/mozilla-firefox-security

?

reply
PKop
28 minutes ago
[-]
Sort of a tautology to just assert that someone saying good things about AI is an AI enthusiast and therefore their opinion should be dismissed. He also happens to have been a kernel maintainer, his experience as he's describing it should count for something.
reply
stratos123
4 hours ago
[-]
"On the kernel security list we've seen a huge bump of reports. We were between 2 and 3 per week maybe two years ago, then reached probably 10 a week over the last year with the only difference being only AI slop, and now since the beginning of the year we're around 5-10 per day depending on the days (fridays and tuesdays seem the worst). Now most of these reports are correct, to the point that we had to bring in more maintainers to help us."
reply
sevg
1 hour ago
[-]
Is there a reason you’ve copy pasted the first paragraph from the link? It doesn’t add anything to the discussion, and also doesn’t help as a tl;dr because it’s literally the first paragraph. Genuine question!
reply
throwatdem12311
57 minutes ago
[-]
Reports being written faster than bugs being created? Better quality software than before the 2000s?

Oh my sweet summer child.

This is some seriously delusional cope from someone who drank the entire jug of kool-aid.

I’d love to be proven wrong but the current trajectory is pretty plain as day from current outcomes. Everything is getting worse, and everyone is getting overwhelmed and we are under attack even more and the attacks are getting substantially more sophisticated and the blast radius is much bigger.

reply