How SQLite is tested
125 points
2 hours ago
| 9 comments
| sqlite.org
| HN
bastardoperator
1 hour ago
[-]
Over a decade ago, the maintainer of SQLite gave a talk at OSCON about their testing practices. One concept that stood out to me was the power of checklists, the same tool pilots rely on before every flight.

He also mentioned Doctors Without Borders, who weren't seeing the outcomes they expected when it came to saving lives. One surprising reason? The medical teams often didn't speak the same language or even know each other's names.

The solution was simple: a pre-surgery checklist. Before any procedure, team members would state their name and role. This small ritual dramatically improved their success rates, not through better technique, but through better communication.

https://sqlite.org/src/ext/checklist/3070700/index

reply
piker
1 hour ago
[-]
Always makes me a bit envious as well as awestruck. What a joy it must be in a lot of ways to be able to grind and perfect a piece of software like this. Truly a work of craftsmanship.
reply
stouset
11 minutes ago
[-]
You can literally just do this. I’ve never gotten fired from a software engineering job for moving slower and building things that work well, work predictably, and are built to last.

Over a career of doing this, you get dramatically better at higher levels of quality even in earlier passes, so the same level of added effort provides increasing levels of reward as you gain experience.

reply
Seattle3503
1 hour ago
[-]
I love sqlite, it's a great piece of software. The website is full of useful information, rather than the slick marketing we are used to, even on open source projects.

With that said, I find it strange how the official website seems to be making its way through the HN front page piecemeal.

reply
1986
58 minutes ago
[-]
This one is probably popping today because of the simonw post yesterday about using an LLM to basically one-shot port a lib across languages with the help of an extremely robust test suite
reply
realityfactchex
56 minutes ago
[-]
I was pleasantly surprised recently when planning to "upgrade" a light web app to be portable between SQLite and DuckDB, and the LLM I was working with really made the case that SQLite is better if concurrent operations were to occur.
reply
montroser
1 hour ago
[-]
This looks so very cool, and so then all the more thought provoking that the tests themselves are closed-source, unlike the rest of the codebase. In this evolving world of rapidly improving llm coding agent productivity, the idea that the tests are more important than the implementation starts to ring true.

I was thinking about sqlite's test landscape as described here, in relation to simonw's recent writing about porting/recreating the justHTML engine from python to js via codex, nearly "automatically" with just a prompt and light steering.

reply
Lorkki
6 minutes ago
[-]
It's made a lot of sense in general if you think about the business models around open source products. An extensive test suite gives you the ability to engineer changes effectively and efficiently, meaning that you can also add additional value on top of released versions better than everyone else.
reply
BinaryIgor
2 hours ago
[-]
No less impressive than the SQLite project itself; especially 100% branch coverage! That's really hard to pull off and especially to maintain as the development continues.
reply
tigranbs
2 hours ago
[-]
Based on the stability track record, I was more curious about how SQLite has done the anomaly testing. Sadly, the article has just a few words about it.

Truly one of the best software products! It is used on every single device, and it is just pure rock-solid.

reply
6SixTy
1 hour ago
[-]
Considering that the support tier where you get access to the testing suite is 150K/year, I don't think they will be spilling any beans soon.
reply
Trung0246
10 minutes ago
[-]
Interesting, TH3 is proprietary.
reply
dev_l1x_be
23 minutes ago
[-]
What is the story with Fossil? Is it used outside of Sqlite?
reply
forgotpwd16
9 minutes ago
[-]
SQLite & Fossil* were created by same person (once a member of Tcl Core Team). Fossil few years after SQLite (was on CVS before). A rationale is given in: https://sqlite.org/whynotgit.html. The one other big project using it is Tcl/Tk. (Can say Tcl x SQLite x Fossil form a trinity of sorts with one using the others.)

*The homepage is available in: https://fossil-scm.org/home/doc/trunk/www/index.wiki.

reply
gjvc
10 minutes ago
[-]
I can't answer that but it's a great thing that an entire Fossil repo lives in a single sqlite file.
reply