Red/Green TDD
1 points
2 hours ago
| 1 comment
| simonwillison.net
| HN
tomwphillips
2 hours ago
[-]
I agree it works well. Although as a long-time TDD practitioner it is mildly frustrating that it has taken LLMs to get more people to realise it works!
reply
eesmith
14 minutes ago
[-]
I've always found TDD frustrating, especially in its Red/Green expression.

I find that my tests become too low-level, as I build up component by component. This hinders large-scale refactorings because my mental planning wants to avoid the extra effort of rewriting the tests to any new interface.

That refactoring can also some of the tests unnecessary, so it felt like I was going through extra work for a small benefit which wasn't worthwhile at that stage.

I also found that red/green TDD's focus on "confirm that the tests fail before implementing the code to make them pass" (quoting the link) makes me think less about writing tests which aren't expected to fail, but if they do fail they indicate serious design problems.

As an example, I once evaluated software package which was fully developed under TDD. It was a web app which, among other things, allowed anonymous and arbitrary users to download files with a URL like example.com/download?filename=xyz.txt

There were no tests for arbitrary path traversal, and when I tried it out with something like filename=../../config.ini I got access to the server's config file.

Now, it wasn't quite that bad. They required the filename end in only a handful of extensions, like ".pdf". Thing is, the developers didn't check for NUL characters, and their server was written in Java, which passes the string directly to the filesystem API, which in true C style expects NUL-terminated strings. My actual filename was more like "../../config.ini\0.pdf", with the NUL appropriately encoded as a URL path parameter. The Java code checked that the extension was allowed, then passed it to the filesystem call, which interpreted it as "../../config.ini", which gave me access to the system configuration - including poorly hashed admin passwords with almost no preimage resistance that I was able to break after a couple of hours of thinking about the algorithm.

The explicit NUL test is needed as a security test in Java. In Python it's a different class of error as Python's filesystem APIs raise a ValueError if the string contains a NUL.

I don't at all mean that good and useful software can't be written with TDD, nor that TDD is useless. Rather, it's that Red/Green TDD as a development practice appears to de-emphasize certain types of essential testing which don't fit into the red-green-refactor paradigm, but instead require a larger development methodology outside of TDD.

As for me personally, I'm strongly influenced by rapid prototyping - "spike and stabilize" I believe it's called - where the code goes through possibly several iterations for the API and implementation to stabilize to the point where the overhead of writing automated tests is outweighed by their benefit.

And these tests include tests which should pass, but which check boundary conditions, unexpected input, and the like.

To say nothing of choosing the right way to hash passwords, which doesn't easily fit into any test-based framework. :)

reply