Show HN: Protoc-gen-dal – Generate data access converters from protobuf
1 points
11 hours ago
| 1 comment
| github.com
| HN
I've been building services where the API layer and database layer inevitably drift apart. Different field names, timestamps as int64 vs time.Time, nested messages becoming JSONB columns - the usual pain. I got a bit sick of hand-writing the same boilerplate converters between my API protobufs and GORM/Datastore entities, so I built this.

Yes - coupling API and DB schemas isn't always best practice. Your API should evolve independently of storage. Leaking internal details to clients is bad. DB optimizations (denormalization, indexes) shouldn't dictate your API shape. But really for a lot of the services Ive bene building - especially at the early stages - there is a lot of "similarity" - 80-90%. Id be doing the same `user.Name = dbUser.Name` lines hundreds of times, and the "clean separation" was mostly ceremony. And the drift just caused more bugs early on - you know forgetting to update a converter when adding a field, subtle type mismatches, copy-paste errors and so on.

This lets me have a single source of truth for my schemas and declare "transformations" as it goes through the layers and the target models/converters are also generated. You could use something like tRPC but then you are stuck with a single language eco system. I really wanted to not have that coupling. Now I can move fast when the schemas are similar, while still allowing divergence when I need it (decorators for custom logic, explicit field mappings).

Currently I generate targets for GORM (Go) and Google Cloud Datastore (so I can spin up small projects with postgres storage or AppEngine storage). Definitely rough around the edges and probably missing features you need, but it's been useful for my own projects. Still lots of things planned (and doing other languages is on the roadmap)

Would love feedback - especially if you've found better patterns for this problem.

flashgordon
11 hours ago
[-]
Author here. For those wondering - Claude Code was used but not in the "generate me this huge X with a single shot prompt after 8 hours of thinking" mode :).
reply