It occurred to me that a durable stream per session would be a much neater architectural foundation for much of what you'd want from such a tool. As a unique feature, it also made live tailing straightforward because the player can read from the same stream the recorder is appending to.
The alternative architecture is likely an ingest firehose which is then indexed, with associated complexity and latency. You'd have to string together multiple data systems like a message queue, a metadata database, and blob storage and/or an OLAP database.
Here the only dependency is S2, which has an open source version you can self-host called s2-lite (https://news.ycombinator.com/item?id=46708055).
How it works:
- one S2 stream per browser session
- large rrweb events (like a full snapshot) get framed across multiple binary S2 records and reassembled on read
- active sessions are tailed with an S2 read session, and bridged to the browser over SSE
- session listing relies on stream names encoding reverse timestamps, as S2 returns a lexicographic order listing
- relying on fencing tokens so a stopped session can't be written to again by a late recorder
- retention and GC are handled via S2 stream config, so no background job needed
Curious to hear from folks on the tool or the stream-per-session model!
Genuinely curious, because it's not something I think most people are aware of when they browse the web.
And to be honest, they shouldn't need to be. Browsers shouldn't be app hosts unless the user wants them to be.
ballpark, how much does this cost to run?
On cost, it's running the collector Node app (I'd expect a few $ per month at low volume), and the S2 stream backend.
If you use the S2 cloud service, cost is basically just the rrweb bytes. The rates are $0.075/GiB to write, $0.05/GiB-month to store, $0.10/GiB to read back over the internet. See s2.dev/pricing.md for an agent-friendly summary.
Assuming a typical few-minute session is ~1 MiB of events, ingesting it, storing it a month, and replaying it a couple of times (unlikely!):
1k sessions/mo ≈ $0.35
10k sessions/mo ≈ $3.50
100k sessions/mo ≈ $35