Fast KV Compaction via Attention Matching
42 points
8 hours ago
| 2 comments
| arxiv.org
| HN
cadamsdotcom
1 hour ago
[-]
Superficially it sounds like this could create a bit more of a move toward doing compaction on some continuous basis, or compacting in batches once you hit the context limit, rather than starting fresh with a summary and system prompt..

Feels like high fidelity, fast compaction could be a path to “solving” long context.

reply
speedping
19 minutes ago
[-]
This is big for long-horizon tasks
reply