I haven’t looked under the hood here but to make simple text replacement via command line is an LLM even required? A human driven command line tool to do basic substitution on batches of files reliably would be amazing.

I originally started this project just to build a TA agent for a professor who didn't have any TAs (my wife). So as you can imagine, it was critical that it could properly write comments and only edit with track edits mode on... and do all of this without accidentally breaking the structure of the doc that couldn't be read.

It's since then expanded to cover everything from editing tables, hyperlinks, footnotes, and a lot more. Now it's a pretty powerful tool that can trivially fill out a MNDA form, mark up a contract, author a poetry booklet, and fill out an invoice, which is now the eval suite where the numbers in the title come from.

You might be asking, "why did you do all of this?" Well, I'm building an agent harness for normies that are not gonna know what a token even is but just want their stuff not to take an epoch and a half to run. So I've got to make the tools be MUCH more optimal than they've even been.

I figure putting them out to the community and inviting all of you to help me might be a way to do that =).

Very cool. So much of the 'capability overhang' of AI can be addressed with tools like this--data manipulation etc without LLMs having to galaxy brain everything in token space

I like it