Think of a typical loop we may ask of Claude Code today (assume we are not using TDD): run some test suite with fail fast mode, diagnose if the failure is due to recent feature changes (pass reference to backend/frontend, github issues, PRD,...). Ask CC to decide if test failed due to feature change and then update the test. Perhaps ask CC to use sub-agent to investigate and fix (if deemed so). Commit each fix, move on to next.
I know, this has so many ways to make blunder but I am talking about the agent here, not our error-prone test maintenance. What if we had an agent that had context of your codebase, deterministically ran test suite, linter, hooks, etc. The "English" prompt would become a code loop with the LLM only brought in to decide if a test has failed because of feature change. Also, we can extract git log, JIRA and what not.
Each tool here is real code. Executable code that calls others and only prompts when they meet edge cases. Edge cases are defined but we can now accelerate the maintenance of these tools using agents themselves. But the system is built on "programs that do one thing and do it well" and then reach out to an LLM for its specific edge case. The agent is how these executables work with each other.
There is this ACM blog post called "Manual Work is a Bug" [0] that was originally written to help humans automate processes using code. I find it just as applicable today as when it was written. You and the LLM look at what has to be done and then figure out the scripts/tools to make it happen. You then tie those tools into a system.
The more I use the above the more it makes sense and the worse the whole "just commit the prompt" seems like nonsense.
As coding agents have accelerated my work, I just build tons of tooling around existing software. Or in rare cases build new ones. If we zoom out of software engineering, we will still be in the realm of files - text or binary. That does not change.
The question is - do we let agents run the tools or the "programs" call the LLMs. The OS is the new agent, but not the same sense of "agent". I want LLMs to be lightly sprinkled in a future "agent" OS, not the other way around.
OP's idea "everything is a text file" is good and I use it too. My plans are saved as task.md files, numbered and named. Work items are checkboxes inside the file, closed work items are checked and a comment is added on the same line to provide feedback about the implementation.
I also keep a current-state-of-the-world document, it should be <20KB of text, keep the essential decisions and intents. Loading it allows resuming in <30s.
Something I never saw anyone else do - I save all user messages in a chat_log.md file which is referenced for intent alignment and state recovery. I consider the chat log on the one hand, and coded tests on the other hand as the two walls, the agent works in the mid section between them.
https://horiacristescu.github.io/claude-playbook-plugin/docs...
What I am saying is the opposite - use Claude Code or whatever else - generate actual "programs". Basically scripts. We have tons of ways for "programs" to interact with each other. Then have clearly defined edge case handlers - think "try/catch". How far do you want to go down the rabbit hole in the "catch"? Do you want to re-write a new version of the "program" itself? I do not know, but this type of a system is what Unix already is, with the addition of programs themselves reaching out to LLMs in well defined edge case handlers.
The API is basically what you see as a user of Claude Code or Pi or whatever. You can make new sessions, send messages to sessions, configure which MCPs get started, etc.
I’ve been poking at something similar to what you’re talking about via that route. My client prompts the agent to do a thing, and then afterwards launches deterministic things to check it which can either re-prompt the original session or start a new session.
Eg it automatically runs the tests afterwards, and will send a new prompt in the original chat to fix them if they fail. I also briefly poked at a security analyzer that gets changed files via git and makes a new session to check whether there are security issues and propose a fix that then gets sent to the original session.
If you want a circular loop where the LLM can adjust its own workflow while keeping it deterministic, you can let the agent modify the ACP client that drives it.
https://www.langchain.com/blog/tuning-the-harness-not-the-mo...
My harness is a Claude Code plugin with its own brainstorming, adr, and planning skills with associated review and interview skills. Behavioral testing related to acceptance criteria is built in. Everything in my harness is gated to prevent ratholes.
I recently inflated a docker container to execute a set of work with Claude in unsafe mode and immediately saw problems with everything it was doing…and then I realized I had not installed my harness.
Running Claude without an engineering harness is like driving a car without brakes or a steering wheel.
I feel like Docker Compose / K8S / VM / Dagger.io layers are close but can't quite always recurse flexibly and aren't always simple to run with. Networking / devices / auth are often awkward choke points
> When in doubt, simplify. Remove, trim and minimize. Reproduce issues in as small cases as possible, understand the full design completely, there is no shortcuts for this.
If it had a lossless, massive context window (100m-1b tokens), then it will squash everything. Give it bash + r/w and it can in theory /goal anything.
I think there's something to be gained in a production environment be siloing agents for reproducebility/auditability, but I suspect that will go away in the future.
There's that video of a silly demo someone made of an OS that was just nested copilot instances that generated the HTML of each window, which allowed you to do whatever you could imagine. It was seen as silly because it was, but that seems truly transformative.
I build precision-editing tools for AI coding agents (hic-ai.com) and worked out thousands of JSON-wrangling and regex issues, so I can verify they are indeed a bit of a pain, across all possible failure modes that AI coding agents and models and harnesses can produce. Anyway, I completely agreed with everything in your article, though I would suggest however that agents need *three* things at runtime to fix a defect: great logging and a clear error response (just like you have it), but also, precision-editing tools that enable agents to make the minimal, surgical change without touching or copying any other portion of the file. These actually change not just the feedback but also the options available to the agent and capabilities in the midst of the workflow to self-heal. If Ambiance adds a kernel to buffer the LLM from the outside world, HIC Mouse adds a "kernel" or buffer between the LLM and its own environment and file system. Anyway, this is such a cool project. Please reach out if you ever add MCP support for Ambiance -- I'm happy to release a new version of Mouse that supports it. Again, great work.
{ aislop pitch}
> Again, great work.
i can bet you didnt actually read the op. i hate these comments so much. selfish and rude.
your comment exclusively about your aislop project
He also argued that his project implemented a kernel pattern that acted as a buffer between the LLM and the outside world. I too implemented a buffer pattern in my own project.
He built an AI harness, I think it's a nice project, and if it extends MCP compatibility, I do think HIC Mouse would be a nice integration with Ambiance.
Yes, I think Ambiance is a great project and I am wishing the author all the best luck and success in the future. Yes, I gave it a GitHub star. Sorry I didn't give fainter praise?
By the way, I wrote every line of my project myself; I wrote every word of my research myself; I wrote every word of the copy on my website, including all legal provisions, docs, and blog posts, myself; I write all my comments on HN myself. I'm proud of my work and stand by my project, which ironically is dedicated to reducing AI slop and boosting accuracy so that AI agents can perform surgical, precise code repairs without ever touching or copying any other part of the code base. Let me know what you're doing to reduce slop, other than accusing people falsely of generating content with AI.