I've been watching the rise of AI agents with a mix of excitement and dread. We're building incredible tools that can browse the web, but we're forcing them to navigate a world built for human eyes. They scrape screens and parse fragile DOMs.
We're trying to tame them to act like humans. I believe this is fundamentally wrong. The goal isn't to make AI operate at a human level, but to unlock its super-human potential.
The current path is dangerous. When agents from OpenAI, Google, and others start browsing at scale and speed, concepts like UI/UX will lose meaning for them. The entire model of the web is threatened. Website owners are losing control over how their sites are used, and no one is offering a real solution. The W3C is thinking about it. I decided to build it.
That's why I created AURA (Agent-Usable Resource Assertion).
It's an open protocol with a simple, powerful idea: let website owners declare what an AI can and cannot do. Instead of letting an agent guess, the site provides a simple aura.json manifest.
This gives control back to the site owner. It's a shift from letting AIs scrape data to being granted capabilities. We get to define the rules of engagement. This allows us to increase what AIs can do, not by letting them run wild, but by giving them clear, structured paths to follow.
A confession: I'm not a hardcore programmer; I consider myself more of a systems thinker. I actually used AI extensively to help me write the reference implementation for AURA. It felt fitting to use the tool to build its own guardrails.
The core of the protocol, a reference server, and a client are all open source on GitHub. You can see it work in 5 minutes:
Clone & Install: git clone https://github.com/osmandkitay/aura.git && cd aura && pnpm install
Run the Server: pnpm --filter aura-reference-server dev
Run the Agent: (in a new terminal) pnpm --filter aura-reference-client agent -- http://localhost:3000 "list all the blog posts"
You'll see the agent execute the task directly, no scraping or DOM parsing involved.
The GitHub repo is here: https://github.com/osmandkitay/aura
I don't know if AURA will become the standard, but I believe it's my duty to raise this issue and start the conversation. This is a foundational problem for the future of the web. It needs to be a community effort.
The project is MIT licensed. I'm here all day to answer questions and listen to your feedback—especially the critical kind. Let's discuss it.
The issue with this is that website owners don't want to do this. Take Reddit removing the API for example. Everyone just switched to scraping the Reddit website instead for any remaining third party clients.
Yes, APIs were supposed to be a compromise to lower the resources needed on both sides, but Reddit's stock price is linked to the value of "their" data, so...
Alternatively, malicious website owners may make incorrect Aura files to mislead user agents. Then we're back to screen scraping as the ground truth, because behaving like a human is the best way to avoid discrimination.
And that s really what aura is trying to address. If we dont figure something out the web could easily end up just being a handful of big ai sites, and all the smaller, independent sites might just fade away.
The goal with aura is to give control back to the people running the websites. It s not about blocking ai but about giving sites a clear, standard way to say "here s how you can work with me meaningfully...". This means an agent can do something specific and useful without costly, aimless scraping, and it lets site owners build cool, new features just for ais.
and you re right to worry about malicious manifests but that s a trust problem. A site that lies in its aura file would get a bad reputation fast just like a phishing site does now.
At the end of the day aura is a bet that we can build an open, capability based web where site creators can join in on the AI revolution on their own terms. Time will tell if it's the right technical answer, but it s a conversation i think we absolutely need to have to keep the web diverse and creative.
Are any websites actually using it (for llms.txt: https://llmstxt.site/)? Why do I need to npm install anything instead of writing a text file?
-llms.txt tells an AI here is a clean, simple version of this page for you to read --Aura tells an AI here is a capability called create_post, it needs a title and content, and you can use it by POSTing to the /api/posts endpoint.
llms.txt is for reading, aura is for doing.
You also asked why you need npm install. You dont! To add Aura to your own website, you only need to create one static file - public/.well-known/aura.json. That's it. It s just as simple as creating a llms.txt file.
The npm install command in my project is for people who want to download and run my full reference implementation (the example server and client). It is not a requirement for the protocol itself.
And for your last question, Aura is new so no other websites are using it yet.
This honestly made me laugh out loud.
Which of course means this is never going to fly with any site that needs to show you ads.
Think of it this way -instead of an ai trying to find and click the "post comment" button it can just use the "post_comment" capability the site offers through a clean api. While this seems to sidestep the ad model it actually enables a more direct one. A site could specify that a particular action, say a premium translation feature, requires a small payment or an api key. It s a way to get paid for the actual value you provide, not just for ad views.
This could even change how search works like a search engine that indexes what sites can do, not just what they say. Your personal ai could then find and execute the "book a flight" capability from multiple airlines to find the best deal for you all without you ever having to load a webpage. It s a different way of thinking about the web s economy moving from attention to action.
Have you considered something like <script type="text/llm"> or Link: <https://api.example.com/llms/foo>; rel="llm:foo", or just normal content negotiation on individual pages?
However you are right that context is critical for individual URLs. I handle that with the dynamic aura-state http header. While the manifest is the static map of everything possible, the aura-state header in each response tells the agent what's available right now for that specific page or state. (e.g "you are logged in so create_post is now available").
So i get the best of both worlds efficient site-wide discovery dynamic *state-aware execution
For example, aura manifest says the create_post capability needs auth. If an agent ignores that and POSTs to /api/posts without a valid cookie, our server's API will reject it with a 401. The manifest doesnt do the blocking, the backend does. It just tells the cooperative agents the rules of the road ahead of time.
So the real incentive for an agent to use Aura isnt about avoiding punishment, it s about the huge upside in efficiency and reliability. Why scrape a page and guess at DOM elements when you can make a single, clean API call that you know will work? It saves the agent developer time, compute resources, and the headache of maintaining brittle scrapers.
So;
robots.txt tells good bots what they shouldn't do.
aura.json tells them what they can do, and gives them the most efficient way to do it, all backed by the server's actual security logic.
In that light, I guess your proposal makes a certain amount of sense. I don't think it addresses what a lot of web sites want, but that's not necessarily a bad thing. Own your niche.
Otherwise, it just seems you vibecoded the wheel.
OpenAPI is fantastic for describing a static API for a developer to read. But the web is more than that its a dynamic stateful environment built for human interaction. The current trend of forcing AI agents to navigate this human-centric web with screen scraping and DOM manipulation is brittle and I believe, unsustainable. Its like sending a robot into a grocery store to read the label on every single can instead of just asking the manager for the inventory list.
This is where Aura tries to be different in two key ways
Control & Permission:not just Documentation: Aura is designed from the website owner's perspective. It's a way for a site to say "This is my property and here are the explicit rules for how an automated agent can interact with it." The aura.json file is a handshake a declaration of consent. It gives control back to the site owner.
Statefulness(This is the big one): An OpenAPI spec is stateless. It cant tell an agent what it can do right now based on its current context. This is what the AURA-State header solves. So for example before you log in the AURA-State might only show you list_posts and login capabilities. After you successfully call login the very next response from the server includes a new AURA-State header that now unlocks capabilities like create_post and update_profile. The agent discovers its new powers dynamically. This state management is core to the protocol and doesn't really have a parallel in OpenAPI.
You re right to be skeptical and as I said in my post maybe Aura isnt the final answer. But I strongly believe the web needs a native capability-aware layer for the coming wave of AI agents. The current path of brute force interaction feels like it will break the open, human-centric web we ve all built.
> I actually used AI extensively to help me write the reference implementation for AURA.
So that's why. You drank the kool aid.
You can tell that OP is a big AI believer by the final sentence. That's gotta be one of the most ChatGPT lines I've ever read.
This is nothing like robots.txt, it is much more like a sitemap. In fact, this design goal is almost word for word the point of the semantic web in general. You may find that there are existing working groups for similar resource description frameworks. Given how poor adoption of semantic tagging has been, I somewhat doubt sites start doing it just for LLMs.
Incidentally, I thought the whole point of an AI agent was that it could read and understand things by itself. I welcome any improvement in the semantic content of the web, but isn't scraping kind of the point?