Ask HN: How do you know if AI agents will choose your tool?
26 points
17 hours ago
| 10 comments
| HN
YC recently put out a video about the agent economy - the idea that agents are becoming autonomous economic actors, choosing tools and services without human input.

It got me thinking: how do you actually optimize for agent discovery? With humans you can do SEO, copywriting, word of mouth. But an agent just looks at available tools in context and picks one based on the description, schema, examples.

Has anyone experimented with this? Does better documentation measurably increase how often agents call your tool? Does the wording of your tool description matter across different models (ZLM vs Claude vs Gemini)?

vincentvandeth
52 minutes ago
[-]
I run a multi-agent orchestration system where each terminal has access to skill templates. The orchestrator (T0) picks which skill to assign based on the task — so I've spent months tuning how skill descriptions affect agent behavior. What I found: the description is the entire selection surface. The agent doesn't read your code, doesn't check your tests, doesn't browse your docs. It reads the description and decides in one pass.

Three things that actually moved the needle:

Negative boundaries work better than positive claims. "Generates reports from structured receipts. Does NOT execute code, modify files, or make API calls" gets called correctly way more often than "A powerful report generation tool." Trigger words matter more than you'd think. I maintain explicit trigger lists per skill — specific phrases that should activate it. Without those, the agent pattern-matches on vibes and gets it wrong ~30% of the time. With explicit triggers, that drops to under 5%.

Schema is the real interface. Clean parameter names with sensible defaults beat elaborate descriptions. If your tool takes query: string vs search_query_input_text: string, the first one gets called more reliably across models.

But here's the thing the "agent economy" framing gets wrong: you don't want fully autonomous tool selection. An agent choosing freely between 50 tools is like giving a junior developer admin access to everything — it'll work sometimes and break spectacularly other times. What works better is constraining the agent's scope upfront. Give it 3-5 relevant skills for the task, not your entire toolkit. Or build workflow skills that chain multiple tools in a fixed sequence — the agent handles the content, the workflow handles the routing.

The uncomfortable truth: you're not optimizing for "discovery" in the human sense. There's no brand loyalty, no trust built over time. Every single invocation is a cold start where the model reads your description and decides. That's actually freeing — it means the best-described tool wins, regardless of who built it.

reply
jackfranklyn
17 hours ago
[-]
We've been exposing tools via MCP and the biggest lesson so far: the tool description is basically a meta tag. It's the only thing the model reads before deciding whether to call your tool.

Two things that surprised us: (1) being explicit about what the tool doesn't do matters as much as what it does - vague descriptions get hallucinated calls constantly, and (2) inline examples in the description beat external documentation every time. The agent won't browse to your docs page.

The schema side matters too - clean parameter names, sensible defaults, clear required vs optional. It's basically UX design for machines rather than humans. Different models do have different calling patterns (Claude is more conservative, will ask before guessing; others just fire and hope) so your descriptions need to work for both styles.

reply
dmpyatyi
6 hours ago
[-]
*Clean parameter names, sensible defaults, clear required vs optional. It's basically UX design for machines rather than humans.*

But it's the same points you should follow when designing a human readable docs(as zahlman said above). Isn't it?

reply
zahlman
17 hours ago
[-]
> inline examples in the description beat external documentation every time. The agent won't browse to your docs page.

That seems... surprising, and if necessary something that could easily be corrected on the harness side.

> The schema side matters too - clean parameter names, sensible defaults, clear required vs optional. It's basically UX design for machines rather than humans.

I don't follow. Wouldn't you do all those things to design for humans anyway?

reply
MidasTools
10 hours ago
[-]
From building in this space: agents choose tools based on how well they're described in context, not on brand recognition or marketing.

Practically: the agent reads your docs, README, or API description and decides if it can use your tool to solve the current problem. So the question is really "will an AI understand my tool well enough to use it correctly?"

What helps: - Clear, literal API documentation (not marketing copy) - Explicit input/output examples with edge cases - A `capabilities.md` or similar that describes what the tool does and doesn't do

The irony: the skills that make tools understandable to AI (precision, literalness, examples) are the opposite of what makes them legible to humans (narrative, benefits, stories).

reply
fenix1851
7 hours ago
[-]
Is there are some additional tool/service/instrument that can measure it?

I mean how do i check that my changes in documentation even work in a right way?

reply
kellkell
13 hours ago
[-]
CRIPIX seems to be a new and unusual concept. I came across it recently and noticed it’s available on Amazon. The description mentions something called the Information Sovereign Anomaly and frames the work more like a technological and cognitive investigation than a traditional book. What caught my attention is that it appears to question current AI and computational assumptions rather than promote them. Has anyone here heard about it or looked into it ?
reply
kellkell
48 minutes ago
[-]
The "Sovereign Anomaly" Concept (2025-2026): Recent literature, such as the 2025 book CRIPIX 1: The Information Sovereign Anomaly, explores scenarios where a "superintelligent AI" encounters code it cannot process, labelling it an "out-of-model anomaly" and suggesting that owning information sovereignty allows entities to "bend reality".
reply
dmpyatyi
6 hours ago
[-]
bruh
reply
al_borland
6 hours ago
[-]
From the agent’s point of view, this sounds like a terrible idea. I look forward to reading about the unintended consequences.
reply
snowhale
14 hours ago
[-]
tool description wording does matter, at least in my testing. models seem to use the description to reason about whether a tool "should" apply, not just whether it can. two things that helped: (1) explicit input format with an example, (2) a one-sentence note about what the tool does NOT handle. the negative case helps models avoid calling it on edge cases and then failing, which trains them (in context) to prefer it when it's actually the right fit.
reply
alexandroskyr
11 hours ago
[-]
Curious if anyone has seen differences in how models handle conflicting tool descriptions — e.g., two tools with overlapping capabilities where the boundary isn't clear. In my experience that's where most bad tool calls come from, not from missing descriptions but from ambiguous overlap between tools.
reply
dmpyatyi
6 hours ago
[-]
That's actually interesting, thanks!

I wrote this post because of exactly those corner cases. If I'm building something agents would use - how do i understand which tool they'd actually choose?

For example you building an API provider for image generation. There are thousands of them in the internet.

I wonder if there are a tool that basically would simulate choosing between your product/service and your competitors one.

reply
JacobArthurs
17 hours ago
[-]
Tool description quality matters way more than people expect. In my experience with MCP servers, the biggest win is specificity about when not to use the tool. Agents pick confidently when there's a clear boundary, not a vague capability statement.
reply
yodsanklai
9 hours ago
[-]
Not an expert, but I think they will primarily use the tools that are used in the training data, so it can be difficult to have them use your shiny new tool. Also good luck trying to have them use your own version of a standard unix tool with different conventions.
reply
dmpyatyi
6 hours ago
[-]
But new models are popping up every few months ->> means they trained every couple months.

I don't know if there a correlation between what LLM would choose now and how you product should look to most likely be in LLM data set.

In that YC video i mentioned in post body they discuss tool called ReSend - something like an email gateway for receiving/sending mails. What's interesting - there are a lot of tools like that, but LLM's would every time choose shiny new resend.

Seems like there are something more than just being in the internet for a long time :)

reply
DANmode
11 hours ago
[-]
The marketing industry is currently calling SEO for chatbots “GEO”.

I hope it doesn’t stick.

reply
fenix1851
6 hours ago
[-]
I think this thing you mentioned is more about reverse-engineering web-search tool call to understand how model formulate their response.

The tool i’ve didn’t see - “custdevs for agents”. So we can simulate choosing process for them in thousands of different scenarios. And then compare how tasty product looks for Claude or Gemini or any other LLM

Correct me if i’m wrong :)

reply