Ask HN: Why isn't Google indexing information about the AT Protocol?
29 points
3 hours ago
| 7 comments
| HN
I've been observing this for a while, where very basic queries about atproto stuff doesn't show up. But yesterday I found a query that makes it VERY obvious: "list of public atproto relays". Here's DuckDuckGo:

1. https://firehose.directory

2. https://atproto.at/relays

3. https://atproto.wiki/en/wiki/reference/core-architecture/relay

4. https://pulsar.feeds.blue

5. https://leaflet.pub/12022731-ae4f-4a13-9f7a-5738b7a83c2e

Of those results, Google only has 3, the only one on the list that... doesn't have a list of public atproto relays. None of the other sites are present anywhere. Trying not to assume malice instead of incompetence here, but it's really ironic that one of the ecosystems with the strongest ties to the open web, with users creating dozens of new websites every week, isn't getting indexed.

entropyie
1 hour ago
[-]
Maybe they should have used a name that wasn't already taken...

https://en.wikipedia.org/wiki/Hayes_AT_command_set

reply
UI_at_80x24
1 hour ago
[-]
Lol, that's what I thought this was about too.

Why don't these people/companies think about how hard it will be to search for their $thing !?

reply
wmf
1 hour ago
[-]
Google search has generally rotted. Can't get promoted for maintenance.
reply
therobots927
1 hour ago
[-]
That’s best case scenario. Worst case is that they’re censoring the internet on a massive scale but have rolled it out slowly enough that no one noticed
reply
bilbo-b-baggins
1 hour ago
[-]
They are. There’s been a bunch of studies already showing how Google skews its results to align with oligarch interests.
reply
whateverboat
2 hours ago
[-]
https://imgur.com/a/OS7P9jh Seems working for me.
reply
iameli
1 hour ago
[-]
In the AI overview yeah, but do you see the sites below? I wonder if it's being tuned to prefer generating its own "list of X" implementations and downranking other peoples' lists.
reply
EnglishMobster
2 hours ago
[-]
Can confirm that I can also reproduce this. Interesting.
reply
runningmike
2 hours ago
[-]
I got 2300 results. DuckDuckGo Uses google. Google is very personalised. They use many dirty tricks to give personalised results. Use another box and check the results again.
reply
Peanuts99
2 hours ago
[-]
Doesn't DDG use Bing?
reply
iameli
2 hours ago
[-]
Are any of those results a list of public atproto relays?
reply
runningmike
2 hours ago
[-]
Yes.
reply
iameli
2 hours ago
[-]
Could you link it? Curious to see how buried it is for me. Personalization is one thing but on my results I'm getting Tor relays, Nostr relays, IPFS relays, and Fediverse relays and I haven't yet seen a list of atproto relays. I'm on like page 6 now.
reply
seanhunter
2 hours ago
[-]
Use incognito/private browsing mode to get a genuine sense of whether something isn’t in the index or whether google decides that even though you’re asking for it you probably don’t want it.
reply
danabramov
2 hours ago
[-]
I can reproduce the same in incognito mode.
reply
fragmede
1 hour ago
[-]
https://firehose.directory/robots.txt isn't one, but comes up in Google despite that.

https://atproto.at/robots.txt is good and findable with a direct search

https://atproto.wiki/robots.txt is empty but findable on Google.

https://pulsar.feeds.blue/robots.txt is 404 and thus not indexed by Google.

https://leaflet.pub/robots.txt is good. A logged in google search for this finds this for me.

Tell the people at https://pulsar.feeds.blue to fix their robots.txt if they want to get indexed.

reply