Apparently Google hates us now
138 points
1 hour ago
| 14 comments
| twitter.com
| HN
phyzix5761
5 minutes ago
[-]
Why would Google need to direct traffic to the website when they've already scraped and trained their models on the data? Content creators and legitimate websites were wham-bammed and thank-you-ma’amed.
reply
hungryhobbit
25 minutes ago
[-]
They're a wiki. Wiki spammers are relentless now.

Source: a small wiki I help manage, for an obscure game with <10k players, recently had to disable new signups, because the spam was so bad (and it was stuck on an old version of MediaWiki, which didn't have CAPTCHA-support).

On a popular wiki, and it sounds like this one was fairly popular, I imagine even CAPTCHA's won't be enough to stop wiki spammers. If those spammers were posting more than just "buy my penis pill" garbage (e.g. they were putting links to malware sites), Google probably, and somewhat legitimately, saw them as a source of such malware.

I imagine the fix for the OP is a thorough audit/cleansing of all malicious content on the wiki, followed by some sort of appeal to Google (which will no doubt take months, if they even respond at all, because ... Google).

Really OP's only hope is that the Google team responsible for this has an Italian Pokemon fan; otherwise they are probably screwed.

reply
zeitg3ist
18 minutes ago
[-]
We have very good anti-bot system set up with a good number of Cloudflare fine-tuned rules, limited permissions for newly created accounts, and a very dedicated team of volunteers that patrol the recent edits constantly. I cannot exclude that somewhere on a rarely visited page (out of 37k+) there is a spam link, but I doubt it’s the reason for the deindexing. I think this would also appear on the Google Search Console.
reply
teaearlgraycold
4 minutes ago
[-]
An organization I'm involved with has had to add Anubis (https://github.com/TecharoHQ/anubis) because of the recent wiki attacks from LLM scrapers. It's finally fixed our outages.
reply
righthand
24 minutes ago
[-]
Social sites should have all have a tree-based invite system. This would allow wiping out spammers and their enablers in a single hit. It would allow vetting of good actors too.
reply
Sayrus
16 minutes ago
[-]
You still need criteria to handle reputation: does an account invited years ago and now spamming affects the reputation of the inviter, how much? What about the hacked accounts?

For small platforms it makes a lot of sense, for larger the potential for abuse is still there in different forms.

reply
ajkjk
7 minutes ago
[-]
I feel like the dream solution is more like tree-based content: you see content that is vouched for by people you vouch for; if someone's account is compromised then their vouches get updated to not matter anymore, cutting their whole tree off at the root to make it invisible. Spammers should end up in largely disconnected components of the trees.
reply
charliebwrites
21 minutes ago
[-]
That’s literally how Facebook started

I remember begging my older step brother for an invite since he had the college email to get in

reply
CalRobert
19 minutes ago
[-]
Interesting to compare this site and lobste.rs for that
reply
threecheese
16 minutes ago
[-]
Both from safety and volume perspectives, I’d imagine. Openness has value.
reply
p4bl0
25 minutes ago
[-]
The same thing happened with my blog a few weeks ago. It was well referenced for years and suddenly almost all of my entries are not indexed anymore. The Search Console indicates that the URLs were crawled but are currently not indexed, and contrary to technical problems, there nothing I can do to fix it, I just have to accept that most of my articles cannot be found via Google anymore.

EDIT: I don't actually think it is related, but now that I think of it, the timing corresponds with when I started setting up TDMRep to forbid using my content to train LLMs.

reply
judah
2 minutes ago
[-]
Same. I've been running a personal blog for over 20 years. Last year, I couldn't find any links to my blog on Google. Went to Google Search Console to find all my links are "Crawled by not indexed", with no reason given.
reply
frouge
37 minutes ago
[-]
I can even tell you that Google hates us all
reply
georgemcbay
22 minutes ago
[-]
Google neither hates nor loves any of us, the only thing it cares about as an institution is cramming as many advertisements in front of as many people as it can get away with to generate increasingly ridiculous piles of money.

This is not meant to be a defense of Google, which is (like virtually every large corporation) completely sociopathic.

reply
EvanAnderson
9 minutes ago
[-]
Public corporations were multicellular biological organisms made up of individual cells working toward the collective goal of continuing the organism's existence. Each cell received nourishment from the corporation, in the form of monetary compensation and other benefits. Some cells have a more direct role in the "reasoning" process of the organism than others.

These organisms aren't collectively sentient, though it could be argued some of the constituent cells are. They are able to influence their environment by using individual cellular consciousnesses to communicate with conscious constituent cells in other organisms.

Ultimately, the organism itself has the goal of producing value for its owners. The methods the organism used to achieve these goals are somewhat opaque to the owners, and potentially inscrutable to the individual constituent cells. If the owners stop receiving value the organism becomes unable to nourish its cells and it dies.

Recently these organisms have become biological / technological hybrids including unconscious computational models in their reasoning process. That increases the inscrutability and opacity of the process by which it reasons. It's likely the unconscious computational models will eventually be tasked with communicating with similar models in other organisms, at which point the inscrutability will probably increase by an even greater amount.

reply
logicchains
19 minutes ago
[-]
All large companies are sociopaths, but few tech companies treat their paying customers with the level of contempt that Google does.
reply
xp84
8 minutes ago
[-]
I'm not sure most of those calling the shots at Google realizes they even have paying customers other than advertisers. Notably though, website publishers and consumers of their massive products like Search, Gmail, and Android are not really customers.
reply
marginalia_nu
24 minutes ago
[-]
To be honest it's probably just jank on Google's end.

There's a lot of delayed cause and effect in search, and it's much easier to make a minor mistake that excludes 0.1% of websites from crawling or indexing than it is to detect that it's happened except from affected websites telling you about it.

reply
paol_taja
3 minutes ago
[-]
You guys made the classic SEO mistake of building a real community site instead of a Reddit thread, a coupon subfolder, or an AI summary.

Scherzi a parte, spero che possiate recuperare presto…

reply
zeitg3ist
1 minute ago
[-]
Grazie! Speriamo anche noi.
reply
computomatic
2 minutes ago
[-]
A wiki with only 11 pages?

Perhaps they will investigate why 541,000 pages aren’t being indexed. In my experience, Google provides adequate tools for identifying and resolving indexing issues.

Google won’t serve pages it hasn’t indexed. Seems they left a lot of relevant details out of that tweet.

reply
astkl
15 minutes ago
[-]
My guess is that the combination of Wiki and Pokemon is highly suspect for Google.

The Pokemon Industrial Complex has advanced astroturfing especially on YouTube/Twitch, where streamers mention the damn things in any second episode, they "accidentally" meet people going to Pokemon conventions in live streams and so on.

Try to audit the Wiki if anyone abused it.

reply
clacker-o-matic
55 minutes ago
[-]
oof that sucks; i really wish there was more info on why google decides to crawl or not crawl a page
reply
cynicalsecurity
9 minutes ago
[-]
Can someone start a new Google, please? Just search, nothing more. I'm willing to pay 10 USD a month for that. API access included.
reply
CharlesW
8 minutes ago
[-]
reply
kevincrane
4 minutes ago
[-]
Yeah Kagi already exists luckily, it’s extremely good and worth the money.
reply
ChrisArchitect
34 minutes ago
[-]
Title could be: Apparently Google hates Pokémon Central Wiki now
reply
m4tthumphrey
29 minutes ago
[-]
No, I think "us" is apt, considering this will eventually affect all sites that rely on traffic from Google search, which is basically every text heavy site.

All we can hope for is that people will stop using search (after eventually having enough of the AI wave) for these sort of niche sites and will bookmark and access them directly in future. I don't have much hope.

reply
cess11
23 minutes ago
[-]
Perhaps they're decommissioning search in favor of LLM:s.
reply
arikrahman
20 minutes ago
[-]
This aligns with their Google Zero doctrine, keep all info internal and make the goal for the user to hit 0 external websites.
reply
CodesInChaos
11 minutes ago
[-]
That's only supposed to happen later this week.
reply
echelon
51 minutes ago
[-]
Pokemon Central runs ads (Google AdSense at that!), which is probably how they pay for everything.

Google is likely their biggest inbound source of traffic, so they're probably experiencing a marked revenue drop as well.

It's unfortunate that so many livelihoods are subject to the capricious whims of a single company. A company that is increasingly seeking to keep users on their engine without sending eyeballs or revenue to any third parties at all.

We're watching Google's "embrace-extend-extinguish" arc for the web. It's not over by a long shot, but they absolutely intend to finish the job.

reply
vrganj
41 minutes ago
[-]
Hi EU. How about one of those lovely anti-trust cases?
reply
spiderfarmer
45 minutes ago
[-]
It’s why I moved to in-house advertising. It’s a lot of work, but I hope it is the right decision.
reply
righthand
28 minutes ago
[-]
Why is it a lot of work? Could you specify some off the more difficult effort? Wouldn’t LLMs help speed this up? This is the one area where I’d think Llms could really take Google down by empowering in house ad platforms.
reply
dylan604
14 minutes ago
[-]
Depends on how in-house you want to go. If you go full in-house, you'll need sales staff to make deals with advertisers. You'll then need a way of hosting the media provided. You'll need a way to deal with media that does not match what you've requested. You'll need a system to allocate ad space accordingly to contracts with ad clients. It's like a whole new department in your company.
reply
pkaye
32 minutes ago
[-]
Its better off if ads go away. Just use ad blockers.
reply
zeitg3ist
27 minutes ago
[-]
We would like the wiki to be free of ads, but hosting costs at our scale are real. Since we don’t like ads either, we compromise like this: users can register for free and never see an ad (they are only served to anonymous visitors); they can also use an ad blocker and we won’t bug them about it.
reply
drcongo
53 minutes ago
[-]
reply
scrollop
32 minutes ago
[-]
Thank you
reply
pcdavid
20 minutes ago
[-]
This might be useful: https://addons.mozilla.org/fr/firefox/addon/toxcancel (Redirects to xcancel.com (a mirror of x) when the browser is about to load an x.com page).
reply