All of Google Offline?
28 points
1 month ago
| 16 comments
| HN
pcthrowaway
1 month ago
[-]
Noticed no domains were resolving. Turned out Google's DNS server (8.8.8.8) is down.

Has this ever happened before? I'm scared.

Interestingly my VPS in google cloud is the only google product I can see that's working right now.

reply
TacticalCoder
1 month ago
[-]
Wife and I have both been working from home for hours, including using GMail / Google workspace and didn't notice anything wrong...

But then my DNS is always CloudFlare's 1.1.1.3 (no known porn sites, no known malware sites resolving). I used to use 8.8.8.8 in the past but I'm a happy user of CloudFlare's DNS.

More specifically: I use unbound as a DNS resolver and unbound uses 1.1.1.3 then locally I use dnsmasq to cache DNS answers for a little bit.

reply
worksonmymach
1 month ago
[-]
TIL! I'd love a 1.1.1.4 (plus adblocker) and 1.1.1.5 (plus adblocker plus social media blocker)
reply
gtirloni
1 month ago
[-]
NextDNS let's you manage that plus a ton of other features.
reply
asdf6969
1 month ago
[-]
Fixed as of a few minutes ago. Thanks oncalls. Sweet dreams :)
reply
worksonmymach
1 month ago
[-]
Or thanks follow the sun sres
reply
cynicalsecurity
1 month ago
[-]
reply
gadtfly
1 month ago
[-]
Gmail, Youtube, etc.

Checked from Western US and Canada.

reply
linksbro
1 month ago
[-]
Best of luck, Google oncalls.
reply
fusl
1 month ago
[-]
Looks like it was returning some SERVFAILs on 8.8.8.8 and .4.4
reply
everfrustrated
1 month ago
[-]
Cloudflares quad one had an outage overnight also. Curious.
reply
Youden
1 month ago
[-]
I haven't seen any outages. Response times have been consistently low, around 1.5ms: https://prnt.sc/W2W8Jj1NvkOo

Maybe it's only your area?

reply
RedShift1
1 month ago
[-]
Works for me (EU). DNS servers 8.8.8.8 and 8.8.4.4 working.
reply
mellow-lake-day
1 month ago
[-]
Everything was down for me

I was using Google DNS, guess that was a mistake

reply
carlheinzc
1 month ago
[-]
All working here from (South-Africa)
reply
hbh187
1 month ago
[-]
Google's DNS server was down.
reply
mrlucax
1 month ago
[-]
All WFM (Brazil).
reply
davidt84
1 month ago
[-]
All WFM (UK).
reply
guessmyname
1 month ago
[-]
Yes → https://downdetector.com/status/google/

This website shows that people started reporting issues with Google around 1:28 AM (Pacific Time), and it currently shows a spike of 7,419 reports.

EDIT: Spike went down at around 2:04 AM (Pacific Time). Thanks, Google Site Reliability Engineers, now go to sleep!

Just a reminder: DownDetector doesn’t actually check if a website is down. It collects data on how many people have reported issues or asked about a site’s status. So, while it doesn’t always reflect the real-time status of a website, it can give a rough idea of the scale of the problem—assuming the site really is down.

Getting “Error 502 (Server Error)!!1” from 142.250.217.78 (“A” DNS record).

reply
danpalmer
1 month ago
[-]
Looking forward to the postmortem for this one.

Worth noting though that SREs at Google are rarely woken up. One of the main differences between SREs who support a service and dev owners who may also be on call is that the SREs will be in some sort of follow the sun configuration of 12 hour shifts. My team in Sydney partners with San Francisco for example, so worst case for me is a page at midnight just before I hand over, worst case for them is about 5am. So not usually anything happening in the middle of the night, at worst a late night or early morning, but never both.

reply
holistio
1 month ago
[-]
Everything works.

(Hungary)

reply
ale42
1 month ago
[-]
all fine here (CH)
reply