Trains halted across Germany because of communication system problem
120 points
by sva_
2 hours ago
| 27 comments
| apnews.com
| HN
mmoll
1 hour ago
[-]
If this weren’t Deutsche Bahn, I’d say it’s a cyber attack. Given that this is Deutsche Bahn, though, it may just as well be a maintenance issue.
reply
fnordian_slip
1 hour ago
[-]
That's what happens when you ignore critical infrastructure for three decades.

Of course, if the government were to correct the mistakes of the past, it would get worse for another decade. The necessary repairs would cause a lot more delays, and voters would then say "Were giving them so much extra money, and it gets worse? Unacceptable!". So I fear we'll continue to have these problems forever.

reply
JumpCrisscross
1 hour ago
[-]
> when you ignore critical infrastructure for three decades

To be fair, Deutsche Bahn is currently spending “€107bn between 2025 and 2029” on infrastructure upgrades [1].

[1] https://www.ft.com/content/db75e347-b13b-4753-8130-6301bb55c...

reply
wolfi1
33 minutes ago
[-]
that could have been a lot cheaper of they would have spent in the past (their spending seems to have been very low)
reply
okanat
1 hour ago
[-]
They need to spend at least 3x that and they need to bring redundant workforce to fix Germany. It is completely broken now.
reply
JumpCrisscross
59 minutes ago
[-]
> They need to spend at least 3x that

According to whom?

reply
okanat
50 minutes ago
[-]
Look at how much Switzerland spends per capita vs Germany. €477 vs €115. And Swiss kept their infrastructure well unlike Germans.

source: https://www.allianz-pro-schiene.de/themen/infrastruktur/inve...

reply
JumpCrisscross
39 minutes ago
[-]
> Look at how much Switzerland spends per capita vs Germany. €477 vs €115

Your chart shows close to €200 spent by Deutschland per capita in 2024, before the abovementioned spending splurge (about €30/person/year). (The numbers 477 and 115 never appear in your source.)

€230 in Berlin purchases about as much as €371 of CHF in Zürich [1]. So no, I’m not seeing evidence Germany needs to further 3x capital expenditure to unfuck its system, and that’s before observing it spends more than Italy per capita, and Italy’s intercity rail is fantastic.

[1] https://www.paritydeals.com/ppp-calculator/switzerland-vs-ge...

reply
uxhacker
5 minutes ago
[-]
What is surprising is that GSM-R is 2g. Does not 2g have many security issues?
reply
warumdarum
8 minutes ago
[-]
Could also be a russian firembomber gig worker getting brushed under mnt carpet of societal stability. Anything to keep this powderkeg of parallel societies going ..
reply
dfltr
1 hour ago
[-]
For DB, this type of outage is referred to as "Tuesday".
reply
eqvinox
1 hour ago
[-]
For context, in case people are less familiar with German politics:

DB is in a misbegotten state of privatization, started in the 90ies. The government spun it out into a private company but still owns 100% of it. They were trying to pump it up so they could sell it for good money. They did that by skimping on everything including maintenance, to try and make the numbers look good.

Except they never got to whatever magic numbers they wanted before the maintenance debt came rearing its ugly head and now everything is royally screwed. And because it's a private company, there's a whole bunch of barriers limiting how much they can even subsidize the thing at this point.

Not sure if this is better or worse than the UK's Network Rail story, but at the end of the day the only thing that will solve this is if they re-nationalize the tracks & infrastructure. What kind of an idiot thought including that in the privatization is a good idea is beyond me. It's not like you can build a 2nd railway network in order to get free market & competition. (For comparison, imagine privatizing the entire road network, village street to Autobahn.)

reply
martinald
8 minutes ago
[-]
Well, the EU insists that track & train operations are separate. (ironically the UK _is_ combining passenger operations and track somewhat back together, which is only possible because of brexit).

The bigger issue tbh is the enormous cost inflation in civil engineering in general. This seems to be a problem everywhere. There's no doubt some of this is caused by material cost increases, labour shortages etc, but I'd say the huge amounts of regulation added over the years is really a core driver of this.

reply
ahartmetz
41 minutes ago
[-]
The one good thing is that they failed to take it private. Imagine how bad it would be with the current maintenance backlog and no public funding.
reply
JumpCrisscross
59 minutes ago
[-]
> Not sure if this is better or worse than the UK's Network Rail story

“About 72 per cent of Deutsche Bahn’s intercity trains arrived within 10 minutes of their scheduled arrival time in the year to January 2025, compared with 78 per cent of British long-distance trains, according to the FT analysis.

Any interaction with the German rail network is also one of the biggest factors affecting the punctuality of long-distance rail travel in Central Europe” [1].

[1] https://www.ft.com/content/d3b6e6b5-eddb-4230-b866-932d284ce...

reply
eqvinox
50 minutes ago
[-]
The UK's railway network was only privately owned from 1994 to 2002 though, everything after that is already under the umbrella of re-nationalisation, which didn't go super well either (my knowledge about that is rather vague). Not sure how useful 2025 numbers are in this context.

[ed.: to be clear - AFAIK they are in the same state currently, private company but 100% government owned. But there's a huge distinction in that the UK has made the decision to move back in the direction of nationalisation. In Germany, some people still pretend this is somehow fine and just needs to get cleaned up before the privatization can continue.]

reply
lmm
21 minutes ago
[-]
> Not sure if this is better or worse than the UK's Network Rail story, but at the end of the day the only thing that will solve this is if they re-nationalize the tracks & infrastructure. What kind of an idiot thought including that in the privatization is a good idea is beyond me. It's not like you can build a 2nd railway network in order to get free market & competition. (For comparison, imagine privatizing the entire road network, village street to Autobahn.)

If you privatize the infrastructure and trains together you can at least compare one region against another, even if they're not directly competing. Trying to operate the trains separately from the tracks was a disaster in the UK and lead pretty directly to two mass casualty incidents.

reply
hobofan
1 hour ago
[-]
Probably someone forgot to renew the TLS certificate.
reply
gpvos
1 hour ago
[-]
You may not be far off. Word is that it's a failed software update.
reply
gruselhaus
1 hour ago
[-]
My 100 bucks are on an expired certificate in the trust chain. the same kind of issue that took down almost all Verifone payment terminals in Germany in 2022.
reply
polyomino
1 hour ago
[-]
These are effective targets for hybrid warfare for that very reason, plausible deniability
reply
ed_balls
1 hour ago
[-]
Same thing happened in Poland and it was confirmed that Russians did it.
reply
thih9
1 hour ago
[-]
Do you have a link?

Was it similar to what we’re seeing now (nationwide, radio related)?

reply
Kwpolska
1 hour ago
[-]
https://hackaday.com/2023/08/29/polish-railways-fall-victim-...

tl;dr: Trains can be stopped by a transmitting a simple, documented tone sequence over analog radio.

reply
eqvinox
1 hour ago
[-]
Ah, good, not the same thing then.

Honestly, DB are perfectly capable of clusterf*cking their GSM-R without help from Russia.

reply
justsomehnguy
1 hour ago
[-]
> it was confirmed that Russians did it.

>> It’s believed the perpetrators of the attack were supporters of the Russian war effort, as the stop signals were also joined by broadcasts of the Russian national anthem and a speech from Russian President Vladimir Putin. The attacks have some significance to the invasion of Ukraine, as Poland has been a hub for crucial weapons deliveries supporting the defence of Ukraine.

Yes, yes, it's a code of honour not to use the someone' else national anthem, sure. Especially if you need to bolster the population support for some ongoing cause.

reply
Bluebirt
1 hour ago
[-]
You mean neglect?
reply
thih9
1 hour ago
[-]
Neglect is basically unscheduled maintenance.
reply
6LLvveMx2koXfwn
1 hour ago
[-]
Neglect is basically scheduled unmaintenance.
reply
gpvos
1 hour ago
[-]
Thirty years of it.
reply
wolfi1
36 minutes ago
[-]
and/or incompetence
reply
bflesch
1 hour ago
[-]
It's russian hybrid warfare against Germany. Since invasion of Ukraine there have been numerous cable cuttings on train tracks, several train derailments, some fires.

It has become so bad that police helicopters are regularly patrolling train routes at night to spot sabotage as early as possible. People complain about the flight noise at night which was not there before.

So as a person working in cyber security, I'd put this into the sabotage bucket.

reply
fhars
31 minutes ago
[-]
It's not, it was a scheduled software update.
reply
felooboolooomba
1 hour ago
[-]
There was also a very peculiar train crash in the UK just a few days ago. A train hit a stationary train. That shouldn't really happen in this day and age. Sabotage was the first thing that came to my mind.

https://www.bbc.co.uk/news/articles/c4gy60gg6k5o

reply
beejiu
1 hour ago
[-]
Yes, and just yesterday a passenger train was routed into the path of a freight train due to some points failure. It does make you wonder. https://www.railmagazine.com/news/points-failure-results-in-...
reply
6LLvveMx2koXfwn
45 minutes ago
[-]
The trains heading for each other was not a direct result of the points failure but a direct result of the manual operation of the points after the failure. Everything was on go-slow so no risk of collision. This was human error.
reply
OJFord
1 hour ago
[-]
Maybe. OP isn't saying it's necessarily malicious interference though.
reply
Blahah
1 hour ago
[-]
Not a stretch to imagine that it is though. Germany has some very effective radical vandals who make statements by interrupting infrastructure.
reply
wolfi1
54 minutes ago
[-]
could still be incompetence, one newspaper says an update has gone wrong
reply
section_me
1 hour ago
[-]
The UK buys most of their trains from Deutsche Bahn (German Rail) and just brands them differently.

British person living in Berlin.

reply
ahartmetz
1 hour ago
[-]
Deutsche Bahn doesn't manufacture rolling stock. They buy it from Siemens, Stadler, Talgo, Alstom etc...

Edit: AFAIK, some of it - mainly high-speed trains - is designed according to DB specs and subsequently offered under a new name (and with changes) to other train companies. For example DB ICE 3 (manufactured by Siemens) / Siemens Velaro.

reply
SiempreViernes
48 minutes ago
[-]
You can probably buy some of the older rolling stock from DB thought.
reply
ahartmetz
19 minutes ago
[-]
It does happen occasionally, but DB tends to run, maintain and sometimes upgrade its successful (~reliable and widely introduced) rolling stock until it falls apart or is grossly outdated (40+ years old). Rail passenger numbers are increasing, so there is no need to sell stuff to downsize.

The Flixtrain company uses 40+ years old IC (intercity) cars - they have no air conditioning and it's really loud inside with open windows, especially so in tunnels. That is the kind of stuff that DB sells.

reply
bennofs
43 minutes ago
[-]
reply
gpvos
1 hour ago
[-]
Incorrect. They wouldn't fit in the tiny UK loading gauge (profile). UK trains are indeed variants of continental models, but made to custom size, and many (most?) of them in the UK.
reply
ErroneousBosh
35 minutes ago
[-]
Leide nicht.

Trains in Germany and the UK for main-line running both use 1435mm gauge. UK trains are not a custom size.

reply
LargoLasskhyfv
33 seconds ago
[-]
https://en.wikipedia.org/wiki/Structure_gauge / Lichtraumprofil are different.
reply
phatfish
14 minutes ago
[-]
Everything about UK rail is custom (apart from the gauge). Apparently it's one of the (many) reasons HS2 is such a mess.

They were trying to run trains faster than typical continental high speed lines, which meant custom design work that needs loads of additional testing and certification. Rather than just use the Spanish or French high speed designs.

reply
gpvos
34 minutes ago
[-]
They are. Rail gauge and loading gauge are different concepts. Use Wikipedia.
reply
gib444
58 minutes ago
[-]
> The UK buys most of their trains from Deutsche Bahn (German Rail) and just brands them differently.

This is totally incorrect.

We buy our trains from French/Swiss/German/Spanish/Belgian manufacturers, or build them ourselves in eg Derby.

We do not buy our trains from DB.

reply
gib444
1 hour ago
[-]
There was a GSM-R outage in the UK last month ago too [0]

[0] https://www.railforums.co.uk/threads/nationwide-gsmr-outage-...

reply
ripbozo
1 hour ago
[-]
Word on the german bahn reddit seems to be that a buggy software update is the cause. Remains to be seen if this is the real cause
reply
segmondy
1 hour ago
[-]
AI vibes all the place!
reply
modinfo
1 hour ago
[-]
"IT Outage: No train service nationwide. Due to a nationwide outage of the GSMR digital rail radio system, all trains are being held at stations. We are working around the clock to resolve the issue.

Our technicians are working around the clock to resolve the outage.

Please continue to check your travel connection immediately before departure using the travel information service at bahn.de, the DB Navigator app, or by calling the travel information hotline at 030/2970."

https://www.bahn.de/service/fahrplaene/aktuell

reply
gpvos
43 minutes ago
[-]
Trains are being started up again (staggered because the current draw of so many accelerating trains could cause problems) since about 10 minutes past midnight, 30 minutes ago.
reply
d2kx
1 hour ago
[-]
It's a GSM-R issue. See Tagesschau (German): https://www.tagesschau.de/inland/gesellschaft/deutsche-bahn-...
reply
RetroTechie
2 minutes ago
[-]
I wish someone would do a deep-dive post mortem once everything is sorted out.
reply
wrs
1 hour ago
[-]
>This special mobile communication standard is designed to make communication fail-safe

Mmm, nope.

reply
NamTaf
1 hour ago
[-]
It did fail safe though?

Interference led to the network stopping, not trains just racing towards each other due to bogus line authorities. That is, by definition, fail-safe

reply
ExoticPearTree
1 hour ago
[-]
If nothing works, eveything is safe, no?
reply
fhars
29 minutes ago
[-]
That is the point of failing safe. It would me much worse if some of the trains kept running...
reply
gpvos
1 hour ago
[-]
The fallback for GSM-R is the normal GSM network, but according to informed guesses I've read, the handsets still need to authenticate using their GSM-R credentials (it's just normal GSM roaming), and that's failing too.
reply
lxgr
34 minutes ago
[-]
Very interesting, I always thought it was completely separate infrastructure by design. I wonder what they'll do once commercial GSM networks throughout Europe are shut down?
reply
gpvos
30 minutes ago
[-]
Upgrade to 4G or 5G.
reply
lxgr
27 minutes ago
[-]
In the long term, sure, but https://en.wikipedia.org/wiki/FRMCS is some way away, and I doubt they'll upgrade cab radios to 5G for the fallback path alone.
reply
mfiro
1 hour ago
[-]
It doesn’t surprise me at all. Deutsche Bahn got so bad in the recent years that Switzerland started turning some German trains around at Basel (border) to protect its own timetable from DB delays.
reply
dgellow
1 hour ago
[-]
Any HNer blocked in a DB train who can share with us the experience?
reply
desertrider12
1 hour ago
[-]
I’m sitting in an ICE in Munich that was supposed to leave a few minutes before I saw this story on HN. First the conductor announced a 30 minute delay because the radio wasn’t working, and then they bumped it to 2 hours. They didn’t say it was a systemwide problem.
reply
okanat
1 hour ago
[-]
I would get out and look for a hotel before all of them get sold out. Probably tomorrow too.
reply
desertrider12
55 minutes ago
[-]
Luckily: Update from announcer is that trains can start again at 12:25 AM and they reduced our delay by 30 minutes. But there’s still a huge line of riders at the DB service desk.
reply
brazzy
32 minutes ago
[-]
Had a very similar experience in Munich years ago. That time it was because a train engine on fire on the tracks leading out of the station...
reply
icefo
15 minutes ago
[-]
We departed around 30-45 min late from Basel sbb (in Switzerland) in a night train that goes through Germany.

They told us about the communication issues but what surprised me is that they told us that the Deutsch bahn replaced the locomotive with one of their (that was near the border I guess) so we could depart.

reply
mcbetz
56 minutes ago
[-]
In Erfurt since 2,5 hours. Out of office train driver keeps us updated from chats with fellow drivers (their sources say it is due to software update), radio is fixed now and trains processed one after another (starting with super fast ones - Munich > Berlin, e.g. - so the tracks get emptied quickly). Other interesting observations: when our train stopped, all hotels were already fully booked, as were coach tickets (Flixbus) that would run in the early morning. Crazy how fast people react to shocks.
reply
dgellow
41 minutes ago
[-]
Yeah, I experienced that a few times in airports with massive disturbance. You could see all the hotels getting fully booked almost live, then when you eventually arrive somewhere with a booking you still have to wait an hour due to how long the queue at the hotel reception is. Always a crazy experience
reply
okanat
39 minutes ago
[-]
I am a daily user of S-Bahn. I know 2 alternative routes from every single station from home to work. I even started to memorize their departure times. DB prepares you for the worst.
reply
jtwaleson
53 minutes ago
[-]
I was at a conference in Frankfurt, traveling back to Amsterdam with my cofounder and got stuck in Oberhausen. We have an early flight tomorrow and there's no trains in NL due to a strike tomorrow morning, so we decided to take an uber home.

At first the delay was 30 minutes. Then 2 hours. After 1h30 with zero updates we decided to bail. Just checked and nothing is moving yet, so we made the right call.

reply
dgellow
44 minutes ago
[-]
Yeah, definitely. Hope you can get to your plane on time and that you can expense the uber trip, that’s a pretty long ride from what I see on gmap
reply
gpvos
1 hour ago
[-]
The same as usual I suppose: stopped at a station in a tiny village, without any information. Train staff will provide water, but that's about it.
reply
dgellow
48 minutes ago
[-]
That sucks, sorry for this
reply
gpvos
40 minutes ago
[-]
Don't be sorry for me, I was only relaying earlier experiences.
reply
DanielleMolloy
1 hour ago
[-]
Downdetector shows parallel disruption spikes, similar pattern as end of last year, not as widespread yet. https://downdetector.com
reply
InTheArena
39 minutes ago
[-]
The jet lag team must be in Germany again. Sam, you being deuschbahnned?
reply
mproud
14 minutes ago
[-]
Anyone else watch Season 2 of Hijack?
reply
_def
1 hour ago
[-]
Same problem happened two years ago. You'd think that would be enough time to figure out a failsafe routine
reply
LauraMedia
1 hour ago
[-]
Seems like the failsafe also failed today.
reply
LargoLasskhyfv
6 minutes ago
[-]
It's the heat.

Some system somewhere in need of cooling.

Does not get it, because BAHN.

Crapping out.

Cascades of disbelief.

R2D2-like cybernetic seizures.

Endless commuter pleasures.

No mischief at all.

Just bad techno-thrall...

reply
puttycat
1 hour ago
[-]
A truly chaotic week in Europe, alongside the UK train crash and the unprecedented heat wave.
reply
RetroTechie
14 minutes ago
[-]
It could be chaos on Dutch rail too the coming day. Trimmed down schedule on some of the busiest routes (due to the current heatwave). Add to that a labor union strike on Wednesday June 24 (that's today local time as I write this).

Although NL rail travelers are usually quite good at anticipating, and adjust their schedule. So might just be a quieter-than-usual day.

reply
DanielleMolloy
1 hour ago
[-]
Downdetector shows parallel disruption spikes, similar pattern as end of last year, not as widespread yet.

https://downdetector.com

reply
lschueller
1 hour ago
[-]
Mainly Meta Services, which seem to spike, isn't it? And Google Fiber
reply
DanielleMolloy
1 hour ago
[-]
A bit more, traderepublic, tiktok, snapchat, X, AWS, CloudFlare

https://xn--allestrungen-9ib.de/en/

These could be based on different scales of numbers since it is midnight in Germany

reply
lxgr
29 minutes ago
[-]
And a football world cup game just ended.
reply
lschueller
1 hour ago
[-]
Ah, now it's more obvious.. True! Thank you
reply
pulkitsh1234
1 hour ago
[-]
Interesting, I just took an OBB train today from Zurich to Amsterdam, which passes through a lot of Germany.
reply
gpvos
1 hour ago
[-]
Its return train is currently stuck at Oberhausen.
reply
ratio53
1 hour ago
[-]
I wonder how they managed to tell trains to stop.
reply
lxgr
43 minutes ago
[-]
Stop signals. For legacy train control systems, these still work visually and via wires. ETCS (starting with Level 2) does use GSM-R, but everything is fail-safe: No active communication, no movement authority, so the "virtual signal" display in the cab will pretty quickly also show "stop".
reply
Glawen
1 hour ago
[-]
Deutsche Bahn trains stop themselves all the time, no need to tell them
reply
sc11
1 hour ago
[-]
Signalling still works, so you can let the trains continue to a safe place like a station and then not let them leave until the radio issue is resolved
reply
PLenz
53 minutes ago
[-]
That depends, cab signaling for example needs radio to work
reply
ngruhn
43 minutes ago
[-]
Is that news? Sounds like status quo.
reply
hdgvhicv
1 hour ago
[-]
Can passengers tell, I thought German trains were always disrupted!
reply
mmoll
1 hour ago
[-]
It is telling that I thought “that’s why all trains were late this afternoon” before I realized that the issue occurred only minutes ago.
reply
porphyra
1 hour ago
[-]
reply
usernametaken29
1 hour ago
[-]
Honestly can’t tell the difference between this and a regular day r/dbsucks
reply
ChrisArchitect
1 hour ago
[-]
reply
sva_
1 hour ago
[-]
@dang please update if you see this

And merge with https://news.ycombinator.com/item?id=48651613

reply
lyu07282
1 hour ago
[-]
Happened before at a smaller scale, crazy high redundancies in GSM-R mean this is likely sabotage:

https://en.wikipedia.org/wiki/October_2022_German_railway_at...

https://en.wikipedia.org/wiki/GSM-R

reply
okanat
1 hour ago
[-]
I wouldn't be surprised if it turns out to be incompetence at this point.
reply
sleepybrett
1 hour ago
[-]
vibe coding the rail software.
reply
Etheryte
1 hour ago
[-]
If you know anything at all about the Deutsche Bahn, you'll know that it's most likely self-sabotage, in other words, incompetence.
reply
mschuster91
49 minutes ago
[-]
Current suspicion on the German rails reddit is a software update gone wrong.

My personal suspicion, GSM-R is 90s GSM, they'll likely have a fried HLR & VLR because in any GSM network these are fundamental, without them you can't even get roaming from public phone networks working as there is no way for the public network to authenticate GSM-R subscribers.

reply
dgellow
1 hour ago
[-]
reply
fhdkweig
1 hour ago
[-]
and submitted by the same user sva_

https://news.ycombinator.com/submitted?id=sva_

reply
gus_massa
1 hour ago
[-]
The other article is only one sentence long. I guess the OP posted it and later found this that is a better source. (Un)luckily both reached the front page. Usually dang/tomhow will make a cleanup soon to avoid duplication and keep the discussion in a single thread.
reply
Havoc
1 hour ago
[-]
Gee I wonder which country could be behind it
reply
moffkalast
1 hour ago
[-]
It's either that or starlink, some railroads in Germany go through areas without any mobile network signal. Think about how crazy that is in 2026 when everything expects everyone to be online 24/7/365.
reply
hdgvhicv
1 hour ago
[-]
They aren’t using starlink for safety critical comms
reply
ortusdux
1 hour ago
[-]
It's my understanding that most rail/rail collisions are the result of poor communication.
reply
t0mas88
1 hour ago
[-]
The railroads have their own mobile network, GSM-R, it's in the article...
reply