TS-2026-009: Insecure argument handling in Tailscale SSH permitted root access
153 points
9 hours ago
| 14 comments
| tailscale.com
| HN
tptacek
8 hours ago
[-]
This is such a venerable and ancient class of bugs, going at least as far back as AIX 3. Glad to see they're still makin' 'em like they used to.

(If you had SSH access to a host in your Tailscale ACL, you could log in as `-i` and get a root login.)

reply
unprovable
25 minutes ago
[-]
It's so old skool, it's almost new again... almost... Still waiting for that third (or fourth?!) wave of XXE (or similar) bugs.
reply
RossBencina
7 hours ago
[-]
I'm somewhat alarmed that the context that this bug was running in was capable of root login. Is there a reason that an SSH login process would, by default, have enough capabilities to facilitate direct root login?
reply
anilakar
3 hours ago
[-]
We did Tailscale-like SSH reverse tunnels at scale first in 2013 and the main issue has always been that there are no good libraries. Bash scripting around the OpenSSH binaries is pretty much the only way to go.

There's Paramiko, but Python is still a huge liability in memory-constrained systems.

reply
formerly_proven
1 hour ago
[-]
libssh, libssh2. These are totally independent and unrelated code bases, libssh is maintained by Red Hat mainly for ansible and some other tools, libssh2 was created for curl. libssh2 is client-only, libssh can also be used to implement servers.
reply
yjftsjthsd-h
6 hours ago
[-]
If it runs as your user and can only log in as you, then I wouldn't expect it to be able to become root. But if it can log you in as different users, I would expect that 1. it needs to run from root, and 2. it can log in as root.
reply
iririririr
7 hours ago
[-]
you can also add parameters to env vars in some popular cloud providers for the same effect.
reply
doublepg23
8 hours ago
[-]
I’m a heavy Tailscale user, so I do trust them quite a bit, but I never used the Tailscale SSH feature. I feel like OpenSSH’s security record is pretty unbeatable, not sure why I’d swap over for such a security-sensitive tool.
reply
OJFord
3 hours ago
[-]
The SSH vulnerability here only applies if the attacker is already on the network. It violates your Tailscale ACLs, but it's not arbitrary external root ssh access. Arguably that's a more secure starting point than vanilla ssh to publicly accessible machine.
reply
semi-extrinsic
2 hours ago
[-]
OTOH, if you run vanilla ssh on a publicly accessible machine where only port 22 is open, sshd only allows publickey-based authentication and the only accepted key types are FIDO2/U2F hardware-backed keys, it's probably more secure again (less attack surface).
reply
fodkodrasz
2 hours ago
[-]
With a plain VPN like WireGuard when they get access to your network, they don't have plain ssh, not to mention root ssh access to hosts. This is a serious issue.
reply
bakies
7 hours ago
[-]
Yeah pretty much just use tailscale as a vpn.. do one thing as they say.
reply
jdiff
8 hours ago
[-]
I've used it before to access my tailnet machines through a browser on a machine I can't download software on.
reply
Shorel
4 hours ago
[-]
I just don't use stranger's machines to access my personal stuff. Possibly compromised stranger's machines. I don't see the benefit about that, as I have more laptops than I need.
reply
dgacmu
7 hours ago
[-]
I used it for a bunch of remote monitor boxes to have a way of centrally managing ssh access to things that were often on- and off-line. It was simple and convenient and access was easily revocable.
reply
isatty
8 hours ago
[-]
Convenience for the most part but in general, I agree. I like having it as an option.
reply
jcarrano
1 hour ago
[-]
> usernames were passed as arguments to getent(1) to retrieve the corresponding passwd entry

Always try to use actual API/system calls (in this case getpwnam) instead of calling sub-processes.

reply
eviks
3 hours ago
[-]
> Tailscale SSH now rejects usernames with leading dashes.

Is the proper fix not restricting users not possible in these poorly designed ancient systems?

Similarly re another issue: why not just fix the permission issues instead of restricting users?

> Tailscale now disallows the use of UIDs or numeric-only usernames via SSH to avoid this ambiguity

reply
pferde
1 hour ago
[-]
The proper fix would be to not use getent CLI tool in their logic, but instead use proper system APIs for looking up user account entries, like one of earlier comments here already mentions. This is shocking amateur hour!

My guess is they hastily threw together something hacky in early development, and forgot to replace it with a real, safe solution later.

reply
rfoo
40 minutes ago
[-]
The issue here is there is NO single system API for looking up user account entries on Linux.

It's implemented in libc. So you need to link to libc. Tailscale is a Go binary, and they probably prefer it to be statically-linked. glibc NSS implementation also REQUIRES you to load `.so` so you just can't emulate it in Go.

Then, "link to libc". Which libc? glibc? musl?

reply
pferde
22 minutes ago
[-]
But of course there is, it's part of POSIX, implemented in libc. And if you're using a higher level language, they all have their own wrappers around libc/POSIX APIs. Here is golang's: https://pkg.go.dev/os/user
reply
Chu4eeno
34 minutes ago
[-]
> The issue here is there is NO single system API for looking up user account entries on Linux.

Yes there is, and you answered in the next line, it is implemented in libc.

If you want to check authentication use libc don't try to implement crypto and authentication yourself.

reply
sam_lowry_
1 hour ago
[-]
There is a whole class of security issues where fixes are worse that the issues themselves. Case in point, the OpenSSH itself that sends 100 packets on each keystroke to avoid timing attacks.
reply
fragmede
1 hour ago
[-]
Why is that worse than the issue itself? If someone could figure out, say, my root password via an ssh timing attack, that seems bad. Sending 100 packets for each keystroke to protect against this seems cheap in the face of that.
reply
drnick1
6 hours ago
[-]
I'll stick to my 100% self-hosted Wireguard setup, thank you very much.
reply
m_mueller
6 hours ago
[-]
Why not tailscale plus head scale for self hosting?
reply
diarrhea
4 hours ago
[-]
I do not understand this rebuttal.

I also run self-hosted Wireguard. Initially on a Debian box, nowadays it is integrated into my router (admittedly, this is closed source). For around 6 years at this point.

The whole thing could not be easier and simpler. It has never randomly broken on me. It is fast. It is free. There is no middle man, no vendor.

I never understood the popularity of Tailscale, though that is on me. I'm sure it is a great product, I just never tried it, do not seem the target audience.

What confuses me is the often accompanying, sometimes aggressive anti-selfhosting stance in these sorts of threads. I do not see this in other topics, e.g. someone mentioning they run Jellyfin isn't met with "why not Plex?". Where does that come from? We are on HackerNews, not ProductShillNews, aren't we? I guess self hosting Wireguard is too boring to warrant any further discussion? The VPN equivalent of a Toyota Corolla.

reply
MobiusHorizons
4 hours ago
[-]
I think Tailscale is popular because of how plug and play it is for most people. Although the main reason I use it over self hosting wireguard is the NAT busting it does, which has so far worked flawlessly for me with no setup aside from installing on both devices. There is nothing wrong with self hosting wireguard, but it doesn't actually do the same job as tailscale.
reply
mr_mitm
2 hours ago
[-]
Wireguard by itself also doesn't allow for 2FA or expiring keys. Not as relevant for private use, but some orgs need it for compliance. The idea was always that things like that need to be implemented by an application on top of it, so you end up with something like tailscale eventually.
reply
mfru
3 hours ago
[-]
as someone who uses tailscale: exactly this.

i have my homelab only reachable via tailscale and can access everything i would ever want on the go that way. it was a matter of 15 min to get it all working.

reply
diarrhea
4 hours ago
[-]
NAT busting is a great point.
reply
hdgvhicv
1 hour ago
[-]
My WireGuard uses (either at home or at work) are very much mobile client to single network

Where Tailscale comes into its own is automatic managing of mesh networking (like an “sdwan” solution). The other thing it excels at is firewall busting - if you have a firewall (with or without address translation) which only allows outgoing traffic to be established (with UDP timeouts for session) then Tailscale also works in a similar way to turn/stun.

If I needed that capability then I’d be looking at Headscale. I don’t need it though.

Remember that this is hackernews, not slashdot. Where the community used to be far smaller and the technology far smaller it was quite normal for everyone to understand basic building blocks of ip addresses, use open source software, wear t-shirts threatening to replace people with a small shell script etc.

It’s not the same community, many people here have no real understanding of computer fundamentals, but instead have expertise in specific narrow areas. They also have little interest in things like free software, but do have an interest in building a new billion dollar company to sell to a behemoth.

reply
xorcist
34 minutes ago
[-]
> it excels at is firewall busting

Some would consider that an anti-feature. Firewalls are not to be busted. Nothing good lies at the extreme end of working around overly strict policies. Change the policy instead.

reply
m_mueller
2 hours ago
[-]
it wasn't meant as a rebuttal, I'm genuinely asking. tailscale + headscale was just recommended to me, hence that's what I'm using for self hosting. is wireguard's client roughly equivalent to tailscale's? especially tailscale's always-on nature is very appealing.
reply
fragmede
3 hours ago
[-]
How do I install wire guard on my mom's Apple TV?
reply
apexalpha
2 hours ago
[-]
I also tested tailscale, headscale and netbird and found all these drain your battery on mobile.

Others report no issues but I had massive drain on iOS even with only 4 connections open.

Native wireguard is unnoticeable.

reply
nicman23
4 hours ago
[-]
i really dislike that there is no way to do dhcp for new clients and that i have to manually define peers in each "exit node"
reply
gear54rus
1 hour ago
[-]
Because wireguard to tailscale is like git to git GUIs. It's solid base but never should be used separately without a proper wrapper if one wants to keep one's sanity.
reply
dust-jacket
18 minutes ago
[-]
But are there any good self-hosted wrappers?

I ditched wireguard for tailscale for the ease of managing it. I'd much rather run my own independently but CBA with the config editing hassle.

reply
doctorpangloss
5 hours ago
[-]
haha self hosted wireguard, an opportunity to find out AllowedIPs: 0.0.0.0/0 does the opposite of what you think it will do
reply
sva_
1 hour ago
[-]
If you don't care to invest half an hour into learning some basics of how computer networking and in particular CIDR notation and subnet masks work, maybe it is not for you.
reply
hdgvhicv
1 hour ago
[-]
Allow any (ip4) traffic to enter the tunnel, install a route in the default routing table to make that happen (well the second depends on the client)

Does it do the opposite of that?

reply
s_ting765
2 hours ago
[-]
I don't see the point of publishing a security bulletin if you are not going to timely push the fix to artifacts on all affected platforms. Tailscale needs to do better on their release process, docker hub shows last update was 8 days ago.
reply
dust-jacket
15 minutes ago
[-]
I don't mind having a bulletin so much as the claim that it's fixed in 1.98.9 or newer, when that release doesn't appear to exist yet. Feels pretty weird practice to advise upgrading to a non-existent version.
reply
dijit
1 hour ago
[-]
so you can disable or mitigate it…
reply
luciana1u
6 hours ago
[-]
tailscale ssh: replacing a 25-year-old battle-tested codebase with a startup's Go rewrite and then acting surprised when it has bugs
reply
e40
8 hours ago
[-]
So, giving access via tailscale but using OpenSSH is safe, right?
reply
lugoues
8 hours ago
[-]
Yes, this only involves their wrapper that is managed by ACL rules.
reply
iririririr
7 hours ago
[-]
as much as handing control to a remote third part is, yes.
reply
cevn
6 hours ago
[-]
Good point. I self host headscale but it also has the ssh feature, probably also insecure.
reply
OJFord
3 hours ago
[-]
Not necessarily, it's a clean room implementation. Even if leading dashes was known/documented/tested to implement they might have done it differently. And maybe it was an implementation detail that it was ever allowed, but that's a weird username, headscale implementation happened not to allow it, and nobody ever noticed the discrepancy.
reply
modeless
8 hours ago
[-]
Tailscale SSH has caused me other problems in the past because it takes over port 22. I'm not a fan.
reply
newdee
2 hours ago
[-]
It takes over port 22 on the Tailscale interface only. Only had problems with this when I’ve wanted to hit a host’s non-Tailscale ssh service via Tailscale. Otherwise it’s been great for me
reply
LeoPanthera
7 hours ago
[-]
That is what it's supposed to do, though. It's not a secret.
reply
mintflow
6 hours ago
[-]
>>> We would like to thank Anthropic and Ada Logics for reporting this issue.

it seems anthropic also use tailscale or it's just being discovered by the mythos model?

reply
tristanj
5 hours ago
[-]
I presume Ada Logics has access to Anthropic's Mythos model via Project Glasswing, and Ada Logics discovered this exploit during their vulnerability research.
reply
mintflow
6 hours ago
[-]
pure logic error, the undergoing tailscale rust rewrite can't help this too:)
reply
bestouff
4 hours ago
[-]
If it used one of the standard arguments-handling crates (e.g. "clap") there's no way it can happen.
reply
pferde
1 hour ago
[-]
If it used one of the standard system APIs for looking up user accounts (e.g. getpwnam(3)), there's no way it can happen.

This is incredibly bad engineering, on level of a SQL injection, in 21st century. Something a highschool student experimenting with scripting could come up with, but not a supposedly professional software company.

reply
bestouff
43 minutes ago
[-]
Agreed.
reply
mintflow
6 hours ago
[-]
that said, the limit impact perhaps is only affected multiple users in tailnet if the ACL is not configure correctly

As single tailnet+single user, perhaps it's just okay

reply
kbumsik
8 hours ago
[-]
Why own numbering instead of CVE?
reply
vngzs
8 hours ago
[-]
It lets organizations (Tailscale) control the timing and narrative around the disclosure more directly. Organizations sometimes avoid the bureaucracy of going through CVE Numbering Authorities by self-publishing. Often a CVE assignment follows self-disclosure, especially when there's pressure to interoperate with vuln-scanning/compliance tooling
reply
bigfatkitten
7 hours ago
[-]
And sometimes it’s just impossible to get a CVE number in a reasonable amount of time, or indeed at all.
reply
wereHamster
5 hours ago
[-]
Some reasons why an org might want to become their own CNA: https://daniel.haxx.se/blog/2024/01/16/curl-is-a-cna/
reply
cyberax
8 hours ago
[-]
> "Tailscale SSH now rejects usernames with leading dashes."

Really? That's the fix?

A proper fix is to use "--" to separate arguments.

reply
valleyer
7 hours ago
[-]
A proper fix is not to shell out to a command at all; use getpwnam(3) or similar.
reply
catlifeonmars
5 hours ago
[-]
“--“ doesn’t work on all versions of getent.

A better fix is to call “getent passwd” with no user controlled arguments and then parse the resulting list. This gets rid of the input sanitization problem entirely.

reply
cyberax
4 hours ago
[-]
Are there any actual systems that can run Tailscale and that have faulty getent?
reply
sedatk
8 hours ago
[-]
Their fix just future-proofs it in case the same bug gets reintroduced.
reply
turbert
6 hours ago
[-]
A correct implementation would be to just call glibc directly, this seems like a hasty fix to get the patch out the door. The history of vulns from bad shell escaping is as old as bash, whenever possible you probably shouldn't be mixing code and data, especially in a security critical application like this.
reply
raggi
5 hours ago
[-]
The fact that there is no portable way to link the relevant functions that works reliably across all distributions of Linux is a failure of POSIX and GNU, and unfortunately is largely the Linux distribution story in a nutshell.

Your answer is mostly correct, except that when you tug on that thread the shelf comes off the wall, the plaster comes with it, and then it cracks the water pipes on the way to the floor.

reply
cyberax
7 hours ago
[-]
This is just a dirty fix. It adds weird restrictions and masks issues.

Refactoring external invocations to use safe argument handling is a better way to fix it. Along with tests that exercise weird names.

reply
raggi
5 hours ago
[-]
reply
cyberax
1 hour ago
[-]
All the hallmarks of an LLM fix right there...
reply
sedatk
7 hours ago
[-]
I argue the opposite: there’s no better fix for this. You can write the most elegant fix, whatever it is, and prevent that from happening only on the codebase that’s fixed. That doesn’t mean that the codebase will always be the only authority on authentication.

The username policy fixes this issue for good, regardless of whatever you write in the future, or whatever new mechanism is introduced.

It’s a restriction for sure, but it’s not a nonsense restriction? Who would have a username starting with a hyphen? I didn’t even know it was possible until today.

reply
turbert
6 hours ago
[-]
> I argue the opposite: there’s no better fix for this

The better fix would be to not have the username pass through a parser looking for cli flags in the first place.

reply
OJFord
3 hours ago
[-]
How do you propose to do that when username is a possible argument?
reply
farfatched
5 hours ago
[-]
Sadly, yet another path to root via Tailscale.

If their scope grows, and they run so much as root, it won't be their last.

reply