Porting Tailscale to Plan 9
349 points
1 day ago
| 20 comments
| tailscale.com
| HN
bradfitz
1 day ago
[-]
Happy to answer any questions!

A bunch of us are currently in https://meet.google.com/qre-gydb-mkv chatting about this. (Edit: the hour is over; we all left)

The earlier Apr 1st blog post was https://tailscale.com/blog/tailscale-enterprise-plan-9-suppo...

reply
undersuit
23 hours ago
[-]
I've never set up a Plan 9 system... does this allow the distributed systems communications to run through my Tailnet?
reply
MisterTea
22 hours ago
[-]
Yes, you could do something like keep a small root fs or pack everything into the kernels paqfs to boot into a Tailscale VPN and pull root from another 9 machine on the VPN. Then pull resources in from other machines including non 9 systems.

Either way it makes VPN easy between 9 and non 9 machines. Otherwise Plan 9 can do it's own VPN-like over tls or ssh tunnels and bind remote network stacks to a local namespace. But that makes seamless Unix and Windows comms difficult.

reply
bradfitz
22 hours ago
[-]
> Otherwise Plan 9 can do it's own VPN-like over tls or ssh tunnels and bind remote network stacks to a local namespace

Note that one of Tailscale's main party tricks is NAT traversal, when both machines are behind different NATs and can't otherwise get a connection open to each other. (And then Tailscale ultimately falls back to a relay server on the internet if it can't get a direct connection for IP packets)

reply
MisterTea
21 hours ago
[-]
For situations where you have no control over the NAT then this is indeed the case.

Though, 9front lets you run your own NAT giving you an Internet facing 9 machine you can serve a TLS tunnel from directly. So the server side is solved making the client side NAT a non issue.

reply
bradfitz
21 hours ago
[-]
If your 9front machine is in a position on the network whereby it could serve a NAT, you don't have many networking problems at that point. Almost all operating systems can do NAT in such a position.

I'm talking about two machines deep in somebody else's network or where you don't control the router/NAT.

reply
bradfitz
23 hours ago
[-]
I think so! Caveat is I've never really used Plan 9 outside of single-user VMs.
reply
mfro
1 day ago
[-]
Russ Cox is an absolute legend for committing to this joke.
reply
pbohun
23 hours ago
[-]
Someone needs to convince Russ that it would be hilarious to have a full featured web browser in Plan 9.
reply
MisterTea
22 hours ago
[-]
On 9 front there's vmx which is hardware virtualization. You can boot a Linux kernel with an nfs root from the local machine and use headless vnc to run a browser in a vnc client window.

I'd also like to point out that most users of Plan 9 dislike web technology because it's a giant nightmare of code. No one human can even begin to comprehend the code base of Chrome, let alone Firefox - programs that are as big, if not bigger than the kernels they run on. That is an absurd state to be in - your runtime requires a billion dollar company to maintain. Even open source Firefox needs millions in funding.

Whereas a single human can grasp plan 9 code from the kernel to user space. That's the runtime I want, something I can understand. The process is the container on plan 9 so you have everything you need to build distributed apps without a web browser. It's human scale distributed computing. I'd like a future without the "modern" corporate scale web.

reply
pbohun
21 hours ago
[-]
Oh yes I absolutely agree. I would definitely like to completely replace the web. It's just that in order to (currently) do my banking, pay my bills, book airline tickets, order from Amazon, etc. I must use a browser. If I could escape all that I would run Plan 9 exclusively without another OS or hacks to access a browser from another OS/virtual machine.
reply
pjmlp
6 hours ago
[-]
Even though most of the UIs I work on nowadays are Web based, I miss the native apps with Internet protocols, and most of my side projects are native apps, nothing to do with Web.
reply
MisterTea
21 hours ago
[-]
Totally get it. Vmx on 9front with a Linux or BSD VM is the way to go if you want to try to go 100% 9. If you like you can experiment with this on a used laptop with supported hardware, thinkpad best. It's not lightning fast at the moment (patches welcome) but it works well enough.
reply
facile3232
20 hours ago
[-]
> If you like you can experiment with this on a used laptop with supported hardware, thinkpad best.

I would avoid laptops altogether, honestly. Not a great fit.

reply
MisterTea
17 hours ago
[-]
That is the opposite of good advice. http://fqa.9front.org/fqa3.html#3.1
reply
facile3232
16 hours ago
[-]
I suppose it depends on what you want from 9front. If you want a terminal with decent power management you should just buy a mac.
reply
naikrovek
13 hours ago
[-]
> a single human can grasp plan 9 code from the kernel to user space.

Is that true? I cloned the 2015 release of plan 9 a week or so ago and it had around a million lines of C. Can a single person hold all of that? I sure as hell can’t.

reply
MisterTea
2 hours ago
[-]
You can, just not all at once.

And which plan 9 release and when? Ghostscript and Python were originally distributed with 9front which are both HUGE compared to the rest of the system. Remove those and its much, much smaller. Unsure if ghostscript was included in vanilla 9 from the labs. Python was included in 9front because it was necessary for mercurial. Once git9 arrived python was nuked from base and removed many lines of code. Ghostscript is next to go from base once pdffs is running (patches welcome.)

reply
facile3232
20 hours ago
[-]
> You can boot a Linux kernel with an nfs root from the local machine and use headless vnc to run a browser in a vnc client window.

Not only is the VNC redirection unnecessary, so it is the entire filesystem. You could just render the vm directly to the window and boot a read only image. Plus then you don't have to deal with VNC.

reply
fiddlerwoaroof
23 hours ago
[-]
Doesn’t plan9 support frame buffers over 9p or something like that? You could probably write a wrapper that just forwards a Linux browser to a plan9 window
reply
moody__
23 hours ago
[-]
This has been done already: https://github.com/aiju/jsdrawterm
reply
samtheprogram
23 hours ago
[-]
This looks like the opposite -- accessing a plan9 system from a web browser?
reply
moody__
22 hours ago
[-]
Yes you're correct, my apologies. There has been work on this going the other way as well: https://github.com/michaelforney/wl9. But there's still a lot more than can be done. There are vague plans to test the waters implementing something like this in to our vmx(1).
reply
fiddlerwoaroof
23 hours ago
[-]
I think this is the opposite direction?
reply
adriangrigore
22 hours ago
[-]
There are solutions, like VNC to some UNIX-ish machine, but, yeah, a native browser would be cool! 9front has a hypervisor, you could run something in there. https://man.9front.org/1/vmx
reply
fiddlerwoaroof
20 hours ago
[-]
So, something I’m thinking about here is that the 9p vision has always seemed really cool to me: expose all the resources in the network in a unified way that enables the whole network to be used as if it was a single computer. But, since this is a protocol-oriented vision of computing, it enables arbitrary implementers of the protocol to participate “natively”, even if they aren’t actually plan9 systems.
reply
numbsafari
21 hours ago
[-]
Many years ago a roommate and I had an HPUX machine running IE on HPUX just so we could forward X session to our FreeBSD and Linux desktops and not have to use our Windows machine for anything other than PC games.
reply
facile3232
20 hours ago
[-]
It's probably easier to just run a VM directly into a window.
reply
adriangrigore
22 hours ago
[-]
Yeah, convince Russ and some investors! :D I would laugh my ass off for years at this joke! Yeah, please do this next year's April Fools'!
reply
lunarlull
17 hours ago
[-]
> I would laugh my ass off for years at this joke!

I don't really get the 'joke'? Porting a full web browser to Plan 9 would seem like a cool project - where's the humor?

reply
adriangrigore
16 hours ago
[-]
You're making me explain this whole post. :p

Their April Fools' jokes are real and work as you can see in the submitted link.

So basically, a Plan 9 web browser, would be a great April Fools' prank! (because, again, their "pranks" are real and work)

reply
lunarlull
11 hours ago
[-]
Ah thanks for explaining lol, got it!
reply
facile3232
20 hours ago
[-]
Atp it's probably easier to just smash firefox/linux into a vm
reply
packetlost
1 day ago
[-]
I unironically wish there was an enterprise version of Plan 9. I've been writing most of my scripts in `rc` (something my coworkers put up with because we use nix and I can pull it in automatically with dirnev) and it has been great.
reply
yjftsjthsd-h
23 hours ago
[-]
I would worry less about other people being able to run rc scripts and more about them being able to read/edit them.
reply
packetlost
23 hours ago
[-]
they're routinely very short, and the only non-obvious syntax for someone familiar with a C-like language is the ~ command and redirecting to stderr. They're pretty much always easier to read (and write) than bash scripts in general because of how little weird/surprising syntax there is. Not being a derivative of ALGOL has its perks.

Most scripts are write-once:read-never, especially if you actually implement -h/--help

reply
eddythompson80
23 hours ago
[-]
> Most scripts are write-once:read-never, especially if you actually implement -h/--help

I guess the answer is always “it depends”, but that generally has never been my experience with most things. Are you over-engineering the shit out of every script to the degree the script itself is a Turing complete machine and with enough —-help flags anything is possible? Most 40+ year old Unix tools with a thousand flags have their limits and you have to script around them to achieve things you want.

In my experience, eventually a business need will arise that require you to change a script. Are your coworkers comfortable changing these scripts or are you in the mind set of “that’s a simple enough change, I’ll do it”

reply
packetlost
19 hours ago
[-]
> Are you over-engineering the shit out of every script to the degree the script itself is a Turing complete machine and with enough —-help flags anything is possible?

No. They're 30 line scripts with 0-5 or so flags. They mostly exist to remove choices from other utilities. Put another way: create named (and namespaced) abstractions by making choices and slapping a name on it. They're functions.

> In my experience, eventually a business need will arise that require you to change a script. Are your coworkers comfortable changing these scripts or are you in the mind set of “that’s a simple enough change, I’ll do it”

They're small enough that they can be ignored if they don't do exactly what you want. I'm fine changing them, but it's most likely they'd just get rewritten or bitrot after I'm gone.

reply
packetlost
19 hours ago
[-]
Not work related, but here's some examples of what these scripts look like (this is how I mirror public repos locally using cron/timers): https://paste.sr.ht/~chiefnoah/3b8990fc0b8eb3f50e511d5d4051a...

Even if you aren't super familiar with rc, it's not that weird to look at. I find it way more readable than (ba)sh syntax.

reply
kristianp
14 hours ago
[-]
One benefit of rc is this[1]:

> The most important principle in rc’s design is that it’s not a macro processor. Input is never scanned more than once by the lexical and syntactic analysis code

I worked at a unix shop that deleted most of a working drive because a shell script was modified while it was running. Luckily they kept daily backups on tape. This was about 17 years ago.

[1] https://www.scs.stanford.edu/nyu/04fa/sched/readings/rc.pdf

reply
LukeShu
7 hours ago
[-]
Scanning input just is unrelated to the "modified while running" problem. The "modified while running" problem is a read-buffering problem.

For example, consider the following change:

    -echo $x; rm -rf /n/foobar/
    +rm -rf /n/foobar/
     ^^^^^^^^^^^^^^^^
If the shell's first read() reads 16 bytes (indicated above with "^"), then the file is changed, then the shell reads the rest; then the shell will see "echo $x; rm -rf /" regardless of whether or not it scans the input multiple times.

I am unfamiliar with the read-buffering done by either of the 2 main implementations of rc, and so am unable to comment on whether it does things to avoid this problem. But if it does do things to avoid it, those things are orthogonal to the "not a macro processor / input is never scanned more than once" thing.

reply
moody__
23 hours ago
[-]
Could you expand more on what you would like out of an "enterprise Plan 9"?
reply
packetlost
21 hours ago
[-]
the distributed computing model is pretty nice in theory (maybe not in practice) and the uniform system APIs are also nice. The userspace tools in particular are just plain better (structured regex commands are quite a bit better than ed-style and I find myself using them far more frequently in vis than I do in vim, they're far more composable and intuitive).

The biggest thing is the heavy reliance on union file systems (and file systems in general) and an extremely simple syscall API. It's a heterogeneous-networked-node OS so it handles realistic workloads natively with primitives designed for it instead of piling complexity on top of Unix-like APIs (ie. Linux). I dunno, I just think a lot of the modern "cloud native" stack is unnecessary if you had an OS actually built for the workloads we have.

reply
moody__
21 hours ago
[-]
There aren't really union filesystems per se, the plan 9 kernel provides unions through its namespace model. In my opinion part of the reason why the userspace tools can be as nice as they are, are due to the use of file system interfaces and the simplistic syscall API. Could you elaborate more on the issues you see with the use of these?

In regards to using it for a "cloud native" stack, the issue is that people want to run code that isn't designed for Plan 9. You could build whatever backplane type thing you want out of plan 9 but the end goal is still likely to be to run some web app or REST api server. Unless someone does a great deal of effort to port all of those environments that people want (nodejs, modern python, etc) you're going to be stuck using a VM and losing a lot of the benefit.

This feels similar to what Joyent did with lxzones in SmartOS, where the backplane was solaris based but the apps they were running for clients were using Linux. It's hard to make the plan 9 backplane better enough to warrant dealing with integrating the guest and host environment.

reply
zozbot234
19 hours ago
[-]
> Unless someone does a great deal of effort to port all of those environments that people want (nodejs, modern python, etc) you're going to be stuck using a VM and losing a lot of the benefit.

It should not be a huge deal of effort since as you mention the plan9 syscall API is simpler than on Linux. The added plan9 support could then also serve as a kind of "toy" backend that could make the rest of the code more understandable in other ways.

I'd even argue that OP's early experiment with such a port of tailscale shows precisely such an outcome.

reply
adriangrigore
16 hours ago
[-]
The problem is porting the compilers! There is no C++ compiler on Plan 9.
reply
packetlost
13 hours ago
[-]
Yeah, but also the type of person who is going to use Plan 9 isn't going to want to write C++, of all languages.

The general problem stands though, almost no languages support Plan 9

reply
packetlost
19 hours ago
[-]
> There aren't really union filesystems per se, the plan 9 kernel provides unions through its namespace model.

Yes, this is what I'm referring to. It's really many filesystems unioned into one namespace that is controllable per-process.

> In my opinion part of the reason why the userspace tools can be as nice as they are, are due to the use of file system interfaces and the simplistic syscall API. Could you elaborate more on the issues you see with the use of these?

I didn't say I had any issues, I said I preferred them! Aside from a lack of familiarity and needing to install plan9ports on other systems, I haven't had issues.

> In regards to using it for a "cloud native" stack, the issue is that people want to run code that isn't designed for Plan 9. You could build whatever backplane type thing you want out of plan 9 but the end goal is still likely to be to run some web app or REST api server.

Right, language support is the biggest issue with running on Plan 9 from that perspective, at least for "server" workloads. Excluding graphical APIs, the basic stuff (file IO, networking, etc.) isn't all that hard to add to a language (it of course depends). The real trouble is things that have no equivalent in Plan 9, such as mmap and shm.

> This feels similar to what Joyent did with lxzones in SmartOS, where the backplane was solaris based but the apps they were running for clients were using Linux.

This is also what Oxide is doing. Their rack's OS is IllumOS but their customers are expected to only interface with the OS via their tooling and instead provision VMs.

> It's hard to make the plan 9 backplane better enough to warrant dealing with integrating the guest and host environment

If I were doing it, I would do it the other way! Run Plan 9 in a backplane/hypervisor and target it from the language level. The nice part is the systems programming model!

reply
zozbot234
18 hours ago
[-]
> Excluding graphical APIs, the basic stuff (file IO, networking, etc.) isn't all that hard to add to a language (it of course depends).

You could implement a modern graphical API on top of virtio-gpu, which would give you low-level access to accelerated graphics.

> The real trouble is things that have no equivalent in Plan 9, such as mmap and shm.

Some uses of mmap and shm actually seem to have a near-equivalent already in plan9's segattach. Other uses would require some implementation of distributed shared memory (i.e. implementing the usual CPU concurrency model over the network) to be made feasible while keeping to the usual networked-OS focus of plan9.

reply
packetlost
17 hours ago
[-]
Right, you could do it... maybe. Some languages/libraries/runtimes could have specific expectations around the specifics of mmap that can't easily be papered over, but I suspect it would be a minority of cases
reply
zozbot234
21 hours ago
[-]
It could be used to replace k8s-based deployments (also Docker Swarms, etc.) since system interfaces on Plan 9 are namespaced and containerized "out-of-the-box" as part of its basic design (and this is one of the most prominent additions compared to *NIX). It's not a hacked-on feature as with Linux.
reply
raggi
21 hours ago
[-]
In case y'all missed it in the first post, and you just want to try this out, it's working in this v86 image:

https://copy.sh/v86/?profile=custom&m=768&vram=16&hda.url=ht...

You can start tailscaled and tailscale inside the VM. It may take a while to come online sometimes due to limited proxy availability.

Edit: alt gives you the third button. To start a terminal, hold alt and right click, select new, release alt, and right click drag to size the terminal window.

reply
adriangrigore
1 day ago
[-]
Webinar in progress (Google Meet) https://ftp.plan9.ts.net/webinar
reply
packetlost
23 hours ago
[-]
It just wrapped up, for those who would have otherwise been interested.
reply
0xbadcafebee
1 day ago
[-]
I like the premise of the joke, but then as the explanation ran on... I suddenly became depressed. So much broken stuff, so much complexity.... to, what, make a network tunnel? If all this extra work was the joke, that would be funny.
reply
rsc
1 day ago
[-]
We had to do some Plan 9 work, which makes sense when doing something new, but the actual Tailscale implementation is far _less_ work than for other Unixes.
reply
badc0ffee
23 hours ago
[-]
It sounds like the Go compiler is better after this effort - fewer Plan 9 special cases in the code.
reply
kanwisher
1 day ago
[-]
wholly cow was not expecting them to patch the plan9 kernel to make this work
reply
nasretdinov
21 hours ago
[-]
Why not though? Seems like relatively little amount of work was missing since clearly no one seriously done something like this before :)
reply
renhanxue
1 day ago
[-]
> In 1999, Intel introduced the Pentium III processor with SSE instructions.

I kinda expected this paragraph to continue with

> This has made a lot of people very angry and been widely regarded as a bad move.

reply
o11c
1 day ago
[-]
Better than MMX at least.
reply
breckinloggins
21 hours ago
[-]
God I love plan9. Making my own os using many of its principles is a retirement project life goal.

EDIt: I reserve the name “chaos10” for this project, since - like SerenityOS - there will be no plan.

reply
fultonb
23 hours ago
[-]
This is so cool to see. Plan9 was a wonderful part of my COVID isolation, and I miss playing with it. This might have inspired me to spin up a 9front VM this weekend.
reply
bradfitz
22 hours ago
[-]
Note that the 9front patches to run Tailscale are still in progress. I was just told they'll be ready in a couple weeks.

For now only 9legacy (with all the latest changes) works.

reply
MisterTea
22 hours ago
[-]
> This might have inspired me to spin up a 9front VM this weekend.

Please do! Just be careful with your sysupdate.

reply
mcdow
1 day ago
[-]
Rob Pike is in shambles after this devastating betrayal
reply
rsc
23 hours ago
[-]
Not sure what the betrayal is? He contributed a quote for yesterday's post. https://tailscale.com/blog/tailscale-enterprise-plan-9-suppo...
reply
bakul
18 hours ago
[-]
from the above post:

  > April 1, 1999
  >
  > FOR IMMEDIATE RELEASE
Forward to the past?
reply
pests
17 hours ago
[-]
This was explained in the post. 1999 was when Intel released the Pentium 3 with SSE instructions, which caused the first major issue that had to overcome.
reply
pvg
1 day ago
[-]
I'm sure it takes more than that to shamble an Olympic Silver Medal winner in archery.
reply
tiffanyh
1 day ago
[-]
Isn't that a joke.

He didn't actually go to the olympics.

https://wiki.c2.com/?RobPike

reply
pvg
23 hours ago
[-]
It is pretty clear that neither you nor these 'c2' characters have lettered in anything, including the Olympics.
reply
calvinmorrison
23 hours ago
[-]
The 9fans list had this one for April Fools:

Given the huge maintenance cost of immature computer architectures such as mips, 386, arm, arm64 and amd64, we decided to put our focus on the more mature and stable achitectures:

power64 and itanuim.

Therefore, all architectures other than power64 and itanium are thereby frozen, conserved and promoted to end of life.

reply
chaz6
1 day ago
[-]
Please consider RISC OS next!
reply
fidotron
23 hours ago
[-]
Three button mouse dependent UI solidarity!

Golang on RISC OS would be a truly ludicrous porting effort.

reply
chrsw
23 hours ago
[-]
My employer-controlled browser won't let me access that URL. At first it was cert errors and now it's just blocked.
reply
neckro23
23 hours ago
[-]
IME some web security gateways block the Tailscale domains entirely, presumably because it's a VPN that can bypass said gateway.
reply
bradfitz
23 hours ago
[-]
Weird. It's just Vercel on AWS. We have no alerts firing about any probers having cert errors or anything.

I wonder what your employer/policy doesn't like.

reply
jen20
21 hours ago
[-]
I've found that tailscale.com sometimes gets blocked by overzealous content filters as "VPN software".
reply
pjmlp
19 hours ago
[-]
Great, maybe Inferno as follow up?
reply
Gualdrapo
1 day ago
[-]
Yet I still wonder how cool things would be if Plan9 was the most popular and used OS
reply
gbraad
22 hours ago
[-]
I do expect tailscale drive shares to use 9P in that case ;-P
reply
bradfitz
20 hours ago
[-]
Indeed. We ran out of time :)
reply
naikrovek
13 hours ago
[-]
rsc, rob pike, and bradfitz are three people I could talk to for hours, completely wasting their time, especially about Plan9.

That OS fascinates me.

I remember early in my career when an expert I worked with could sit with me and patiently show me how to do something and let me ask questions for however long it took me to understand well enough what to do and how to swim if I fell in the deep end of whatever they wanted me to do. It was some of the fastest upskilling that I have ever done in my career, like getting a bachelors degree worth of very specific knowledge in three hours.

I don’t know C and I don’t know enough about Plan 9 to use it productively for anything, but it has some extremely cool and useful features that I want to know more about and learn how to use, even if it is only so that I can lament the non-existence of those features in the big three operating systems today.

If I had the money I would probably pay to get face time with all three of those folks for expanding my Go knowledge and rsc and rob pike for the plan 9 understanding that I have always wanted, but have never been able to give myself.

reply
stonogo
20 hours ago
[-]
Seems like the real story here is that the Plan 9 port of Go is not particularly healthy, and that it's easier to modify an OS kernel than it is to fix Go?
reply
bradfitz
20 hours ago
[-]
The Plan 9 port of Go _was_ not particularly healthy. It is now.

Fixing Go to not special case Plan 9 benefits all platforms--- all operating systems use the same code paths now, making the code simpler.

reply
facile3232
1 day ago
[-]
Plan 9 gets tailscale before a browser! Somehow this makes sense
reply