* "Man who built ISP instead of paying Comcast $50K expands to hundreds of homes": https://news.ycombinator.com/item?id=32411493
* "NLNOG: Getting Fiber To My Town [video]": https://news.ycombinator.com/item?id=24424910
* "Jared Mauch didn’t have good broadband–so he built his own fiber ISP": https://news.ycombinator.com/item?id=25753360
* "How To Create Your Own ISP with Jared Mauch": https://www.youtube.com/watch?v=kJH9Emr99KI
The issue is that wisp owners are cowboys, who dont ever follow through with good practices. Their networks, often due to guides like this website, end up riddled with technical debt (Technical debt is usually but not always branded Mikrotik).
Starlink doesnt offer layer 2 services. Starlink doesnt offer half rate backup services. Starlink doesnt offer installation. Starlink can often be very congested.
A wisp operator can:
1. Pull fibre into an area and then distribute it via 60GHz.
2. Pull fibre a bit further away and use decent APs with good MIMO.
3. Use profits from the above to pull fibre closer and ultimately overbuild themselves in areas with enough density
4. Service extrmely deep rural customers who dont have other options.
5. Service MDU's with a reverse model of 60GHz to the building, then fibre to the appt.
What actually happens in practice is that anything more complex than bouncing 5ghz off of a tower is too hard, hiring someone intelligent to do it for them is too expensive and too hard and so small wisps just sell to bigger wisps who sell to fibre carriers or go bust.
Why this association? At least on the consumer side, I've really enjoyed using Mikrotik's stuff for my homelab. Is it just a sign of "someone not wanting to pony up for Real Networking Gear" or something similar?
The big questions come at the tower. Juniper doesn't have a massive POE tower switch. The options there can be all bad. Its this huge mess. AC or DC, backup or no backup etc etc. And there are always drawbacks. I had a customer purchase a massive cambium tower switch with all the bells and whistles only for it to occasionally lock up and forward on all ports like a hub. Netonix can be good, however their failure rate is higher than I would like, and the support always tries to blame your grounding on returns. But the interface is easy to teach to junior techs, and other than some weird issues with linux partition size and their vlan implementation they just go. The ubiquiti tower switches are a lot of fun. I dont know if its still the case but for a few firmware versions they had half the config only available in a legacy interface. But once you made a change in the legacy interface, all the labels you set in the new interface are lost.
RF last mile for residentials, Ubiquiti/Cambium. They both work well in some respects and suck in others.
NEC for backhaul if you can get it. Otherwise there are some decent ubnt/cambo options also. ubnt has come a long way in 60GHz.
If you want real telco stuff look at Edge core or Nokia.
I love my Mikrotik homelab setup, but it's held together by carefully crafted per-box configs that are not easy to change all together (e.g. adding another VLAN, etc).
Maybe it's just my newbie status at this, but hopefully there's some cool piece of software that I just don't know about yet!
However, there is only a limited "Quick Settings" option, which supports most basic configurations. For anything more advanced, you do actually have to set it up yourself.
However, the problem with recommending Ubiquiti in the context of an ISP, is that once you hit a wall with it, you are done. You cannot change things that are not supported to be changed, you cannot make it do something it was not "designed" to do.
On Ubiquiti, you can ssh into the VyOS-derived OS on the EdgeRouter (if talking about EdgeOS), but anything you do there cannot be accessed from the webui anymore, and changing anything in the webui will of course destroy what you changed manually over SSH.
At that point, you have an offbrand Linux router that is hindering you, and you either go with MikroTik or upgrade to Cisco equipment. But that Ubiquiti is going on a shelf.
In general, it's quite different running an ISP and running a homelab.
With MikroTik, you can make ANYTHING and everything. And there is no paywall for additional features like some vendors.
Finally, to underline my point, it is required to learn MikroTik's RouterOS before starting an ISP, if you don't want to end up with problems.
Anyway, they ship buggy code on overtaxed hardware that keeps me up at night.
ROS 7 was a disaster from 7.0 to 7.11, and it shipped on hardware that couldn't be rolled back to 6.
MPLS and OSPF have been huge victims of this. MPLS forwarding table often freezing and not updating. OSPF variables conking out causing protocol flaps, or not adding routes to the route table.
Theres other issues too. I could really go all day.
Great homelab stuff. I use it too.
I'm guessing not Cisco because the license costs?
Mostly right, but the boss hired pretty smart people fresh out of college (cheap).
They are still around and their website still sucks. Last I heard they were doing fine.
But it has to be the correct sort of suck, it's got to suck in the "I'm an old sys-admin and can't be arsed to care about these javascript frameworks" sort of way and not in the "I am in sales and need 150 tracking agents" manner.
https://news.ycombinator.com/item?id=27539165 (June 17, 2021 — 607 points, 153 comments)
https://news.ycombinator.com/item?id=20726906 (August 17, 2019 — 635 points, 95 comments)
https://news.ycombinator.com/item?id=16160394 (January 16, 2018 — 938 points, 193 comments)
The issue is that a lot of the advice is bad, the good advice costs money and is indistinguishable for the lay person from the bad advice, and its very easy to access terrible advice. This website isn't terrible, but it would serve to pay for the best advice possible before spending even more money on the wrong hardware in the wrong area.
the lay person also may not be able to identify who's advice is worth paying for
The the answer largely depends on what you're trusting the advice for. Is it medical advice? Trust none of it. Is it the advice going to cost you a lot of money? I'd ask an expert.
The site here is purely informational-use-at-your-own-risk. No different than anything else on the internet. It'll get you started on your research so you can have some basis upon which to ask/talk to each other. Hey, even links to a community discord where you can talk to like minded individuals and maybe get some more clarity.
Maybe they've got some sites in the middle of nowhere, but I'd bet they worked with fiber vendors to pick places that were easy to hook up to existing fiber, because why not. I think I recall seeing someone had done some sleuthing and found that the teleports were sometimes being placed along rail right-of-ways, where there's often also fiber and maybe even power.
"How do you collect a small fortune with an ISP?"
"Start with a large fortune"
Latency and speed are slow, and some of the providers mess with/block certain traffic (IKEv2 etc).
Most of the major FWA providers are mobile phone carriers, so both mobile and FWA customers actually use the same tower/spectrum/etc so the general increase in demand (especially from mobile given they can't stop selling new mobile servcies) sometimes results in a good service in Year 1 degrading to poorer service in subsquent years. However the carriers can easily resolve that by adding more 4G carriers, deploying 5G, and even building new cell sites (which kills two birds with one stone -- better coverage for mobile users and more capacity for FWA users).
When FWA first came out I confess that I thought it was a silly idea until my eldery mother accepted an incentive from her provider (cheaper monthly fee) if she moved onto FWA (from ADSL previously). She's zero complaints. And sure enough it works well for a low-end user -- emails, Facebook, WhatsApp, Netflix, YouTube, etc all work just as good on FWA as it does on fibre/DSL/Cable/etc. She happens to be close enough to the tower and that tower has also been upgraded and has heaps of 4G carriers so the service is consisently good.
YMMV but when done well, 4G/5G FWA is a great option for low-end users.
(Yes obviously latency is worse than fibre, I would probably hate it if I was still into FPS gaming etc., but in practice it's fine.)
(When I set it up it was better and cheaper than the copper options, fibre not available. Fibre or Starlink would now probably be better, but still each much more expensive, even ignoring one-time costs. I don't need it/not worth it. Idk about latency but I could certainly get more bandwidth out of one-time costs on LTE too.)
Most people on cellphones don't notice, but it becomes oh so noticeable when your interacting with it every day.
Also, I forgot which equipment they gave me, but it had terrible buffer bloat, and did some nasty nasty arp spoofing when I placed it on my network and made everything bad, and it wouldn't let me put it on XX.2.0/24 and have a static route for XX.0.0/24 so maybe I could keep it roughly contained. Maaaaybe I could have done something with VLANs, but I was done at that point. I'm still grumpy because it took 3 months and contacting the CEO email address to get the bill settled, but they did settle it. Not exactly a risk free trial in my book.
Speed and latency was good though, as long as you didn't hit bufferbloat. I don't remember exactly, but 500M+ down, 200M+ up, < 25 ms ping; CGNAT brought the speeds down to about 100M, but most CDNs are v6 capable now, and most people aren't that bothered by CGNAT. Obviously you and I and a lot of others here have reasons to want their own v4 address, but a lot of people in a lot of places can't get one on any home internet, so. Of course, speed and latency vary a lot by your reception, but I live on a heavily forested and very hilly island; I am at a high point, but surrounded by trees so eh?
What service have you tried that didn't work well for you?
Don't get me wrong, I'd much rather have a wired connection. But if my isp can't even be bothered to give me a functional connection more than 90% of the time...
Besides that, not everyone is willing to pay for a Starlink priority plan.
The priority service wasn’t much faster 6-8 months ago but seems to be much faster from what he been reading and seeing recently, but it might’ve relative to how busy a particular area is
But as any business venture: if you don't know whether you can get the customers, you better have the cash lying around to pay for everything yourself =D
The only credible options are WISP, Starlink, and cellular. WISP is the same price as Starlink and is slower / less reliable. Cellular is cheaper but gets slow at peak hours.
In short, the local WISP has been losing a lot of customers to Starlink and T-Mobile Home Internet.
Starlink is a replacement for dial-up, satellite, and in some cases legacy DSL- pretty much nothing else.
You can push gigabit[1] ptp over the air these days for pretty cheap. Starlink seems to top out around 200Mbps in the best case.
[1] https://store.ui.com/us/en/collections/uisp-wireless-airfibe...
https://ca.store.ui.com/ca/en/pro/category/all-60ghz-wireles...
If you can get a fast wired connection, do it. Starlink is for people who can't, and it's far and away the best option for them.
The target audience for Starlink in the continental US seems to be people who's other options are traditional satellite, dial-up, or sometimes DSL (typically implying you're in a more rural area, but not always). For people in those situations, Starlink can be a good alternative.
However, if you have access to modern cable or FTTH... well, it's not a substitute.
That's not too bad. However, there's no speed listed. Third party reviews state 100/10 Mbps (up/down), which is not too bad considering the same site states a UK average of 75/15.
The vast majority of the UK has pretty slow speeds.
That said, unlimited internet subscriptions over fiber can be had for as little as 20 USD per month, which is far cheaper.
I lived in a village in Colombia, population about 2000, which had four competing wireless ISPs. The quality was extremely low, but so were the prices.
For now they're having to launch a Falcon 9 twice a week to build up the constellation, but with Starship intended to be able to launch the equivalent of several F9 launches in one go, maintaining the constellation will become much easier.
You can cost share, if you can get your neighbors to also sign up... but there's only me and one other crazy person who will do it, because the cost is too high. Not a lot of people are interested in paying $50k to upgrade from 50-100M down to 100m or 1g symmetric. It's better, but not enough better; especially when T-mobile 5G can get you 500m+ down/ 200m+ up with no installation.
My life experience has been that there's a "sane person's mental model" and then there's Municipal Bureaucrazy and nere the twain shall meet
So, yes, it's possible there is already trenching and it could even be that said conduit are owned by the taxpayer, but as far as random citizen pulling fibre through them ... I have fears. That story gets worse with the number of articles that I have read where Local Incumbent ISP bribes^Wlobbies a city councilperson to slow roll or straight-up deny any such request, making it doubly not a technology problem
https://en.wikipedia.org/wiki/List_of_internet_service_provi...
https://www.infrastructureontario.ca/en/what-we-do/projectss...
Out of ~10 countries I have supported WISP's in, I have never seen a country with less of a barrier to entry.
We did have a theory that our customers were rural, and that the cities might present different issues. But trying to do what they did in Australia would be add 2 zeroes to the end of every cost.
No. That's not the case at all.
> so that other providers can act as "backups", so "that never happens again."
Also, no.
Also what do you mean by interconnect? BGP IP peering is a thing for a while. Big folks peer with big folks.
Are you aware of this?
* https://mobilesyrup.com/2023/11/06/bell-and-telus-must-offer...
Or you can do what Beanfield did and build out your own fibre infrastructure to customers.
In 2005 Google wanted to do a nice gesture and offer free wifi across SF. The Board of Supervisors literally asked Google how much they were intending to pay the city in order to offer free wifi... SMH.
It of course never happened.
I also had mostly-good experiences with Google Fiber (née Webpass) coming to the condo building, although despite their name it came in via WISP on the roof. I could have thrown a rock and hit the Webpass building from my condo but ... at least it wasn't Comcast and the whole building got gigabit IPv6 service for what any one of the owners would have paid for it from Comcast. I bet our units being pre-wired for Ethernet made the situation easier, but depending on the size of the HOA I'd guess not insurmountable even if one had to pull cat6 later