Why we can’t quit Excel
30 points
by thm
4 hours ago
| 16 comments
| bloomberg.com
| HN
https://archive.is/84dmE
cjs_ac
3 hours ago
[-]
When we write programs in a conventional programming language, what we're doing with the data is shown front and centre, and the data itself is sidelined or often not even present. In a spreadsheet, this is reversed: the data is prioritised, and the formulae are hidden.

There's an immediacy to a spreadsheet: the user can start with very literal actions on data, and slowly introduce abstractions like formulae. In conventional programming, the programmer has nothing but abstractions with which to work.

It's reminiscent of one of Fred Brooks' remarks about how showing the data structures makes the algorithms obvious, but showing the algorithms reveals little about the data structures.

reply
xtiansimon
2 hours ago
[-]
“There's an immediacy to a spreadsheet…”

The immediacy is you are looking at the data. What you’re going to do with it may not yet be known Is it the right data? What are the values? What are the features?

I write scripts at work and tell colleagues if they want my help they should first work it out in Excel and write an outline of the details. If they can’t do that, then the problem may not be a candidate for _script automation_.

Our pipeline is not fully automated, but that’s how I describe it. The rest is Excel Carpentry.

reply
wolvesechoes
2 hours ago
[-]
I think similar remarks can be made in regard to Jupyter notebooks and their popularity among scientist, engineers and data people. Yes, they are still more in the conventional programming realm, where the code is in the foreground, yet to some extent they bring more immediacy. The fact that you can look and investigate at the data you are dealing with at every step (print the table, make a plot etc) is powerful.

Another thing would be of course powerful REPL environments.

And there are also visual environments like Matlab's Simulink and Modelica. Can you write control algorithm or a bunch of differential equations in any language? Yes. But control engineer can just take a quick look at screenshot of a Simulink model and already have a good idea on what is going on there. This cannot be said about the code.

This may be rather unpopular among hackers, where TTY clones and vi emulation rules everything, but computers and software seem to me most powerful and enabling outside the usual coding context of putting streams of characters into the source file using a keyboard.

reply
pjmlp
1 hour ago
[-]
As developer starting coding in the 1980's, that always valued high level approaches to computing than plain bits and bytes, I find this a kind of tragedy, given that productive tools helps everyone.

I love a quote from from some VB folks, that while others laught of them using VB, they were laughing all the way to the bank themselves.

My only complaint about visual environments, is that not all offer modular tools, the EEE IC version of a module or function, some are quite primitive only allowing one screen for the whole flow.

I wonder what place will be left for the TTY clones and vi emulation folks, when everything on the system is driven by systems programmed in agentic tools, command line wizardry replaced by MCP like tools and such.

reply
Eddy_Viscosity2
2 hours ago
[-]
Even just regular matlab gives you some of that data immediacy because you can just ctrl-enter on a block of code which will execute just that block and show you results in the console. Very handy for seeing what various bits do to the data.
reply
kingstnap
3 hours ago
[-]
I've had a lot of experience with dealing with exactly this problem in ML.

There is no replacement for visualizing the data and intermediates. You have to see what is going on, it's the only way to catch bugs and issues and make good performance improvements.

Like actually looking at the gradients and weights and activations and predictions and stuff often shows you that something janky is going on, its great for adjusting architectures. Looking into specific high loss test samples and mispredictioms and stuff will show you that there are problems with your data or normalization and whatnot.

The issue is that there are basically an infinite number of intermediates you could potentially look at. So ignoring almost all of them is the only thing that scales and you have to be extremely deliberate.

reply
pjmlp
3 hours ago
[-]
It has the appeal of Smalltalk or Lisp inspired live coding environments, gone mainstream with an approach that people can better understand.

Note all the audio programming tools getting embraced by music folks as well, it is coding, but made approachable.

reply
sega_sai
3 hours ago
[-]
That is an interesting take on spreadsheets. I wonder if it is possible to establish some way to switch back and forth between 'programming view' vs 'spreadsheet view'.
reply
PetitPrince
2 hours ago
[-]
I think the best we have at the moment debugger view; in other words a snapshot of the state of the system. Maybe with a dashboard you can see some progression over time ?

(I somehow though of the videogame Factorio and of that thing called Labview, but I cannot form a coherent thought about it)

reply
IAmBroom
1 hour ago
[-]
Yes, Excel allows you to view the entire spreadsheet as formulas in cells (which may simply be values).

It's very, very rarely useful.

reply
Libidinalecon
3 hours ago
[-]
I have always had vague ideas about this with pandas but don't have the aptitude or motivation to take on a project of that size.

I hate Microsoft but love Excel. I was hoping python in excel would be this but I don't think it quite is.

reply
bloqs
2 hours ago
[-]
that fred brookes quote description hit me on a molecular level. what are his most important writings that i should read?
reply
gglitch
51 minutes ago
[-]
You're in for a treat. The quote, in case you haven't looked it up yet, is "Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious." The canonical work is The Mythical Man-Month: Essays on Software Engineering

https://en.wikipedia.org/wiki/The_Mythical_Man-Month

reply
daft_pink
3 hours ago
[-]
It’s pretty simple. Everyone knows how to use it effectively and the cost of training every accountant or finance person is far greater than paying Microsoft a few bucks included in your email anyways. Plus you know it will be compatible with external files.

I worked at a place that used Google Workspace and the most effective people simply supplied their own Excel, because it's just way more efficient than spending hours learning how to use Google's clunky pivot table function, when you already know how to do it well with Microsoft.

reply
Havoc
3 hours ago
[-]
Yeah don’t think it’s going anywhere any time soon in big corporate - at least in finance. The network effects are just too powerful - compatibility, training, integration, skill availability etc.

Every single attempt to migrate to something else has been a comical failure. On the plus side they tend to be rapid failures rather than SAP style multi years

reply
pjmlp
3 hours ago
[-]
As someone that was part of such project, the main issue is that Excel is really programming, plus there are macros, VBA, AddIns, lambda, PowerQuery, and now Python on top.

All those migration projects tend to fail on the details, there is always that one customisation point that is easily done in Excel, but requires a whole team with architects and such for getting the same capability on the replacement tool.

reply
ezst
3 hours ago
[-]
Nice thing is, Microsoft themselves are undoing that with the push to Office as a web-first application. While making it more ubiquitous (they think), they are at the same time cutting loose some of its strongest anchors
reply
meetingthrower
2 hours ago
[-]
And I think it is actually one of the reasons that linux adoption is slow. No real way to run Excel natively. I would switch today but my life in finance is all Excel and Powerpoint.
reply
antasvara
1 hour ago
[-]
Just wanted to shout out the ability to copy, paste, and transpose the data using your mouse and keyboard. The ability to combine and reorganize data without needing to program is huge. It means you can grab data from an online source, paste your results into a report you're writing, quickly flip some data, etc.

Other commenters have covered the major reasons Excel is so popular. But Excel really shines because it's designed for business. I use R and RStudio for a lot of my work, and while it's great, there are little things that it can't do that Excel can.

reply
quacked
1 hour ago
[-]
For small-to-medium size data sets, you can use Power Query/BI to essentially run a relational database and metrics/dashboards on it inside an Excel spreadsheet plus a web page, minus all the features that a real DB has in terms of version control and backups.

I can't leave Excel for that. I can set up a "data integration" in 3 hours that has a highly customizable and (relatively) bug-free front-end, and maintain it myself. The amount of work and knowledge it takes to get the same thing spun up in a proper language with a proper server is 1-3 orders of magnitude more.

reply
erikgahner
3 hours ago
[-]
reply
solatic
3 hours ago
[-]
Spreadsheets are to Finance folk what WordPress is to Marketing folk and what tools like v0 and other LLM-powered web app builders are to Product folk. The promise of having some fancy schmancy technological tool let you do your job by yourself and without needing to collaborate with other people is basically the strongest value proposition that you can make to working professionals.

The fact that all these tools eventually fall apart at scale is basically irrelevant. You can only take away control from their cold, dead hands, and they will never learn the programming skills to build something more scalable themselves. All solutions almost invariably trend towards shifting scaling problems away from the desired frontend that keeps control (or at least the illusion of control) in the hands of business stakeholders (one of the reasons why companies love building all manner of integrations on top of Jira).

And let's be clear, it's not like the alternatives here (i.e. databases) are so easy. Finance folk are used to creating new spreadsheets on their local computer for free and at will; most databases require setting up a server, DNS, certificates, firewalls, most of which have real costs. SQLite naively sounds like a reasonable approach, but by default a table is limited to 2,000 columns while an Excel spreadsheet is by default limited to about 16,000 columns, and yes, stuff like this really matters when you're talking about trying to uproot a favored tool. At the end of the day, most of Excel's limitations are due to attempting to cram a spreadsheet into a single file (same as SQLite); if Microsoft were smart, they'd offer a cloud-only "spreadsheet" (really a database over a full filesystem, or maybe over object storage) without the limitations of ordinary Excel spreadsheets, where Excel-the-desktop-app only downloads to the local client the relevant cells that are actually in view, while adding more options to mirror/load data from external sources into other sheets attached to the same cloud-only "spreadsheet".

reply
pjmlp
59 minutes ago
[-]
That is where alternative approaches to databases like Airtable win among such audience.
reply
steveBK123
3 hours ago
[-]
Excel is the glue holding a lot of corporate workflows together. Like Perl or Python.

I have seen it used for everything from configuring aircraft for sale to quoting bond prices for voice traders.

There's even that famous Japanese artist who uses it for painting.

It's sort of like the OG normies Jupyter notebook.

reply
JaumeGreen
3 hours ago
[-]
Spreadsheets are the killer apps. Since I became pointy haired sheets and documents are the fuel of most what I do. Even things like Jira we use them only for the bare minimum and we refer to sheets to see how things are going. Not that I think it's sane, but it's what works.

I even have done this first day's advent of code, and the first part of day two, in a Google sheet. Formulas only, so no scripting needed.

reply
lordnacho
3 hours ago
[-]
I think when you learn to code, and I mean coding "real" programs, you forget how powerful Excel actually is.

Yes, it's awful in many ways, but it is very accessible to people who don't consider themselves programmers. "I just want to do my thing" is very easily done in a mashup of the spreadsheet and VBA.

We also forget the pain of learning a new technology. People whose first experience is excel also go through this. Shit doesn't do what you wanted. After a while, they can build stuff in excel, but they don't want to learn python, because there's more pain coming.

Most senior devs have transcended particular languages or technologies, so they don't see it. They pay a very small cost for picking up a new tool, so they scratch their heads when they see a guy who wants to run a trading book on a spreadsheet.

reply
apples_oranges
3 hours ago
[-]
It's just so much easier to stay and keep paying Microsoft than switching. But why hasn't any startup found a way yet to disrupt it by giving the users something they actually love more, so they can start lobbying businesses to switch to it, I wonder.
reply
PurpleRamen
3 hours ago
[-]
Excel is a behemoth of functionality which is constantly growing and improving. It's not as simple as building "just another spreadsheet"; and attempts are made, dozens of them. But it seems they all are either not really understand the strength of Excel, or simply admit defeat from the beginning and try to find a niche where they can compete, by looking at specific usecases and target groups. Kinda like all those companies who build hyper-specialized cooking-tools, but still can't beat the versatility of a good knife, so they mainly sell to normal people, not the real experts.
reply
tsimionescu
3 hours ago
[-]
Because the power of Excel comes from the huge amount of features it has slowly accumulated over the decades of its lifetime. You can pretty easily make an Excel 97 replacement today, probably. But people won't be able to use that in these business contexts - they each need some of those obscure 1% features that got added.
reply
pjmlp
3 hours ago
[-]
Because every department has a different use of Excel.

LibreOffice and Google Sheets are still quite far from everything that Excel is capabale of.

I was part of a project to replace Excel, on a lifesciences department that was using it, as you might think about R, Pandas and similar.

Tableau was even a more appealing candidate to them.

reply
Cruncharoo
1 hour ago
[-]
Because I actually do love Excel for my uses, and I have about 15 years of muscle memory for all of the keyboard shortcuts.
reply
koolala
3 hours ago
[-]
Wish it had interesting alternatives like all the C++ alternatives today. Imagine a spreadsheet that could be 'compiled' into a full stack program.
reply
ubermonkey
1 hour ago
[-]
I've been an avid MS hater since the late 1980s, but I absolutely and unironically believe Excel is excellent software. It's the bright spot in a pretty problematic portfolio. Other Office apps have flirted with greatness -- DOS-era Word 5.0 was groundbreaking, for example; Access had a pretty great moment in the sun 30 years ago -- but no other consumer package from Redmond is on par with Excel.

I'm sure there are many orgs that would love to ditch O365 for Libre or Google, but can't simply because there is no real alternative at this point. Excel is too entrenched.

Excel can take people from fairly naive "help me make a table of numbers" tasks all the way to actual programming. Via PowerQuery, Excel gives end users access to data warehouses directly. Sure, a tiny bit of SQL is helpful here, but it's not required. I've written no end of data sanitization / transformation tools using Excel. Sure, I would have RATHER done it in Perl or Python, but Excel can be assumed to be present on the target user desktops; not so with Perl.

It would be cool if there was a better off-ramp for advanced Excel users into more focused tools appropriate to their domains (e.g., R, or Tableau/PowerBI), but if these folks are solving their problems who are we to push?

reply
jmclnx
2 hours ago
[-]
Excel is where it is due to a kind of mild bait and switch. In the DOS and OS/2 days before Windows 95, Microsoft told Lotus and other software companies OS2 was the future, not windows. So Lotus when all in on OS/2 with its spreadsheet.

Then came 1995, M/S released windows 95 and at the same time they had a full software suit running on W95. It took Lotus a while to get a W95 version out. I think the same happened to Word Perfect.

So here we are, too bad Libreoffice was unable to do the same with Linux. But now people are so entrenched I doubt they will only change if forced.

Also, years ago IBM tried to get people on LibreOffice and off M/S, it failed miserably. Many Orgs. in Europe is trying now, I hope they succeed.

BTW I wonder if we are heading down the same path with github ? I hope not but seems we could be.

reply
Jemm
3 hours ago
[-]
For many of the people I see stuck on spreadsheets, Excel is at the limit of what they can comprehend.
reply