They all sound like good recommendations, but there's not much in the way of a total drop in replacement for Splunk.

You can build an ELK stack or something that resembles it, but you have to hire someone to directly maintain it and build out functionality. If you're a megacorp, that might make sense financially.

I used to work at Splunk when they were still a fairly trendy start up, it was fun and I helped build out Cloudworks, Splunk's v2 cloud offering that was a significant upgrade in capabilities for customers vs the previous gen, Rainmaker. By the time I left though, it had a much more corporate feel to it as the C level execs pursued growth at all costs and went on a massive hiring spree, and a lot of the old timers that were incredibly talented and intelligent people were starting to leave for greener pastures.

I think it's a space that's largely overengineered when classic solutions tend to work very well and are FOSS. On the log side, rsyslog, systemd-journal-remote, etc are being overlooked in favor of the behemoths like Splunk, and I think the real opportunity is in reducing the SIEM stack complexity by returning to simple tools that do their job well (unix philosophy).

The problem is then VC's and their companies are trying to monetize in their style, which almost always means using massive funds to dominate a market space and then hold on to that. Serving the customer need has almost become secondary to growth for these types.

What I see in this article is more stuff about the next Splunk, but what I want is an analysis of why people even need splunk (often they don't), and how that means the real opportunity is in returning to basics.

I used to do this for a living and went on to work for a splunk partner.

This company was run like ass from an inside perspective. Made me realize how most of Splunk isn’t for making things secure, it’s to bring your insurance premiums down. I’ve certainly seen insecure setups with Splunk often and it’s a huge myth by having it you’re more secure. Doesn’t count if you run it as root and I was amazed how many major companies did exactly that.

Cured me of taking most of the security space seriously when I saw how the sausage was made. Most of its bunk and games with an insurance premiums. Literally companies would pay to just set it up then never touch it or turn off all the alerts. Didn’t matter though because by having it the insurance premiums went down. Just a money game. Very little to do with security.

After being at, oh, 5-6 Splunk shops I have yet to see one not fall into what I call the "Splunk Death Spiral".

  1) Have a massive logging problem.
  2) Get sold on Splunk.  
  3) Implement Splunk and pour all your data into it.
  4) High-fives all around at the amazing insights you're getting.
  5) Get the bill.
  6) Start rapidly paring down the amount of data going to Splunk to get under budget.
  7) Find you're not getting very good insights any more.
  8) Have a massive logging problem.

I briefly worked for a place which used Splunk for what you'd normally use ELK for. I found it way more forgiving and in many ways easier to deal with than ELK, if only for the 100% certainty that you can run any query on anything, even if it sometimes takes ages. It was an old version, too.