Advent of Sysadmin 2025
122 points
4 hours ago
| 7 comments
| sadservers.com
| HN
0xbadcafebee
2 hours ago
[-]
Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job:

  1.  Get a user to stop logging in as root.
  2.  Get all users to stop sharing the same login and password for all servers.
  3.  Get a user to upgrade their app's dependencies to versions newer than 2010.
  4.  Get a user to use configuration management rather than scp'ing config files from their laptop to the server.
  5.  Get a user to bake immutable images w/configuration rather than using configuration management.
  6.  Get a user to switch from Jenkins to GitHub Actions.
  7.  Get a user to stop keeping one file with all production secrets in S3, and use a secrets vault instead.
  8.  Convince a user (and management) you need to buy new servers, because although "we haven't had one go down in years", every one has faulty power supply, hard drive, network card, RAM, etc, and the hardware's so old you can't find spare parts.
  9.  Get management to give you the authority to force users to rotate their AWS access keys which are 8 years old.
  10. Get a user to stop using the aws root account's access keys for their application.
  11. Get a user to build their application in a container.
  12. Get a user to deploy their application without you.
After you complete each one, you get a glass of scotch. Happy Holidays!
reply
cobertos
55 minutes ago
[-]
Re: 6. ... Github Actions

Github Actions left a bad taste in my mouth after having it randomly removed authenticated workers from the pool, after their offline for ~5 days.

This was after setting up a relatively complex PR workflow (always on cheap server starts up very expensive build server with specific hardware) only to have it break randomly after a PR didn't come in for a few days. And no indication that this happens, and no workaround from GitHub.

There are better solutions for CI, GitHub 's is half baked.

reply
swyx
43 minutes ago
[-]
bugs happen to all of us. whats your better solution - gitlab?
reply
sharts
30 minutes ago
[-]
honestly jenkins really isnt that bad
reply
jagged-chisel
1 hour ago
[-]
> … from Jenkins to GitHub Actions.

Oh, good lord why?

reply
vachina
1 hour ago
[-]
Because sysadmim wants to outsource their responsibilities (and job).
reply
betaby
1 hour ago
[-]
5. and 6. are a matter of taste (trade-offs), the rest is spot on!
reply
daemonologist
57 minutes ago
[-]
You get me the permissions to do half of this stuff, and I'll do whatever you want.
reply
alberth
1 hour ago
[-]
I’d be super interested to see solutions to each, just to learn from.
reply
melvinodsa
3 hours ago
[-]
When I get sad and nothing to do in the world, may be hacking into a sad server's problem seems very interesting
reply
udev4096
27 minutes ago
[-]
I wonder if we could get something like that for k8s, docker and other container ecosystem
reply
NooneAtAll3
50 minutes ago
[-]
what's the deal with 12-days advent calendars lately?
reply
aljaz823
23 minutes ago
[-]
Advent of Code went from 25 days to 12 days starting this year, as it took too much time each year to come up with 25 unique challenges [1].

[1] https://adventofcode.com/2025/about#faq_num_days

reply
swyx
42 minutes ago
[-]
aren't they canonically 12? 12 days of christmas etc
reply
dragonwriter
31 minutes ago
[-]
No, Advent is the liturgical season preceding Christmas, beginning the fourth Sunday before Christmas (which is also the Sunday nearest November 30), it is a period of at least three weeks and one day (the shortest period that can start on a Sunday and include four Sundays.)

The 12 days of Christmas start on Christmas and end on January 5, the eve of the Feast of Epiphany.

12-day advent calendars are a fairly recent invention that mirrors the 12-days of Christmas, but has no direct correspondence to anything in any traditional Christian religious calendar (the more common 24-day format is also a modern, but less recent, invention detached from the religious calendar, that simplifies by ignoring the floating start date of advent and always starting on Dec. 1.)

reply
c0wb0yc0d3r
30 minutes ago
[-]
Advent calendars track time until Christmas. “12 days of Christmas” are the twelve days after Christmas.
reply
d5ve
29 minutes ago
[-]
Don't the 12 Days of Christmas start on the 25th though?
reply
kralos
1 hour ago
[-]

    imagine typing in a terminal...
    you want to delete the previous word so press ctrl+w...
    actually you're in a browser; the window closes...
:sadness:
reply
melvinodsa
1 hour ago
[-]
We used to run terminal in browser using https://github.com/yudai/gotty and the entire dev team remapped their Ctrl+w to Ctrl+`. We did frontend and backend development with this setup almost for 1.5 years. Muscles memory and till this date, always have the fear if my actual terminal will get closed if I use Ctlr+w :P
reply
fduran
1 hour ago
[-]
hello, creator here, sorry about that. In this case you can click again on the "Open the Server Terminal in a New Window" button
reply
teddyh
4 hours ago
[-]
[flagged]
reply
fduran
1 hour ago
[-]
Checking out how the platform works was two clicks away: home -> give me a server.

I don't know of any other SaaS which gives you a VM with one click without any registration but we do it.

In any case thanks for the feedback, I've put a button on this /advent page for clarity, cheers

reply
thatxliner
4 hours ago
[-]
well advent of code also needs an account
reply
npinsker
2 hours ago
[-]
It’s not necessary to see the problems though
reply
unsnap_biceps
2 hours ago
[-]
It's not clear that you will need an account to see the problems. Logged in with my account and it's exactly the same page. It's not Dec 1st everywhere yet, so they might open up for everyone when they do open them up.
reply
stonecharioteer
3 hours ago
[-]
This also has a paid account and a business account.
reply
fragmede
4 hours ago
[-]
how do you want it to work? do you even sysadmin?
reply
jbmsf
3 hours ago
[-]
I see: a page offering something interesting but vague.

If you tell me more, I might sign up. If I have to create an account first, I'm walking away.

reply
teddyh
3 hours ago
[-]
> how do you want it to work?

I would like to see and try to solve the scenarios for myself, not to get meaningless internet points. If you look at their front page, you can do that right now. So why do I have to create an account to even see these special advent scenarios?

> do you even sysadmin?

Yes.

reply
mekoka
3 hours ago
[-]
I think the point is "ok, account is free, then what?"

At 5$/m I might give the paid subscription a try.

reply
tonyhart7
40 minutes ago
[-]
now we need advent of arts,math etc
reply