Reverting the incremental GC in Python 3.14 and 3.15
144 points
3 days ago
| 10 comments
| discuss.python.org
| HN
athoscouto
2 hours ago
[-]
We've been impacted by this. I migrated our services to Python 3.14 so we could attach profilers during runtime.

A couple of services looked like they had a memory leak. Memory was continuously increasing over time. Thanks to Python 3.14, we were able to use memray to understand what was going on. Those services were recreating HTTP clients (aiohttp) for every inbound request, and memory allocated by the downstream SSL lib was growing faster than it was being released.

We ended up rolling back to 3.13, which fixed the issue. I'll try again with 3.14.5.

reply
LaFolle
1 hour ago
[-]
On profilers - profiling will come in 3.15, are you referring to remote exec? It is a great feature I am very exited about, at the same time afraid that the company won’t allow ptrace capability in prod.
reply
athoscouto
7 minutes ago
[-]
yes. remote exec allows me to attach profilers (e.g. memray) directly into a running process. i'm also excited about the upcoming statistical (cpu) profiler from 3.15
reply
davidkwast
4 hours ago
[-]
"Python 3.14 shipped with a new incremental garbage collector. However, we’ve had a number of reports of significant memory pressure in production environments.

We’ve decided to revert it in both 3.14 and 3.15, and go back to the generational GC from 3.13."

Sounds the right move for me

reply
winrid
3 hours ago
[-]
The main benefit of python to me is that while slow, it's predictable. I do think they're going to get a lot more resistance to adding JITs, moving GCs, etc. it will become java with a million knobs to tune. If people want a JIT'd python just use pypy, right?
reply
pron
2 hours ago
[-]
Java lost almost all those knobs a while ago (I mean they're there, but you're better off relying on the defaults). The modern GCs have one or at most two knobs remaining, and even that will become unnecessary next year. As to predictablity, you get maximal pause time of well under 1ms for heaps up to 16TB.
reply
stackskipton
2 hours ago
[-]
As Python using SRE and supporting Python Flask apps, most of us would love JIT in Python assuming it pretty much drop in replacement.

PyPy doesn't have the support it needs and is stuck on 3.11.

reply
zozbot234
34 minutes ago
[-]
Why not just use Go? It has a proper concurrent, non-moving GC that, AIUI, has not been associated with sudden memory spikes.
reply
davidkwast
1 hour ago
[-]
It is the same for me. Predicability is better than any optimization.
reply
sigmoid10
2 hours ago
[-]
And if people want python with java, there's always Jython.
reply
brian_herman
1 hour ago
[-]
Graal vm has support for python 3 unfortunately it’s funded by oracle.
reply
wavemode
53 minutes ago
[-]
If it makes you feel any better (it probably doesn't), the development of OpenJDK and the Java language itself is also mostly funded by Oracle
reply
brokensegue
2 hours ago
[-]
jython has been basically unmaintained for quite some time
reply
sigmoid10
2 hours ago
[-]
Well, they never made the jump to Python 3. But shipping 2.7 interpreters in 2024 was quite an achievement on its own. So their users already know this pain. And from my experience in academia, python 2.7 and java 8 will probably be used for another 20 years before the last machine running that stuff burns out.
reply
arikrahman
2 hours ago
[-]
Jython is unmaintained, I'd recommend Clojure. Use python libraries and code while seamlessly targeting the JVM.
reply
graemep
2 hours ago
[-]
Resistance from anyone who matters to the developers?
reply
bhouston
3 hours ago
[-]
.NET seems to have regularly changed the garbage collector over the years and I do not remember any similar surprises in production. I wonder why they have had better experience?

I thought that by now dynamic garbage collection was a known quantity so that making changes, outside of out right bugs, is fairly safe and predictable?

reply
stackskipton
3 hours ago
[-]
One thing Microsoft does really well is eating its own dogfood and Microsoft feeds a ton of .Net dogs.

So any change to GC starts with massive .Net MSFT code base so they get extremely good telemetry back about any downsides and might be able to fix it in time.

reply
chihuahua
2 hours ago
[-]
I remember working on the Windows Update back-end at Microsoft around 2005, and we had a problem where it would freeze up periodically, and not surprisingly that turned out to be caused by GC. But we noticed it before shipping, and we just tweaked some GC parameters.

So I think it was not a big problem for .Net because it gave you enough control over GC, and because people tested their code before putting it in production.

reply
Weryj
2 hours ago
[-]
Actually there’s a change to dotnet 9 with how it handles the heap and GC which caused major issues for us.

I’ll confess the reason it hit us so hard is because the code quality was so low and wasteful on allocations that it didn’t hide the problem as well as previous versions.

reply
NooneAtAll3
4 hours ago
[-]
I'm genuinely surprised that python change was even possible without PEP
reply
AdamN
4 hours ago
[-]
Yeah it seems like a miss. I guess the thinking was that it wasn't developer-facing and just an internal optimization. But of course any change to garbage collection will change the memory and cpu dynamics of the process in a material way.
reply
giancarlostoro
3 hours ago
[-]
Makes ya miss having a BDFL. Dang I didn't realize he's 70 now.

https://en.wikipedia.org/wiki/Guido_van_Rossum

reply
metalliqaz
1 hour ago
[-]
It's not a change to the language, it's a change to the cpython runtime
reply
Fizzadar
4 hours ago
[-]
Exactly! Would like to understand more how that came about. PEP exists for a reason.
reply
sega_sai
2 hours ago
[-]
I think reverting is not problem per se, but releasing a highly problematic version without proper testing in such an essential component is.
reply
LaFolle
1 hour ago
[-]
Yeah they noted that it went without PEP. Looks like a PEP will come now if it maintains at par perf.
reply
emil-lp
1 hour ago
[-]
If I understand correctly, this is one of the changes that caused the regression:

https://github.com/python/cpython/pull/117120

reply
nodesocket
1 hour ago
[-]
If using containers I believe this change was pushed in image python:3.14.5-slim-trixie
reply
__loam
1 hour ago
[-]
Python is such a mess.
reply
askllk
4 hours ago
[-]
All these issues were known in previous attempts for removing the GIL. But if Instagram/Meta want it, everyone stands to attention and finds out the obvious problems years later. Kind of like in geopolitics.

I hope Meta switches Instagram to PHP/Hack so they leave Python alone.

reply
simonw
4 hours ago
[-]
The no-GIL work (free-threading) is unrelated to this incremental GC work.

Free-threading actually uses its own, separate GC: https://labs.quansight.org/blog/free-threaded-gc-3-14

reply
brianwawok
3 hours ago
[-]
In the world of AI written code, Python just doesn’t make sense. Converted about 100k lines in the last few months to golang and the performance is life changing. Curious if we will see global Python adoption fall by 75% or more in the next few years.
reply
mau
3 hours ago
[-]
I think humans are still accountable for the code generated by agents.

You are free to switch language but you still need to understand it.

reply
tdb7893
2 hours ago
[-]
With a similar amount of experience with both languages I found Go much easier to read. I've always been a bit miffed why Python is seen as easy to read for experienced developers. I get the syntax is good for short code or people with little experience but my experience is those readability benefits went away quickly with time or complexity.
reply
mywittyname
1 hour ago
[-]
Why are you miffed about it? I legitimately hate reading golang with passion and find python to be pretty intuitive, outside of the odd ambitious list comprehensions. I worked in a golang shop for several years, so it's not just an familiarity situation either.

We are just different. That's not something to be mad about.

reply
Yokohiii
1 hour ago
[-]
In my opinion most interpreted languages today tend to produce very dense code. Fancy call chains and closures interleaving. If you look for a subtle bug those are hard to reason about, you have to know the details of a lot of different APIs.

Go is verbose partly for that reason, but a silly loop is a silly loop. The constraints are clear, you only have to do the logic.

reply
__loam
1 hour ago
[-]
Python is a garbage language. Dynamic types are a disaster for maintaining large codebases and we waste enormous amounts of compute running large systems with it.
reply
lijok
51 minutes ago
[-]
We should all go back to writing assembly
reply
dec0dedab0de
2 hours ago
[-]
I think we'll eventually be generating machine code directly. But until then we should be using code that our team can actually read and understand. If you know go, then that works you, Not everyone does.
reply
backwardation_b
3 hours ago
[-]
nothing about the performance characteristics of python changed with AI so why would you use python over golang if performance is a requirement/bottleneck? Trying to understand the reasoning as to me golang and python are equally simple to write and understand.
reply
Yokohiii
1 hour ago
[-]
If language X is a persons comfort zone, that person will often default to it. Python is certainly more widespread then go.

Also, even if it looks like that to you, there are still people that write code with their own hands.

reply
phainopepla2
3 hours ago
[-]
Regardless of whether golang and python are actually equally simple, python certainly has the reputation of being easier to write and read than almost any other language. That is a big part of its popularity.
reply
zozbot234
30 minutes ago
[-]
Python is not really simple though, the semantics are actually quite bonkers. It just has "simple"-looking syntax, but that only helps you for trivial programs where the bonkers semantics does not get in the way.
reply