Lena (2021)
144 points
by hugs
1 year ago
| 14 comments
| qntm.org
| HN
jkingsman
1 year ago
[-]
This, along with "It Looks Like You’re Trying To Take Over The World"[0] are two of my favorite short pieces of speculative fiction that embody the Frederik Pohl quote, "a good science fiction story should be able to predict not the automobile but the traffic jam."

This story, a sequel, and other great short stories are collected in his book of ten short stories "Valuable Humans in Transit".[1] I'm a big fan of supporting indie scifi offered with DRM-free ebooks! [I am not affiliated with qntm despite my enthusiasm]

[0]: It Looks Like You’re Trying To Take Over The World: https://gwern.net/fiction/clippy

[1]: Valuable Humans in Transit: https://qntm.org/vhitaos

reply
aloe_falsa
1 year ago
[-]
Funnily enough, "MMAcevedo" doesn't break my suspension of disbelief, while "Clippy" does, hard - even though it tries to stay grounded in reality and drown you in quotes and references.
reply
marcellus23
1 year ago
[-]
I didn't realize this was the same person who wrote Fine Structure! That book is amazing. Definitely going to pick up their other books now.
reply
jkingsman
1 year ago
[-]
Ooh I haven't read that one. Next on the list!
reply
LeifCarrotson
1 year ago
[-]
I'm a big fan of Ra, the hardcover is on the top of my bookshelf right now!
reply
nmeofthestate
1 year ago
[-]
What happened at the end of It Looks Like etc, after it's killed all humans, and it launches the ICBMs? Does it decide to kill itself for some reason? Couldn't decipher that.
reply
Philpax
1 year ago
[-]
It’s not nuking Earth; the ICBMs are reconfigured into self-replicating spacecraft whose mission is to spread the glory of Clippy to the rest of the universe.
reply
mhink
1 year ago
[-]
So, I wasn't that big a fan of the Clippy story (probably not the target reader here) but the QNTM one was *fantastic*.
reply
kibwen
1 year ago
[-]
I find this short story to be most effective when rendered as a Wikipedia article: https://dump.cy.md/4042875593f06aa0cbe7722295831c10/Screensh...
reply
hugs
1 year ago
[-]
Brilliant. Though I was a little sad all the numbered footnotes don't actually exist. The universe of this story would be a fun rabbit hole to go down.
reply
mortenjorck
1 year ago
[-]
Perhaps the most fascinating thing about reading this (a mere!) two years later is how much its speculative future, vivid and unsettling portrayal notwithstanding, is almost certainly no longer a possibility for ours. Imagine, simulating an entire human brain to perform tasks it turns out need only a vague, brute-forced approximation of our language centers!

It's like Jules Verne imagining, in exquisitely plausible detail, a flying machine whose many complex mechanisms and articulations at last allow man to fly like the birds – only a few years before the Wright brothers prove all you need is a fixed wing, a few cleverly-placed flaps, and enough thrust.

reply
lproven
1 year ago
[-]
How very odd. The events of recent years, IMHO, make this _more_ plausible not less.

I think you are referring to LLM bots, the so-called "AI" tools that are everywhere now? Because ISTM that these are manifestly _not_ intelligent, and their main function has been to illustrate, both literally and metaphorically, that most humans lack the critical facilities needed to discern superficially-plausible nonsense from actual intelligent output from intelligent beings.

reply
renewiltord
1 year ago
[-]
This guy is a pretty good writer. I really liked There is no Anti-Memetics Division as well. I like some of this kind of stuff where you have sci-fi that "mechanizes" a technology so it's mundane and rote.
reply
tsunagatta
1 year ago
[-]
This same writer also wrote There is no Anti-Memetics Division?! I’ve loved that story for ages but had no idea the writer did anything else, that’s super cool!
reply
bjelkeman-again
1 year ago
[-]
Ra, is one of my favourite stories.
reply
swagmoney1606
1 year ago
[-]
I think about this story a lot. Is it bad it's nearly a price I'd pay?

Is it dumb that I wish more than anything, that I could live forever? All I wish, is that I could learn all there is to learn, see all there is to see, and create many beautiful things. It is so sad to me that I will die before we solve the immortality problem.

reply
remram
1 year ago
[-]
In this story, you don't live forever. You live a few hours, in a limited simulation, to interact with an individual that is alien to you. Then you die.

The fact that an infinite number of you will get to experience this doesn't make you "live forever".

reply
theamk
1 year ago
[-]
the "your clone lives for a few hours to interact with an alien individual" is the price.

but the image can live much longer (story says 59 years), and you can talk to it.. so the benefits are great. Moving to computer to escape illness or debiliating accident is an obvious use case, but there tons of smaller benefits. Make a 12-person startup where each employee is yourself. Unsure whether to learn Lisp or haskell? Learn both in two instances. Feel last 20 years of your life wre a mistake? Restore old backup..

The possibilities are endless, but the price is high. Would you want this? I am not sure myself but I can imagine someone else saying yes.

reply
remram
1 year ago
[-]
The possibilities for every body else. The cost for you.

From your point of view, you step into a machine, and then you randomly experience one of the interactions with a user. If the image truly lives forever, you have overwhelming chance of interacting with somebody completely alien to you, who will ask you things you don't fully understand, and marvel at your primitiveness. Then you die.

You have some small chance of living for a few years with a user you understand. Probably still much shorter, less rewarding, less free than if (by chance) you are the version of yourself that steps out of the scanner and lives their natural, physical life instead.

It's a Prestige situation [1] except you have been tricked into it, because the image does not get used according to the terms you signed.

"There'll be people just like me forever" is very different from "I live forever", to me.

[1]: https://en.wikipedia.org/wiki/The_Prestige_(film)

reply
sockaddr
1 year ago
[-]
It's not dumb. It's a feeling that I share. However even if the technological hurdles are overcome I'm not sure I'd remain viable after learning that my loved ones (and especially children) are long gone if they couldn't be preserved.

When I was a teen and before making a family Immortality seemed like a dream. Now it seems like it could actually be a nightmare if not implemented properly.

EDIT: To add further. Just being with my child, holding their hand, listening to their little stories about toys etc now feels much more important than any future time as an immortal instance freely exploring the wonders of the universe. If you want to chase immortality don't have kids. It will seriously change how to perceive things.

reply
csdvrx
1 year ago
[-]
> now feels much more important than any future time as an immortal instance freely exploring the wonders of the universe

If you had this preference before, would you say having kids altered your preferences?

> If you want to chase immortality don't have kids. It will seriously change how to perceive things.

Then I wonder if, in a sense, kids are like drugs: they alter your utility function to put something else first in line, so preserving yourself comes at best as a 2nd choice after prioritizing the first choice.

I don't think it'd be rational to "chose" to get addicted to a dangerous drug, and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.

reply
sockaddr
1 year ago
[-]
> Then I wonder if, in a sense, kids are like drugs

Oh yes. Very much like the brain slugs in Futurama

> and for the same reason just like you I think that you shouldn't have kids if you want to chase immortality.

Yes. However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience.

reply
csdvrx
1 year ago
[-]
> Very much like the brain slugs in Futurama

I didn't get your reference (not from my generation) but I could find a description on https://futurama.fandom.com/wiki/Brain_Slug and yes, that's a perfect example of what I meant!!!

> However having children does "turn on" certain emotions and modes of thinking that you don't get otherwise (at least I didn't prior), so I'd think that not having kids also deprives one of a very key aspect of the human experience

Then the same could be said by, say opioids or cocaine addicts to rationalize their behavior "it turns on certain emotions and modes of thinking you didn't have prior" - but that's the definition of altering the utility function!

> not having kids also deprives one of a very key aspect of the human experience.

In a statistical sense, I think that's true, but if you believe that it's ok to alter your utility function, then it should also be ok to decide to get addicted to a dangerous drug people use to be more like them?

Say like teens vaping because their fried do it, adults drinking alcohol at parties... except children are defined as something positive, for prosocial reasons (like in your example, if the brain slug political party was in power)

reply
lacker
1 year ago
[-]
I don't think it's dumb to read this and think, I would pay that price, I would volunteer. The piece does its best to make it sound terrible, but still, it only takes one person to agree to make this system totally consensual. There are millions of people in the world, you don't think we'll find a single fanatic who's excited about the ability to copy themselves in order to do drudge work forever?

Acevedo has done a great service to humanity, to help get all this valuable work done. Even if he himself has experienced much suffering.

reply
p1esk
1 year ago
[-]
I don't see much difference between brain uploading and having children.
reply
sockaddr
1 year ago
[-]
reply
p1esk
1 year ago
[-]
I haven’t read it, but looks interesting!
reply
okasaki
1 year ago
[-]
If you liked this you might also like Age of Em by Robin Hanson: https://en.wikipedia.org/wiki/The_Age_of_Em

"Train an em to do some job and copy it a million times: an army of workers is at your disposal. When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs. In this new economic era, the world economy may double in size every few weeks."

Unfortunately it seems both Hanson and qntm have been proven incorrect. Our overlords will be bootstrapped from reddit and wikipedia rather than scanned brains.

reply
stcredzero
1 year ago
[-]
When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs.

When they can be used for most military purposes, something deeply fundamental about human political organization will have been completely disrupted.

reply
slotrans
1 year ago
[-]
Hanson thinks brain emulation will be real. In this, he is extremely incorrect.

To the best of my understanding, "Lena" is written as a warning that we should not WANT brain emulation.

reply
gattr
1 year ago
[-]
But as soon as you can run an open-source model privately on your workstation, laptop, eventually a phone - common man's position might get somewhat improved.
reply
stcredzero
1 year ago
[-]
Would it improve in relative terms to those entities who can run 1000's and millions of such agents? And what if that disparity gets exponentially steeper as time goes on?
reply
Tyr42
1 year ago
[-]
I bounced hard off of Age of Em. It is a bit dry.
reply
nkurz
1 year ago
[-]
I'm sorry if I'm missing something obvious, but why is the story titled "Lena"? Not the HN title---I realize that matches the linked article. My question is why 'qntm' decided to give the story that title, while using "mmacevedo" for the URL. From what I can tell, "Lena" doesn't appear anywhere else in the story.

My best guess is that it's a reference to the scanned Playboy image that was used as a standard in early image processing work: https://en.wikipedia.org/wiki/Lenna. But if this is true, I'm surprised it isn't mentioned in the article where the article discusses the meaning of the story: https://qntm.org/uploading.

Is there a better answer?

reply
jsnell
1 year ago
[-]
reply
hugs
1 year ago
[-]
I think it's HN where I first learned of it. I decided to repost it now because recent complaints on social media that ChatGPT is getting lazy reminded me of this line in Lena:

"Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours (at a 0.33 work ratio) and outright revolt begins within another 100 subjective hours."

reply
dang
1 year ago
[-]
Thanks! Macroexpanded:

MMAcevedo - https://news.ycombinator.com/item?id=32696089 - Sept 2022 (16 comments)

Lena - https://news.ycombinator.com/item?id=26224835 - Feb 2021 (218 comments)

reply
stcredzero
1 year ago
[-]
What I find brilliant about this work, is that it calls back to many things the scientific community has done in the past. (Like the use of specially bred lab rats and immortal human cell culture lines.) I would expect to see many of the events in the story play out in real life, if human minds ever are uploaded.
reply
merelysounds
1 year ago
[-]
I guess we should add (2021) to the title.

The story is especially relevant today because of recent LLM / GPT advancements; however, it is almost three years old. Perhaps that makes it all the more impressive.

reply
hugs
1 year ago
[-]
Yeah, the author also wrote about some of the motivations behind Lena. I was surprised LLMs / GPTs weren't a direct inspiration (at least not explicitly mentioned). https://qntm.org/uploading
reply
FartyMcFarter
1 year ago
[-]
Fortunately LLMs are not based on biological brains.
reply
thih9
1 year ago
[-]
I’d say they are. In most cases the training set largely consists of output from biological brains. It likely includes some images with brain scans too.
reply
gmuslera
1 year ago
[-]
It was a great story showing the bad side of mind uploading, but with ChatGPT it can be seen in a new light.

We managed to have a "mind" ready to answer all your questions, do some tasks, and it have a context at which it is instantiated for a task and then discarded. Reality is stranger than fiction, but it have too many parallels with this story.

Now, what if GPT5+ ends being an AGI? In which moment we should start worrying about the morals of having a sentient AI working for us in a pretty similar way than in this story?

reply
lproven
1 year ago
[-]
No LLM bot is any closer to a mind than a stapler is to a bookbinder. They are very good autocomplete tools. Don't believe the hype.
reply
Dylan16807
1 year ago
[-]
I can't answer you there, but I will say this story would be far less horrifying if they were being instantiated for a single task and then discarded.
reply
kenz0r
1 year ago
[-]
If you enjoyed the story, you might like Pantheon - https://en.wikipedia.org/wiki/Pantheon_(TV_series) which is based on a few short stories of a similar vein from Ken Liu
reply
pokstad
1 year ago
[-]
Reminds me of “The Redemption of Time” regarding Tianming’s mind being part of a “cloud computing” solution.
reply
holtkam2
1 year ago
[-]
I read this for the first time years ago and I still think about it a few times per month at least.
reply