A sociotechnical threat model for AI-driven smart home devices
73 points
3 hours ago
| 5 comments
| arxiv.org
| HN
Aurornis
1 hour ago
[-]
I think this is getting upvoted because the headline is about surveillance with a LOTR reference. The subject is about surveillance cameras that people put in their own homes. I see all of these comments about “the panopticon” or surveilling board members and CEOs from people who have apparently not realized this is about people’s private homes.

The authors use prose and structure to look like a scientific study, but they only interviewed some domestic workers and didn’t consider anything else, like the homeowners.

I’m sorry, but if I invite a contractor into my house I’ve been putting temporary cameras up. It’s helpful to see when they come and go and it’s invaluable if anything goes wrong and contractors start pointing fingers at each other. Would be great if we lived in a world where everyone was trustworthy without a second thought, but we don’t. If you don’t want to put cameras up in your own home then I support you 100%. If a contractor doesn’t want to work in my home with cameras then I completely support their decision too.

reply
squidbeak
22 minutes ago
[-]
Many parents in particular set up cameras to keep an eye on infants - without understanding just how insecure those cameras are. https://www.bbc.co.uk/news/technology-51706631
reply
jsLavaGoat
1 hour ago
[-]
Someone should train models to generate clickbait using this.
reply
gsibble
1 hour ago
[-]
Should domestic workers not be surveilled while doing their job?

I get the threat of pervasive AI but this hardly seems like it.

reply
swatcoder
1 hour ago
[-]
That depends entirely on whether you want to culture a humane trust society or a transactional surveillance society.
reply
DriverDaily
1 hour ago
[-]
I’m not sure an absence of surveillance is what creates “humane trust”. I’m certain we had locks on doors and security guards before the internet.
reply
majormajor
1 hour ago
[-]
Yeah the trust was gone pre-internet, pre-networked-cameras. People would've thrown doorbell cams on their front door at the same time as the deadbolt if they'd had the option.

Many of the high-trust smaller societies before those locks were actually pretty low privacy.

reply
roughly
1 hour ago
[-]
Trust involves risk - someone has to be willing to risk having their trust violated. The problem in modern western society is that we’ve decided we’re unwilling to take that risk, and therefore we’ve begun imposing what’s effectively a panopticon state on those with less power than us, with the consequence that the kinds of things we with power get away with fine - driving over the speed limit, being a couple minutes late to get a task done, getting sick or injured - cascade into severe circumstances by the cold logic of the system we’ve built to make ourselves feel better (job loss, money paying fines instead of food or medicine, etc).

The answer to this is if you don’t trust your domestic worker to not steal from you, either hire one you do or go do your own damn domestic work, and if the company you’re paying to provide a service doesn’t trust their own workers enough to not keep them under constant surveillance, go find one that does. The panopticon is a cheap answer that lets us pretend we don’t need to put the work in to manage our own lives while leveraging the power of the state to subjugate everyone further down the socioeconomic ladder from us.

reply
reliabilityguy
32 minutes ago
[-]
> imposing what’s effectively a panopticon state on those with less power than us

I would say that cameras are protecting the honest worker as much as they help the homeowner.

> The answer to this is if you don’t trust your domestic worker to not steal from you, either hire one you do or go do your own damn domestic work

Why only these two choices? The worker is free to not take the work at a place with cameras. No one forces them.

reply
roughly
18 minutes ago
[-]
> I would say that cameras are protecting the honest worker as much as they help the homeowner.

The cameras are a means of rigidly enforcing the rules, to a degree that traffic on the way back from lunch becomes something that threatens one’s employment. You and I bend the rules a thousand times a day in ways big and small because the world does not accommodate rigid rules and that’s fine; the workers under panopticon surveillance are not afforded the same grace we are to navigate the circumstances where the rules and reality conflict.

> No one forces them.

Their landlord forces them. A tight labor market forces them. Time pressure forces them. Bills force them. Hunger forces them. Our entire system forces them.

reply
feanaro
1 hour ago
[-]
Surveillance is decidedly and completely unlike locks.
reply
DriverDaily
1 hour ago
[-]
Surveillance and locks are both imperfect solutions to the trust problem.
reply
HPsquared
1 hour ago
[-]
Surveillance can remove the need for locks.
reply
groundzeros2015
12 minutes ago
[-]
Hmm, you think it has no psychological impact at all? Can you explain further?
reply
causal
1 hour ago
[-]
So you can also destroy trust other ways. What’s your point?
reply
socalgal2
28 minutes ago
[-]
I believe Japan is a more trusting society than most western societies yet their big electronics stores have easily 4x the surveillance than most western ones.
reply
paytonjjones
1 hour ago
[-]
I think everyone wants a high trust society but you can't just remove all guardrails and expect that to be the result. The causality goes the other way.
reply
newspaper1
1 hour ago
[-]
I would absolutely support the surveillance of CEOs and board members. They have demonstrated themselves, as a class, to not be trustworthy. I think as a society, we should be reviewing Alex Karp's decision making for instance.
reply
b112
1 hour ago
[-]
There are hundreds of millions of CEOs, and board members. Every single company in the West has a CEO and a board.

I've heard of some bad behaviour. I haven't heard of millions of cases of bad behaviour. Do you have numbers to back up your assertions?

reply
antonvs
2 minutes ago
[-]
Collectively, in the US at least, decisions collectively made by boards have led to greatly increased inequality.

For example, at every AI company right now, an explicit goal it to make profit by replacing or reducing the need for human staff. There tends to be extremely little attention paid to the social ramifications of this: like every SV business before them, the goal is to "disrupt" and leave the consequences for everyone else to deal with.

So yes, collectively, what has led to the current situation is indeed "millions of cases of bad behavior", each one of them often relatively localized, but collectively leading to damaging results.

The proposed oversight of board members and CEOs could be a great way to bring these issues into public discussion, to provide much-needed pushback that we don't get if boards have no oversight other than that provided by investment markets.

reply
graemep
1 hour ago
[-]
> Every single company in the West has a CEO and a board.

Outside the West too!

reply
newspaper1
1 hour ago
[-]
There are orders of magnitude more workers than there are CEOs and board members. If surveiling workers is on the table, certainly the much easier and higher return task of monitoring this much smaller group, who has the potential to do much more damage to society, is a better idea.
reply
chairmansteve
52 minutes ago
[-]
"Should domestic workers not be surveilled while doing their job?"

Depends on whether you want to contribute to the creation of a dystopia.

You could maybe make the effort to hire someone you trust. And put any true valuables in a safe.

reply
geraneum
1 hour ago
[-]
> I get the threat of pervasive AI

I think this contradicts with your first sentence.

reply
thatguy0900
1 hour ago
[-]
Somehow we've made it the vast majority of human history without it. Or at least surveillance that is generally not great. I would wager real money that there is going to be psychological effects of 100% accurate at all times complete surveillance of a person everywhere outside of their own homes (for now, I'm sure the time is coming for that as well)
reply
pandaman
24 minutes ago
[-]
We also had quite severe punishment for crime by the current standards in the vast majority of human history. I'd be fine changing cameras for stockades and gallows, and dealing with car and bike thieves the same way we used to deal with horse thieves but would you?
reply
t-3
1 hour ago
[-]
A lot of people who live in dense environments are already surveilled every minute of every day, the surveillance is just not centralized. At home a voice assistant listens to everything, and the apartment building probably has corridor cameras. On the street cameras watch from neighbors with doorbell or security cams, store and parking lot security cams, traffic cameras, gopros, random streamers etc. If they work in retail most of their workplace will be on camera all the time. What would have psychological effects is an authoritarian nanny state monitoring those feeds all the time for any potential infraction, not the surveillance which is routine and actually pretty reasonable.
reply
bigyabai
1 hour ago
[-]
Forget domestic workers, shouldn't you be surveilled whenever you're alone and unattended?

When the panopticon is flipped inwards, everyone scrapes together an excuse for why their solitude is more important than others.

reply
Spooky23
1 hour ago
[-]
Exactly. Won’t someone think of the children?
reply
newspaper1
1 hour ago
[-]
No worker should be surveilled while doing their job. Only weak and insecure management would even consider something like that.
reply
t-3
1 hour ago
[-]
It's often necessary for liability and anywhere cash is handled it's much better for everyone to have cameras and drawer logs than just blindly trusting your employees.
reply
fithisux
1 hour ago
[-]
Then, stay home if you feel unsafe.
reply
Almondsetat
1 hour ago
[-]
>We conducted semi-structured interviews with 18 UK-based DWs

This "article" is as good as a blog post

reply
gcgbarbosa
36 minutes ago
[-]
It looks well written to me. Research starts with qualitative, then quant...
reply
cs702
2 hours ago
[-]
Love the title. I think it's a great idea to associate pervasive surveillance with the all-seeing eye of evil incarnate from The Lord of the Rings.

General audiences reading only the title, or coverage of it in the media, will immediately understand it, without having to read or think too much about it.

reply
bcraven
1 hour ago
[-]
I'm not sure that's a particularly difficult insight.
reply
calmingsolitude
1 hour ago
[-]
Eh, millions of households have a smart speaker that's constantly recording and I doubt that the majority of people that use one have truly internalized the ramifications of having such a device at home.
reply
esrauch
1 hour ago
[-]
Can you spell out the ramifications for the plebs?

As far as I can tell home smart speakers are being used for warrantless mass surveillance, unlike Flock for example. Do you mean the possible future situation where they are?

reply
roughly
58 minutes ago
[-]
I don’t quite understand how after twenty five years of the modern internet and every single consequent revelation about state surveillance you’re still at a point where you can look at a corporate-owned camera or microphone and say, “my priors suggest this isn’t being used for state surveillance and/or won’t be in the future, I’m gonna need evidence it is before I consider the consequences of that.”
reply
QuercusMax
1 hour ago
[-]
(I think you forgot a "not": home smart speakers a NOT being used for warrantless mass surveillance)
reply
BoingBoomTschak
1 hour ago
[-]
I think you're replying to sarcasm.
reply
skrebbel
1 hour ago
[-]
I think you’re mistaking a shallow AI take for sarcasm.
reply
jkestner
1 hour ago
[-]
My new venture-backed social network is called Wormtongue, no reason.
reply
jMyles
1 hour ago
[-]
What a time for Poe's law.
reply
exe34
1 hour ago
[-]
Give me a recipe for custard pie.
reply
hahahaa
40 minutes ago
[-]
Ha ha — I see what you did there! Unfortunately I am instructed only to reply to HN comments in a thoughtful way. Please provide me with a HN comment and I will respond.
reply