Reverse engineering Gemini's SynthID detection
98 points
by _tk_
4 hours ago
| 10 comments
| github.com
| HN
coppsilgold
2 hours ago
[-]
Inserting an undetectable 1-bit watermark into a multi megapixel image is not particularly difficult.

If you assume competence from Google, they probably have two different watermarks. A sloppy one they offer an online oracle for and one they keep in reserve for themselves (and law enforcement requests).

Also given that it's Google we are dealing with here, they probably save every single image generated (or at least its neural hash) and tie it to your account in their database.

reply
Tiberium
2 hours ago
[-]
Seems like a very low-quality AI-assisted research repo, and it doesn't even properly test against Google's own SynthID detector. It's not hard at all (with some LLM assistance, for example) to reverse-engineer network requests to be able to do SynthID detection without a browser instance or Gemini access, and then you'd have a ground truth.
reply
ddtaylor
1 hour ago
[-]
I read a lot of comments on HN that say something is not hard, yet don't provide a POC of their own or link to research they have knowledge of.

I also read a lot of comments on HN that start by attacking the source of the information, such as saying it was AI assisted, instead of the actual merits of the work.

The HN community is becoming curmudgeonly and using AI tooling as the justification.

reply
love2read
1 hour ago
[-]
becoming? under most posts that even in passing mention using AI tools there are multiple people raising their noses talking about how much they hate AI use
reply
pixl97
34 minutes ago
[-]
Eh, just the same people that have been killing tech forums and closing posts on stack overflow for like ever.
reply
armanj
3 hours ago
[-]
kinda ironic you can clearly see signs of Claude, as it shows misaligning table walls in the readme doc
reply
rafram
3 hours ago
[-]
Parenthesized, comma-separated lists with no “and” is an even stronger tell. Claude loves those.
reply
TacticalCoder
2 hours ago
[-]
> kinda ironic you can clearly see signs of Claude, as it shows misaligning table walls in the readme doc

This one is such a gigantic clusterfuck... They're mimicking ASCII tables using Unicode chars of varying length and, at times, there's also an off-by-one error. But the model (not Claude, but the model underneath it) is capable of generating ASCII tables.

P.S: I saw the future... The year is 2037 and we've got Unicode tables still not properly aligned.

reply
dgellow
2 hours ago
[-]
I mean, just reading the readme content it is pretty obvious it is Claude
reply
khernandezrt
3 hours ago
[-]
Ok i get that eventually someone was gonna do this but why would we want to purposely remove one of the only ways of detecting if an image is ai generated or not...?
reply
akersten
1 hour ago
[-]
Fundamentally it's a fuzzy signal and people shouldn't rely on it. The general public does not understand Boolean logic (oh, so the SynthID is not there, therefore this image is real). The sooner AI watermarking faces its deserved farcical demise the better.

Also something about how AI is not special and we haven't added or needed invisible watermarks for other ways media can be manipulated deceptively since time immemorial, but that's less of a practical argument and more of a philosophical one.

reply
StarlaAtNight
1 hour ago
[-]
I’m not very well read on the topic and you seen to take a strong “con” stance. Curious to hear why you think it deserves such a demise
reply
love2read
59 minutes ago
[-]
People think that just because they have a way to prove that an image is AI, their worries of misinformation are solved. Better to acknowledge that wherever you look people will be trying to deceive you even if their content won't have as obvious an indicator as SynthID.
reply
lokar
3 hours ago
[-]
It was always going to be available to some people, but not everyone would know or believe that. Now they will.
reply
pixl97
29 minutes ago
[-]
Much like every other thing in the tech world. He'll, it's why AI will kill us off eventually.

If a system depends on every person on the planet not doing one particular thing or the system breaks, expect the system to break quickly.

This is an especially common trope in software. If someone can make software that does something you consider bad, it will happen. Also it's software. There is no difference between it being available to one person or a million. The moment the software exists and can be copied an unbound number of times.

reply
subscribed
2 hours ago
[-]
More likely than not it would be used to deanonymise the author.

So it's a "no" by default.

reply
raincole
3 hours ago
[-]
Uh... you can do this pretty easily since day 1. Just use Stable Diffusion with a low denoising strength. This repo presents an even less destructive way[0], but it has always been very easy to hide that an image is generated by Nano Banana.

[0]: if it does what it claims to do. I didn't verify. Given how much AI writing in the README my hunch is that this doesn't work better than simple denoising.

reply
M4v3R
3 hours ago
[-]
SynthID is visible in some generations (areas with a lot of edges, or text), I wonder if this would make them look better.
reply
m00dy
23 minutes ago
[-]
reply
doctorpangloss
2 hours ago
[-]
Okay... this tests its own ability to remove the watermark against its own detector. It doesn't test against Gemini's SynthID app. So it does nothing...
reply
kelsey98765431
3 hours ago
[-]
if you downscale then upscale it removes the watermark
reply
sodacanner
2 hours ago
[-]
I don't understand all the handwringing. If it's this easy to remove SynthID from an AI-generated image then it wasn't a good solution in the first place.
reply
raincole
2 hours ago
[-]
There is no solution. I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.
reply
DonsDiscountGas
2 hours ago
[-]
If you want to make an AI generated image but don't want other people to know that it's AI, the most obvious solution is to not use Gemini. Synth ID is watermarking. It's only ever going to be useful to good actors, who want an AI generated image and aren't trying to hide the fact that it's AI generated.
reply
dummydummy1234
1 hour ago
[-]
Never underestimate that people are lazy.
reply
sodacanner
2 hours ago
[-]
Sure, and things like this help drive home that SynthID wasn't a solution at all.
reply
levocardia
2 hours ago
[-]
Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.
reply
jfim
1 hour ago
[-]
That's pretty much impossible though.

One workflow that some artists use is that they draw with ink on paper, scan, and then digitally color. Nothing prevents someone from generating line art using generative AI, printing it, scanning it, and coloring it.

And what if someone just copy pastes something into Photoshop or imports layers? That's what you'd do for composites that mix multiple images together. Can one copy paste screenshots into a multi layer composition or is that verboten and taints the final image?

And what about multi program workflows? Let's say I import a photo, denoise it in DxO, retouch in affinity photo, resize programmatically using image magick, and use pngcrush to optimize it, what metadata is left at the end?

reply
pixl97
26 minutes ago
[-]
Next comes registration your camera with the government to ensure you're not doing "bad" things with it.
reply
raincole
1 hour ago
[-]
If the premise is that everyone would just agree on the same protocol, I have an even more unbreakable solution: every image has to be upload to a blockchain the moment it is (claimed to be) created. Otherwise it's AI.

If only everyone just agrees with me.

reply
Diggsey
2 hours ago
[-]
Which works for about 5 minutes until someone leaks a manufacturer's private key or extracts it from a device...
reply
IncreasePosts
2 hours ago
[-]
How many minutes do you think it would take before someone figured out how to crack that?
reply
subscribed
1 hour ago
[-]
On Pixels and iPhones it would be impossible since they have actually secure hardware that could both hold the keys and sign/verify the image.
reply
IncreasePosts
1 hour ago
[-]
The camera module sits outside the secure area, meaning it would need to send data in to be signed. How does the phone know that it's getting legitimate data from the camera module, or data someone else is just piping in? Also, you could probably get a fairly high quality image by just taking a photo of something AI generated in the right lighting conditions.
reply
rustyhancock
2 hours ago
[-]
Yes. This kind of project needs aggressive red teaming, it leads to better products and we need excellent products in this space.

This project proves what red teaming was in place wasn't good enough.

reply
andrewmcwatters
3 hours ago
[-]
> We're actively collecting pure black and pure white images generated by Nano Banana Pro to improve multi-resolution watermark extraction.

Oh hey, neat. I mentioned this specific method of extracting SynthID a while back.[1]

Glad to see someone take it up.

[1]: https://news.ycombinator.com/item?id=47169146#47169767

reply
raphman
3 hours ago
[-]
FWIW, I had Nano Banana create pure white/black images in February, and there was no recognizable watermark in them (all pixels really were #ffffff / #000000 IIRC).

Meta: your comment was marked [dead], like a few other constructive comments I saw in recent days. Not sure why.

reply
andrewmcwatters
2 hours ago
[-]
I suspect they strip the SynthID for these specific cases to prevent exfiltration of the steganography.

I appreciate you pointing it out, but this account is banned. Thank you for vouching though!

reply