Seems like this is pretty clearly a case of fraudulent misrepresentation (https://www.law.cornell.edu/wex/fraudulent_misrepresentation) which kinda nullifies the contract, if I understand correctly:
Fraudulent misrepresentation is a tort claim, typically arising in the field of contract law, that occurs when a defendant makes a intentional or reckless misrepresentation of fact or opinion with the intention to coerce a party into action or inaction on the basis of that misrepresentation.
To determine whether fraudulent misrepresentation occurred, the court will look for six factors:
A representation was made
The representation was false
That when made, the defendant knew that the representation was false or that the defendant made the statement recklessly without knowledge of its truth
That the fraudulent misrepresentation was made with the intention that the plaintiff rely on it
That the plaintiff did rely on the fraudulent misrepresentation
That the plaintiff suffered harm as a result of the fraudulent misrepresentation
Like most claims under contract law, the standard remedy for fraudulent misrepresentation is damages.[Edit: by "nullify" you probably mean "void" or "voidable" which are remedies in equity, and the "never read it" argument carries even more burden there. As the citation notes the traditional remedy for contract issues is damages (i.e., cash payment).]
I've heard suggestions like having white/invisible text in resumes for tricking applicant tracking systems,[0] but it's apparently mitigated by showing recruiters the plain text version of the resume.
[0] example: https://news.ycombinator.com/item?id=36857909
Also, the mitigation can probably be fooled with ligatures since they are only verifying the letters alone as far as I skimmed.
I don’t even understand the threat model. Is my opponent in a court case going to use this on the PDF they give the court? Surely the judge will be pretty annoyed since you can’t even ctrl+f in the files then.
[Edit: The point here is not to prove some massive "gotcha", but rather demonstrate that there are a whole class of vulnerabilities that these pipelines are subject to. There will be follow-up posts that pack much more punch.]
Also, I hope the „lame exploit“ I just edited out was not too offensive, it’s always great when people try to find attacks to make systems more safe.
But, we're working on a lot of these (as we encounter them in developing Tritium), and the point really is just to demonstrate that LLMs can be blind to ineffective implementations of the specs and other tricks.
As mentioned in the accompanying LegalQuants post, we see a lot of these available in the pipelines of applications like Claude for Legal, Harvey, Legora and others.
The most nefarious case here requires crafting a number of custom fonts to do character-swapping. It's less discoverable but may be sanctionable to your point.
But bear in mind this particular "attack" was vibe coded in a day or two and most of the frontier models fail to pick up on it. As "AI native" firms come on line, and aim to be increasingly end-to-end automated, these will become real legal issues.
And there will be a lot of them available.
However, wouldnt this be a rather risky move? Courts authorized the discovery, so I imagine the judge might loose their marbles and throw the hammer at them if this came to light.