if you throw millions of tokens at IDA Pro MCP with the right prompt lets just say security by obscurity fails miserably because there is no obscurity when the LLM chews through the decompilation.
Isn't this what technology progress looks like? Industrial tools allowed mediocre people to improve their productivity by orders of magnitude which is how we managed(in the past) to build so many amazing things with less human toil and suffering than previous generations.
Good, fsck NSA, that's the last organization I'd ever want to have access to Mythos. I hope this administration's incompetence will prevent them from regaining access for as long as possible
If a government can just seize the product of someone else's labour, either they will end up as slave owners or without willing workers.
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
I wouldn't count them out.
That sort of proves the opposite point, assuming you're referring to Dual EC DRBG, because the flaw was noticed very early on, by people who weren't even involved in its development.
They probably already have access to Sentinel, so they wouldn't need to train their own.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
Harness is important for model performance, but weights are surely mode important, without that you would have haiku doing the work.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
John Cook?
Don't forget, its no longer cool to say that now that the public has pushed back. The fact they all changed their tone away from taking jobs tells you that it was all just entirely marketing.
It just feels like people are starting to reach for conspiracy theories rather than engage with the idea that these models might actually be dangerous.
[1]. https://thehill.com/policy/technology/5936339-ai-cybersecuri...
It's google in a box. Great achievement, makes knowledge work faster, but please stop bothering everyone else.
The Uber and Groupon people became billionaires, so the "Simulated Intelligence" folks will also achieve it. No need to worry and drown everyone in these bs stories only non-tech people believe.
There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.
Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.
Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast
If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.
These words make no sense. Anthropic delayed mythos/fable rollout. A mythos model without safeguards would have been a pretty bad idea, and they sacrificed a ton of revenue and risked being scooped by any of the other labs in the meantime. Frontier models are only frontier temporarily until the next lab releases their model. Of course they are a company and need to act in their own best interest.
It is also clearly serious the problems we need to think about as we march quickly towards even more capable systems. Why on earth is it a problem to point this out?
> If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
What a really weird take; they employ some of the best safety and alignment teams in the industry and this is an active area of research that they are campaigning for more attention on. You complain about them “doom trolling” and then complain they don’t do anything about…the doom? No sense at all.
It is perfectly consistent to (1) sound an alarm and (2) March full steam ahead as quickly as they can. If they don’t do (1) that’s unethical. If they don’t do (2) someone else will. I would rather someone like Dario align these models than the CCC. Plus it would be nice not to have a war over Taiwan which is inevitable if China gains enough of the upper hand in this AI race.
The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.
The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.
a link to the PR or Changelog would strengthen this comment that it actually happened?
When this goes we might well see a recession. Not that anyone responsible will be worse off, of course.
Of course this is a profitable technology, and it doesn’t matter if any of the labs are profitable today or not. Running at a loss is a perfectly rational strategy.
dozens upon dozens fired for no reason
so US "intelligence" is going to go even further backwards
* https://www.yahoo.com/news/politics/articles/trump-acting-ch...
November is going to be insanity
What kind of sick joke is that
When you want to reorient the government, it's much easier doing it with a smaller more loyal force. Now introduce tools that make mass surveillance easier and less accountable.
Like that's not a bad thing for them, that's what they want to do.
---
Back to the article, I'm not shocked that a massive LLM company speed running into the brick wall that is the US government; just thought it would be OpenAI, but Sam Altman is truly the best bottom feeder the game.
Also fully believe that Anthropic is hoping that public sentiment is on their side but more Americans hate AI companies than Trump so it's not going to go how they want.
Give it maybe 3-6 months before the Trump Admin talks about openly nationalizing Anthropic.
When you say without reason do you mean without cause?
Yes. But unlike the rest of us, NSA didn't have to if the administration had thought about it for 30 seconds before sending their letter. It's a stupid own-goal.
The DPA only gives that power to the President [1].
[1] https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
This would not be a particularly big stretch here, either.
Understatement. They have 14 offices, only 4 of them are in the US (6 are in EMEA, 4 in APAC).
Did Hegseth pull his supply-chain risk BS?
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
Propaganda.
No, they don't.
https://en.wikipedia.org/wiki/Room_641A
Yeah, they did (and probably do).
Are you suggesting they broke TLS or that they've somehow acquired every private cert generated?
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
And these days (especially post-Snowden), many (most?) companies encrypt data when sending between servers within their own (private network) infrastructure.
Do you know what hypervisor is managing it? :)
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
Perhaps you're exaggerating for effect, but that also undermines your point.
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
What? That's not only relevant to the point, it's incredibly relevant. If the NSA is only able to recruit 2% of the math PhDs they approach, then that's important information.
"More than one" is not particularly useful; you seem to be dodging the question because it undermines your argument.
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math phd market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I don’t think you have context on what math phds are making in entry level positions, post-docs, or adjuncting. I just picked a random entry level NSA role on LinkedIn (doctorate + 0 yrs) and they’re offering solid six digits. There are tenured faculty (post-doc(s) + 5ish yrs) who don’t make that.
IPO incoming.
But Mythos is still only an advanced LLM so I am not sure what all this breathy fuss is about; it sounds like the PR war more than anything.
If the NSA aren't themselves training technologies that are at least as powerful, that would modestly surprise me.
Not that you need an LLM to monitor the risks to the USA. You just need Tulsi Gabbard's emails.
Any citations to your statement that NSA produces nothing? Or do you have a strong argument or evidence to support this?