Anthropic's Mythos has created a divide between the community of security leaders, exploit developers, and the general AI enthusiast community over both the facts of what has been achieved and the correct interpretation of those facts.
My background: I founded a company that focuses on code security (Semgrep) which has built an extremely popular open-source security code scanner. Before that I was at MIT researching binary exploitation – although I'm not a talented exploit developer like some of my friends. I've been asked by several friends to write up my thoughts on Mythos; here's my take on separating hype from reality.
My position: hype aside, Mythos is a significant advance and it's not easily replicable with existing free/flagship models today. The trend of advanced models has been giving offensive teams a big advantage, so I'm glad that Anthropic & OpenAI are gating model access for now. This has implications for the product we build (Semgrep) as well as for practitioners, who will likely need to increase AppSec spend to prevent attackers from finding 0-days first.
Rebutting the Hot Takes
First, rebutting some of the dismissive takes:
"The vulnerabilities they showed aren't a big deal, because humans can find them too."
But humans didn't. Hindsight is 20/20, and with vulnerabilities – which are exponentially distributed in time – it is statistically harder to find a bug in old code vs new code. Finding a 20-year old bug is much harder than finding a bug from yesterday. Any fully automated capability that can find on the order of thousands of new bugs of this nature and produce exploits for them at this scale is novel; the only comparable capability in recent memory is fuzzing.
"Nothing is new except the labs are just spending money to find bugs; we know there are a ton of bugs in these pieces of software, people just don't care about spending the money on <humans/models> to find them"
The Mythos post was the first time Anthropic released the overall cost to discover the vulnerability and build an exploit in one project: $20K. I'm not in the market for exploits, but my sense is that's within striking distance of market price for that vulnerability. And models will only get cheaper, whereas (human) labor costs will rise.
Even more persuasive to me than the Anthropic numbers (which are only inference costs, not other human analysis time) is that small teams are putting up impressive numbers with existing models, like the "100+ kernel bugs in 30 days" research from a small team of two researchers: $4/bug."Mythos isn't a big deal, I gave the vulnerable examples with a description of the vulnerability to this cheaper model and it found them too"
A lot of vendors rushed to show how "our tool can find Heartbleed, too!" after it was initially announced. Finding the same vulnerability after the fact doesn't mean much without an understanding of false positive rates, and also how generalizable your technique was. (Also, Mythos' biggest advance is in exploit generation rather than vulnerability detection–see below).
My colleague Kurt Boberg at Semgrep just published a detailed analysis "Needles and haystacks: Can open-source & flagship models do what Mythos did?" which more clearly illustrates how few models can actually find the vulnerabilities discussed in the Mythos blog post when not given very strong hints."Anthropic is lying; they don't actually have this many vulnerabilities"
In their post, Anthropic publicly committed to revealing other vulnerabilities (even providing cryptographic commitments so we can tell they haven't changed the description later). My conversations with those with early access gives me confidence that the provided sampling is representative and their vulnerability × severity claims are at least the correct order of magnitude.
Post-Mythos Trends
The above dismissals share a common theme of underestimating Mythos or viewing Mythos as a one-off instead of a broader industry trend. The real focus needs to be on what happens when these capabilities continue to improve over time, get less expensive, and become widely available. Here's what I believe to be true in the post-Mythos world:
This trend favors offense, which means defenders will need to spend more on AppSec to keep up. The world has historically benefited from the fact that vulnerability searching is a semi-rare skill and there's not huge demand for it. The models significantly alter the scarcity of that skillset. If models are high quality at finding vulnerabilities and generating exploits with little aid, this calculus changes and the cost to exploit software (already quite low) will drop even more to expose "who has been coding naked" (to paraphrase Buffet). It was before, but now it's an even better time to be on the offense.
Some people are optimistic that this will burn through vulnerabilities and then we'll enter a golden era of fewer vulnerabilities. I don't think we have enough information to say, as it depends on the distribution of vulnerabilities discovered by the model. If everyone finds the exact same vulnerabilities with models, great; if not, trouble ahead, because an attacker only needs to find one different vulnerability for successful exploitation.
Both states suggest a meaningful increase needed in the amount of dollars spent on vulnerability discovery. This may also change the nature of open source, as some teams conclude that making source code available is now too costly for them to afford continuous vulnerability scanning (true or not). Our team at Semgrep had reported some security issues to the Cal.com team shortly before they decided to stop releasing their open-source due to AI-driven security fears.
The biggest jump with Mythos is related to exploit generation, which is a critical capability for correctness of vulnerability production from models. Models still hallucinate a lot: producing a working exploit acts as an oracle, an infallible authority about whether or not this is a real issue, which means the hallucinations can be ignored.
This is my experience as well when giving Opus 4.6 reversed binaries or decompiled code; it wants to run the target and then the exploit, because it can observe whether the program crashed or we have control of the instruction pointer. As another security engineer shared: "Having an oracle is crucially important. So much so that the agent constructed its own when instructed not to run the binary."
That's why Mythos is a big achievement – it can generate exploits in numbers and sophistication that previous models could not at all, which are then used to prioritize exploitable vulnerabilities.
Human vulnerability research will become an artisanal skillset: Mythos is a clear improvement in finding vulnerabilities: I will leave to others whether it is exponential or linear. But looking at it in isolation ("humans can find this too") leaves out the trend, which is dramatic over the past few months.
I recommend Thomas Ptacek's Vulnerability Research Is Cooked for a deep practitioner view on the topic: "Within the next few months, coding agents will drastically alter both the practice and the economics of exploit development. Frontier model improvement won’t be a slow burn, but rather a step function. Substantial amounts of high-impact vulnerability research (maybe even most of it) will happen simply by pointing an agent at a source tree and typing “find me zero days" …So I think we’re living in the last fleeting moments where there’s any uncertainty that AI agents will supplant most human vulnerability research.”
I think there is still a future for human vulnerability research, but it probably looks more like finding the next creative category of attacks (e.g. Spectre) rather than the 1000th Linux kernel 0-day.
Larger than life marketing: Anthropic's momentum and the spirit of fear in the software industry have created a chamber in which the message echoes larger than life. For instance, many of the same things people are saying about Mythos' capabilities could have been said about fuzzing when it first became popular. But the announcement of Google's OSS-Fuzz initiative didn't make the Fed chairman call the banks for an emergency meeting – maybe it should have!
Open Questions
Broader thoughts on what this means for the market and questions I'd like to see answered / things I'm looking out for:
We don't know what the overlap of LLM-discovered vulnerabilities is with existing techniques. Vulnerabilities are very dense (e.g. we are nowhere close to finding all the vulnerabilities in most programs, including hard targets like the Linux kernel). With humans, there is a lot of searching under the streetlights in vulnerability hunting. LLMs are finding lots of things humans missed because they are a different kind of intelligence. Are they also clustered in how they find vulnerabilities?
A key question to understand will be of the hundreds of LLM-discovered vulnerabilities individuals are finding – what percentage of them are overlapping after they get through coordinated disclosure?
All novel techniques have a yield falloff; so will LLMs. This may be closely related to the previous point. If LLMs end up thinking alike, they will concentrate on common areas as well (different common areas than humans!). Maybe human creativity can compensate for this.
Fuzzing actually fell off way faster than I expected; Project Mayhem ended up not being as high impact as I had hoped.
If the yield doesn't fall off, companies will need to significantly increase AppSec spend to make sure they find the 0-days before attackers do (as Google has effectively done by spending $$$ on oss-fuzz to make fuzzing less useful to adversaries).Security is becoming less understandable by humans. Different kinds of intelligence will result in some truly sci fi scenarios. We saw this on our team where a senior security researcher told us "the vulnerability the LLM discovered is a false positive; this exploit code doesn't work." The LLM then showed that the human reviewer was incorrect. To quote Dave Aitel & Dan Geer: "Autonomy means losing the plot. Humans are about to find themselves on the outside looking in, facing security decisions made by AIs whose reasoning isn’t just complicated—it’s fundamentally inaccessible."
You will not achieve security by removing bugs (paraphrasing Brad Spengler). The most exciting thing here from a security perspective is not finding lots of bugs, it's that the models are good enough at code generation that we may be able to finally rewrite projects in secure-by-default languages that eliminate classes of vulnerabilities entirely. Mythos' coding abilities (SWE bench scores specifically) are a huge step up over previous models. I am already talking to teams who are talking about using LLMs to "inline" dependencies; I haven't heard of any large projects to rewrite C to Rust using LLMs but I am sure they exist or are coming.
For Semgrep users
Ironically, the same models that are so capable of finding vulnerabilities are quite poor at writing secure code. Last month at Semgrep, we found two significant vulnerabilities. Guess who the committer was on both? Claude. Semgrep had actually spotted the vulnerabilities, but Claude ignored it. This is something we're fixing now by moving to hooks over MCP.
I sent Semgrep users and customers a message about how we should think about these model announcements and where Semgrep is headed. We're excited for the new world!
.jpg)
