The scoreboard was already lopsided
Chris Inglis, former Deputy Director of the NSA, described the state of cybersecurity as a soccer match where the score was 462 to 456, 20 minutes into the game. That is to say, cybersecurity is all offense, no defense, we are far too focused on attackers and no one is guarding our real goal, the assets we protect. And that was the assessment before large language models showed up. Now OpenAI and other vendors are calling for action on cyber defense and AI.
But recently four things have happened all at once. There is more software in existence than at any point in history, and the curve is getting steeper. AI is writing more and more code, even in top engineering orgs, so less of that software is understood by any human. AI allowed attackers to scale their attacks, and become a bit stronger (while many lament that AI is enabling attacks, all this knowledge was already available on underground forums for the motivated threat actor). And attacks stopped being news events and basically became as commonplace as discussing the weather. Continuous, ambient, always on and while governments have adapted to this to bring in legislation like the EU’s Cyber Resilience Act, users are increasingly aware of cyber threats.
Drew called agentic coding "the steam tractors of software," which I like because it's honest about what a steam tractor actually did. It didn't make farming safer. It made farming bigger, but at the same time everything downstream: supply chains, storage, spoilage, had to be reinvented to cope. Software factories are starting to become a thing, but nobody has reinvented the downstream yet.
What early access to Mythos revealed
We have a little confession to make, Semgrep had early access to the advanced frontier AI model developed by Anthropic known as Mythos before it even had that name, which means our security research team had hands-on experience with the model while everyone else was reading press releases. Our research team was, in Drew's words, very impressed, not a phrase most security researchers hand out casually! We found 100 runtime-verified vulnerabilities across top open source projects.
What people don’t realize though is that we didn’t need Mythos access to find these vulnerabilities, because harnesses matter more than you'd think. Our approach seeded our agent with a starting location based on static analysis, then implementing Skills that define the house rules (no agent escaped containment in our experiments!) and then brought dynamic testing into the mix, allowing the model to confirm a hypothesis and verify its own findings as part of the loop. Our testing showed just how much specialized tools help. A model with a grep and a file reader will use a grep and a file reader. A model with a proper program analysis tool will ask better questions, and then combining that static analysis with dynamic analysis gives an agent a lot of tools.
However it’s expensive, which is both good and bad. Right now cost is the only thing standing between "attackers can do this" and "attackers do this to everyone, constantly." There is no reason to believe that stays true. Inference costs have gone in exactly one direction for three years running. And open weight models have been shown to punch above their weight class for a lot less money in tokens.
So that's the defender's window, but it is not a very wide one.
The two levers: Semgrep Mandoline and the Context Engine
If you want an LLM to find bugs in a codebase, you have two knobs. You can fill its context window yourself, threat models, endpoint inventories, dependency graphs, application logic, all the business context that simply is not recoverable from reading source code. Or you can let the model drive, looking things up dynamically through tools as it decides what it needs.
The first option hits a wall fast. Context windows are finite, and stuffing them with things the model doesn't need is how you get expensive, confused agents. The second only works if the tools are precise.
Mandoline
Semgrep has been working on an internal project code named ‘Mandoline’ which is a program analysis toolkit built for models rather than humans. Forty-seven languages, MCP and LSP interfaces, a serious dataflow and taint-tracking engine, three layers of caching, accuracy verified against benchmarks. It does program slicing and deep reachability, and it shows you a coverage map of where the LLM actually did its work, so you can audit your agent’s audit.
Today, coding agents, whether they’re being used for code or security, rebuild the codebase from text every single time. Grep, read, guess, read again. That reconstruction burns tokens, burns context, burns time, and when the job starts looking too big the agent spawns subagents to break it down, which burns more of everything. Instead, Mandoline loads the code graph once and the model spends its budget reasoning about the application instead of rediscovering it.
The benchmark numbers are the interesting part. Hand Fable 5 the graph and its navigation calls fall by half, 148 down to 74. Sonnet 5 sheds a third. File reads drop hardest of all, 41 down to 10, because one graph query answers what used to take a scattering of greps and reads. The baseline runs couldn't consult a graph at all, so they brute-forced it instead, somewhere between 77 and 107 text searches per run. That's a model reading your codebase with its face pressed against the glass. The exception is Haiku 4.5, which went from 159 to 157 and effectively didn't move. Better tools don't rescue a model that won't use them well, and publishing that next to the wins is more persuasive than leaving it out.
The Context Engine
The other half is the part nobody wants to build because it isn't fun, and it's worth being clear about why Semgrep is spending engineering time here instead of on something that demos well. Go back to the cost problem. An attacker needs one good run against your codebase. You need a good run against every repo you own, every day, forever. That asymmetry is brutal on a defender's budget, and it means the deciding question for defense isn't whether the analysis works, we've established it works, but whether you can afford to keep doing it. Caching is what makes continuous affordable. You stop paying a second time for a conclusion the model already reached last Tuesday.
Memory does something a budget can't. Not everything a model needs to know lives in the code. Maybe you have an endpoint that is unauthenticated on purpose. Or a finding got triaged and dismissed by a human two years ago for a reason recorded in a ticket nobody has opened since. Without somewhere to put all of that, an agent rediscovers the same non-issue on every single run and hands it to you again with fresh confidence, which is exactly how you teach a team to stop reading the tool's output. Semgrep's other finding sounds like it cuts against all this but it doesn't: when it comes to the harness, less is more. Don't script the model's reasoning, and don't build the analysis into the scaffolding. Thin harness, thick context. Give the model excellent tools and a real memory to consult, then get out of its way.
What this means for the rest of us
The honest read is that Mythos-class capability arrived, it works, and cost is the only brake. Attackers get the same models we do, on the same schedule, at the same price. What defenders have that attackers don't is the code itself, the threat model, the deploy history, and the institutional context, but that only works to your advantage if you are using it.
Shift left was never wrong. It was just never sufficient.
Write-time prevention stopped being optional a while ago but it never stopped being hard. But there are thirty years of software already in production that nobody is going to rewrite, and the attackers are hunting through it with the same tools you have.
Where to start
Three things you can do this week:
Watch the talk. Drew's full session is in the video library. The Mandoline benchmark section earns its twenty minutes even if you never touch Semgrep.
Get coverage for AI-generated code. Add the AI Security, Agent Skills, and Shadow AI rulesets. The Agent Skills rules cover techniques from campaigns already running against Claude Code, Cursor, and Codex users. Not theoretical.
Move enforcement earlier than the PR. Semgrep Guardian installs into the coding agent and checks code against your policies the moment it's written. Given how many PRs these things open, waiting for review is waiting too long.
Somebody has to go right and it should be you.