As AppSec teams get better at preventing “classic” OWASP bugs like SQL injection, XSS, and SSRF with modern tools like Semgrep, they’re still struggling to prevent logic vulnerabilities (ex: IDOR, business logic, & broken auth) from reaching production. Broken access control issues, including IDORs (insecure direct object references), now make up a sizable amount of bug bounty findings. One bug bounty platform noted that roughly half of all high-severity finding fall into that category. Relying on bug bounty programs and pentests to catch logic vulns is reactive and expensive. Customers report that a typical IDOR costs ~$25k to fix when factoring in bug bounty payouts and the cost of engineering time. Logic vulns have also been implicated in high profile data breaches.
In theory, traditional static analysis can identify logic vulnerabilities through the creation of complex custom rules—but in practice, few teams have the time, resources, or expertise required to make that feasible. Therefore, logic vulns go undetected in practice. But now with AI-powered detection, AppSec teams using Semgprep get the benefit of AI’s reasoning ability with the precise static analysis of the Semgrep Engine to reliably detect and resolve logic flaws before they hit production.
We’re excited to announce that we are opening up a Private Beta Program of AI-powered detection to catch IDORs and other logic vulnerabilities. If you want to join the waitlist for AI-powered detection from Semgrep, you can do so here.
Semgrep’s AI-Powered Detection: Actually Finding Logic Issues
A key strength of LLMs is their ability to understand context—variable names, class structures, and code comments—which helps them infer developer intent and recognize when access control is missing or data is being mishandled. In theory, this contextual reasoning should enable an LLM that can identify logic flaws.
But because LLMs are probabilistic, they fail to return repeatable results. Independent security researchers like Sean Heelan have demonstrated this lack of reliability:
“o3 finds the kerberos authentication vulnerability in the benchmark in 8 of the 100 runs. In another 66 of the runs o3 concludes there is no bug present in the code (false negatives), and the remaining 28 reports are false positives.”
In addition to being unpredictable, LLMs struggle with false positives. Our security research team conducted a study and found that 88% of the IDORs identified with Claude Code were false positives.
Semgrep AI-powered detection works by leveraging the Semgrep Pro Engine as a tool an LLM can use to add determinism. By combining Semgrep’s structured code scanning with the LLM’s contextual reasoning, the system can systematically surface real security gaps instead of relying on guesswork, which significantly improves accuracy and reduces false positives.
The results have been striking. We ran a research program with customers to test the viability of our hybrid approach. The results were clear: 80% of the participants uncovered real IDORs serious enough to warrant investigation. For most of the teams, these were issues that traditional scanners and code reviews had completely missed. In one deep dive, a customer found that our precision (true positives/ true positives + false positives) was 61% on the repos they tested. Nearly 3x better than our security research team’s results just using Claude Code (which was 22% for IDOR).
How AI-Powered Detection Works: IDOR Example
When LLMs use the Semgrep Pro Engine as a tool, we see significant uplift in our detection benchmarks with the proportion of actual positives that are correctly detected by the system (recall): Semgrep AI performed 90% better on recall compared to just using Claude Code. How?
To simplify how our technology works in practice, we’ll use IDOR detection as an example. Since IDORs often arise when endpoints expose resources without proper authorization checks, our AI-powered detection agent uses Semgrep to enumerate all routes (e.g., from a routes/ directory or controller files). It can then analyze the associated handlers to check whether authentication or authorization logic (such as requireAuth, checkPermissions, or role checks) is present. Endpoints missing these safeguards are flagged as potential IDOR candidates for further review.
The agent then uses Semgrep Pro Engine’s taint analysis to trace how user input moves from sources like request parameters to sensitive sinks like database queries or responses and flags flows that lack proper authorization or ownership checks. This gives our AI-powered detection a precise, code-level understanding of unsafe data paths, helping them detect real IDORs instead of just guessing from patterns. We have illustrated this process in the diagram below:
Join The Private Beta Today
While LLMs have shown the ability to detect vulnerabilities, the data has shown the most optimal code security solution for AppSec teams blends deterministic, rules based scanning with AI-powered detection. Static analysis will continue to efficiently and deterministically find well-known OWASP vulnerabilities like SQLi and XSS, while AI extends coverage to the harder, business logic flaws—IDORs, broken authorization, and access control gaps—that traditional tools miss.
After our research preview with customers did indeed validate our hybrid approach, we’re excited to open up a private beta for a larger group of customers and interested organizations to test out AI-powered detection and help shape the future of Semgrep’s detection technology.
To get on the waitlist, sign up here: https://semgrep.dev/contact/product-join-llm-detection-beta/. Spots are limited so sign up now!
