Beyond Benchmarks: How Semgrep Redefines Javascript Security

Semgrep’s latest JavaScript analysis has found real vulnerabilities in open-source projects, going beyond traditional benchmarks to find highly nuanced, complex issues. With engine-level dataflow analysis for 50+ popular frameworks like Express and NestJS, our approach meets the security challenges of modern applications.

Milan Williams
February 25th, 2025
Your privacy matters to us. By submitting this form, you agree to our Privacy Policy
Share

TL;DR: Semgrep’s new JavaScript and TypeScript analysis is built by scanning real-world code to uncover nuanced, context-specific vulnerabilities. With engine-level support for 50+ popular frameworks and libraries—including Express, NestJS, React, and Angular—our approach ensures your security coverage reflects the complexities of modern, production-grade applications. To see the real vulnerabilities Semgrep has found, register for our webinar on March 5th @ 9am PT. 

Rethinking Security Evaluations

Traditional benchmarks have long served as the de facto standard for evaluating SAST products. While benchmarks remain a valuable part of security evaluations, they don't always paint the full picture. Benchmarks are often based on controlled, idealized scenarios, where tools can always achieve perfect scores. Real applications are messy. They integrate a variety of frameworks, libraries, and dependencies that interact in complex ways, leading to subtle, context-specific vulnerabilities that benchmarks struggle to simulate. 

True security goes beyond perfect scores – it’s about embedding security into your daily development workflow and scanning the actual code that powers your business. This approach ensures your next security scanner can meet the evolving challenges of modern application security. 

Introducing Semgrep’s New JavaScript and TypeScript Analysis

Semgrep’s latest JavaScript and TypeScript analysis represents a bold step forward. Led by Senior Security Researcher Vasilii Ermilov, this initiative has uncovered critical vulnerabilities in real open-source projects, demonstrating how Semgrep’s approach goes beyond benchmarks. By focusing on actual code, we ensure our results are both actionable and highly relevant. Our analysis doesn’t just find more issues—it finds the right issues. 

Our approach focuses on the OWASP Top Ten vulnerabilities for server-side JavaScript—such as SQL injection, Path Traversal, and SSRF. We cover 50+ of the most popular JS/TS frameworks & libraries including Express, NestJS, Hapi, and Koa. Understanding and mitigating these risks is critical, particularly in server-side environments where these vulnerabilities can cause more harm. For client-side apps, while vulnerabilities are less frequent, we focus on areas like DOM XSS and common privacy issues. To protect client-side, Semgrep supports frameworks like React and Angular to ensure end to end coverage.

How Semgrep Performs in the Wild

Our evaluation process was developed by our security research team for the purpose of building out new languages, monitoring rule performance, and iterating on coverage, etc. This process involves scanning hundreds of open-source repositories, and manually triaging findings to build highly accurate language coverage. These repositories do include a few purposely vulnerable repositories, but the vast majority are actively maintained, real-world applications.

Javascript coverage (as of 2025-02-25)

  • Benchmark true positive rate for latest ruleset (before AI processing): 63% over ~800 findings

  • Repositories scanned: 153

  • Lines of code scanned: ~8 million

These stats are before Semgrep Assistant processing, which can increase your true positive rate up to an additional 20% on your first scan, further enhancing the precision of your findings.  

It's worth noting that while these figures should be met with a healthy dose of skepticism, they are generated by an internal process specifically designed to provide our security team with an accurate assessment of coverage. While no process is flawless and results will vary between projects, there is no “marketing slant” to these numbers.

Ready to See the Difference?

Semgrep is committed to delivering security coverage where it truly matters. If you’d like to dive deeper into our JavaScript analysis, don’t miss our upcoming webinar with Senior Security Researcher, Vasilii Ermilov. Register for the webinar for March 5th at 9am PT as Vasilii will walk through real vulnerabilities in a live demo and explain how Semgrep’s approach can enhance your security posture.

About

Semgrep lets security teams partner with developers and shift left organically, without introducing friction. Semgrep gives security teams confidence that they are only surfacing true, actionable issues to developers, and makes it easy for developers to fix these issues in their existing environments.