Welcome to a day in the life of a Semgrep Security Researcher on the Software Supply Chain (SSC) Team.
In this post, we’ll break down how we:
Evaluate security vulnerabilities affecting open-source software packages
Write Semgrep rules to enable users to prioritize fixing issues that matter
Prioritize vulnerabilities
Before reading, you should know about Semgrep’s Reachability Analysis. As a quick refresher, we provide code-scanning reachability analysis. This means that Semgrep researchers review security advisories and reverse engineer patches to identify the functions that will introduce risk and expose you to the vulnerability, if used in a vulnerable way in your code.
Get in line, you pesky vulnerabilities
We want to spend our time and resources addressing the most important vulnerabilities. Most vulnerability disclosures come with a CVSS score, which is a rating from 0 to 10 providing a “qualitative measure of severity”. While it’s not a perfect measure of risk, CVSS scores are a standardized starting point for identifying how dangerous a vulnerability could be. Scores below 7 are considered “low” or “medium” severity, while scores of 7 or above are considered “high” or “critical”.
We also recognize that most vulnerabilities are in software that simply isn’t used by our customers. We utilize the visibility we have from customers to identify which software dependencies have the highest impact on Semgrep users.
We prioritize vulnerabilities using different parameters, such as severity, CVSS, etc. That being said, we provide coverage through other Semgrep generated rules for all vulnerabilities, regardless of severity.
Now to get into rule writing. First, we need a queue of vulnerabilities deserving of a Semgrep rule. Currently, our team manually reviews all High & Critical severity advisories for supported ecosystems dating back several years.
However, thanks to data science, we’re able to constantly expand and strategically improve our coverage based on impact and usage for current users.
The general structure of our queue creation includes multiple tools. We have an internal tool that aggregates advisory information and helps automate parts of the rule writing process.
We use a combination of open and close source workflow engines to help execute cron jobs and alert us of new advisories. Integrations between our ticketing system and code repository enable automation to manage our tasks.
Now onto the fun stuff - getting a deep understanding of the vulnerability and generating a rule.
Getting your hands dirty
You might wonder, “How hard could it be to read a security advisory?” It depends!
Here’s an example that makes our researchers happy
Why? It’s beautiful. The advisory concisely describes the vulnerability, its potential risks, and precisely which functions may be a risk, if used by your code. Additionally, there’s a reference to the HackerOne submission, proof-of-concept code, and the pull requests where patches were applied. All the researcher has to do here is review these details and, if everything makes sense, write the Semgrep rule.
Here’s an example that makes our researchers sad
CVE-2024-25710
Why? The advisory does not say which exported functions of the affected package are vulnerable and lacks a link to the fix pull request. We know something changed between version 1.25 and 1.26 that patched the vulnerability, but over 600 files were changed when comparing versions! There’s no silver bullet solution here; you just have to get dirty and figure it out.
We are so passionate about well written advisories, that one of our Security Researchers gave a talk on this topic alone.
Let’s write a rule
There are generally three phases when writing a Semgrep rule:
Analysis: We are focused on understanding what the vulnerability is and identifying the vulnerable function, snippet, operation, etc.
Rule Construction: We start defining the rule syntax to match the analysis.
Rule Testing: We then test the rule to ensure there are no false positives (FP) and false negatives (FN) with our test code.
For the sake of brevity, we will keep each step at a high level.
Let’s build a rule using this example vulnerability from the package paddlepaddle:
https://github.com/advisories/GHSA-qqv2-35q8-p2g2
Analysis
Researchers generally start with examining the advisory description. Usually, the referenced patch links give us a great starting point!
Our goal is to identify the vulnerable code snippet. In this example, we already have the function name from the description. We love it when advisories are explicit like this. We can validate this using the patch notes.
Since the advisory says that the vulnerability resides in the paddle.utils.download._wget_download function, the next step is to determine the number of ways there are to trigger this function. In other words, are there any other callers or parents of this function?
Let’s start by tracing the function in the source code repository. In our example, _wget_download is called by _download when the method argument is 'wget' - method='wget'. _download is subsequently called by get_path_from_url, which also requires specifying method=wget.
get_path_from_url is further invoked by get_weights_path_from_url. However, the method argument cannot be overridden, making it impossible for this function to eventually call _wget_download. At this point, we have identified the callers of the function _wget_download.
Private versus public functions is an important point here. In our case, the affected function starts with an underscore, as does its parent _download. In Python, if a function name starts with an underscore, it generally signifies that the function is intended for internal use within a module or class, rather than for public use. This is akin to a private method in Java. However, unlike Java, there is nothing to prevent calling this function.
Rule construction
We analyzed the library and located the vulnerable functions.
At this point, we have multiple options to write the rule.
We can flag all three functions (although internal, these can still be called). Since we do not have direct access to user source code, flagging all 3 functions would be considered a broader approach.
We want to be more specific in our rule. From a GitHub code search, we find that get_path_from_url has a lot of usages. So now we know we want to flag the function get_path_from_url.
Python rules in Semgrep are convenient because we can use fully qualified function names in the form: module.submodule. … .function. This will match any usage of the function, no matter how it is imported (for example, import module.submodule.function and from module.submodule import function).
Rule testing
We use our own engine to test rules before they are shipped to our customers! We have a robust automation framework and testing matrix that allows us to be confident in the rules that we ship.
We add code to rule out FPs and test other ways the source code can be written to check for FNs.
Join and get in touch with us
We will be starting the search for our 2025 Summer Internship Program very soon. Look out for our Internship postings on our Careers Page!
You can provide direct feedback to us about our rules and more in the Semgrep platform. You can also reach out to us through this link or join our Community Slack Channel!
