Efficient Dependency Management: Leveraging Manifest Files, Lockfiles, and SemVer Specifications

In this blog post, Kyle Kelly, Semgrep Security Researcher, delves into how practitioners can efficiently manage dependency versions by utilizing manifest files, lockfiles, and SemVer specifications.

The year is 2023, and you’re (hopefully) using some sort of software supply chain security product.  Woohoo you have endless vulnerabilities to remediate, #JobSecurity! 


 If you want to be efficient, check out how Semgrep Supply Chain’s reachability analysis can identify the 2% of vulnerabilities that matter


Plenty of vulnerabilities likely impact your team’s direct or transitive dependencies. How should you prioritize the remediation of these vulnerabilities after they’ve been identified?  Sure, the likely and perhaps best answer is based on severity and likelihood of exploitation - but how about ease of remediation? In this article, we’ll discuss two factors to consider when your goal is to remediate as many vulnerabilities as quickly as possible. 

Breaking vs. Non-Breaking?

SemVer, short for Semantic Versioning, is a versioning scheme for software that aims to convey meaning about the underlying changes with each new release. Understanding SemVer is crucial for making informed decisions about when and how to upgrade your dependencies in dependency management and vulnerability remediation.

Semantic Versioning is typically expressed as three numbers separated by dots, like this: X.Y.Z. Each number has a specific meaning:

  • X (Major): Incremented when there are incompatible API changes. Upgrading when a major version changes might require you to change your code.

  • Y (Minor): Incremented when new features are added in a backward-compatible manner. These updates shouldn't require changes in your code.

  • Z (Patch): Incremented when backward-compatible bug fixes are introduced. These are typically safe to upgrade to.

Using SemVer to prioritize upgrades

Patch Upgrades (Z): Given their nature of being backward-compatible bug fixes, patch upgrades are typically your low-hanging fruit. They are the easiest to apply and are the least likely to introduce breaking changes to your application. Prioritize upgrading to the latest patch version, especially if the vulnerability is fixed in a patch release.

Minor Upgrades (Y): These releases add new features without breaking existing functionality. They might require some testing to ensure that new features don’t introduce unexpected behavior, but they are generally safe to upgrade to. If a vulnerability is addressed in a minor release, it's usually a good candidate for quick remediation.

Major Upgrades (X): These are the trickiest to handle. Major version bumps might come with breaking changes, requiring significant effort to upgrade. However, they might also come with important security improvements. Assess the impact of the vulnerabilities in the context of your application and plan a careful upgrade if necessary.

By leveraging SemVer, teams can make more informed decisions about which dependency upgrades to prioritize for vulnerability remediation. Patch and minor upgrades, being backward-compatible, are typically safer and quicker to implement, helping to speed up the remediation process. Major upgrades, while potentially more time-consuming, should not be neglected, especially when they address critical vulnerabilities.

It’s important to note that while SemVer is an ideal specification and is often used in practice, some dependencies or changes will not be compliant.  For example, the study “Breaking Bad? Semantic Versioning and Impact of Breaking Changes in Maven Central” concluded that ~20% of non-major releases in the Maven Central ecosystem were breaking changes. 

Manifest vs. lockfile versioning

Managing dependencies is a crucial aspect that demands careful consideration in the dynamic landscape of software development. Understanding and evaluating your project's manifest file versions can significantly mitigate risks associated with outdated dependencies.

Understanding manifest and lockfile versions

A manifest file, such as package.json in Node.js projects or Gemfile in Ruby applications, declares the dependencies your project needs, often specifying a range of acceptable versions. The paper, A Large Scale Analysis of Semantic Versioning in NPM, showed that for the NPM ecosystem, the “minor flexible” version specification, i.e.,  ("ˆ1.2.3"), was by far the most commonly used. Additionally, the study determined that 87.32% of all dependencies can receive updates automatically. 

On the other hand, a lockfile, like package-lock.json (Node.js) or Gemfile.lock (Ruby), pins down the exact versions of each dependency, including transitive dependencies, used in your project.

While the manifest file accommodates a spectrum of versions, thus ensuring flexibility and compatibility in various environments, the lockfile is designed to maintain installation consistency by fixing specific versions. These fixed versions in the lockfile can only benefit from updates if they are deliberately upgraded or if the lockfile is regenerated.

The hidden risks in dependency ranges

Specifying a range of acceptable versions in your manifest file can lead to two significant risks:

  • Using outdated versions: Developers might unknowingly continue using older versions of dependencies that fall within the specified range but are riddled with known vulnerabilities.

  • Missing out on security patches: Even if a security patch is available and falls within the specified range, the project might not be using it due to the lockfile pinning an older version.

For example, let’s say we have an npm project with a package.json file containing "decode-uri-component": "^0.2.0" which indicates that the project is compatible with any minor or patch version of the package. A vulnerability is released for version 0.2.0 and a fix is applied in version 0.2.1.  

dependency fixYou might think, “Great! This will get fixed on its own”, and you’d be wrong! Without re-generating your package-lock.json file, which likely pinned the version at 0.2.0, those leveraging the lockfile to reproduce builds will continue to use the vulnerable version 0.2.0. That said, the next time you generate that lockfile, it will get fixed on its own, so it’s important to keep that lockfile up-to-date!

Conclusion

By actively evaluating and managing the versions in your manifest file and lockfile, you can harness the benefits of version ranges while ensuring your project uses the latest, most secure dependency versions. This balanced approach enhances your application's security and stability without forgoing the flexibility provided by version ranges.

Security is a continuous effort; diligently managing your dependencies is key to protecting your software supply chain. Embrace best practices like regular updates and project reproducibility to make the most of version ranges. This proactive stance, coupled with tools that analyze these files, such as Semgrep Supply Chain, optimizes the remediation process and directs your engineering efforts toward robust, efficient cybersecurity measures. 

Join me as I speak about "Finding issues that matter in open source dependencies and fixing them without developer friction" on December 12th.

About

Semgrep Logo

Semgrep lets security teams partner with developers and shift left organically, without introducing friction. Semgrep gives security teams confidence that they are only surfacing true, actionable issues to developers, and makes it easy for developers to fix these issues in their existing environments.

Find and fix the issues that matter before build time

Semgrep helps organizations shift left without the developer productivity tax.

Get started in minutesBook a demo