Fully loaded: testing vulnerable PyYAML versions

❗ tl;dr: By testing the PyYAML API across various released versions, we determined that the yaml.Loader class is unsafe in all released versions. yaml.load(...), yaml.full_load(...), and their _all variants appear to be safe from known exploit payloads after version 5.4. yaml.safe_load(...) is safe in all PyYAML versions.

A few weeks ago, GitHub user shivankar-madaan made a pull request to Semgrep's Python YAML deserialization rule, adding a pattern to detect yaml.full_load(...) which was vulnerable to attack earlier in PyYAML's lifetime. (Thank you for the contribution!) Through pure coincidence, around the same time, one of our developers notified me that the original YAML rule was reporting false positives on yaml.load(..., Loader=yaml.SafeLoader). I updated the rule to filter out the SafeLoader case, but between this and my unfamiliarity with full_load(...), I thought it prudent to go on a quest to understand how the PyYAML API had changed over time. 🗡️🛡️

YAML de-cereal-ization vulnerabilities

Forgive the heading–I'm practicing my dad jokes. 😅

For years, Python static analysis tools have warned on the usage of PyYAML's yaml.load(...) function, and rightly so. Unsafe use yaml.load exposed applications to a deserialization vulnerability, which would let an attacker construct YAML which causes yaml.load to create arbitrary Python objects. The end result is code execution, which means the attacker controls the machine. 😱

An animated gif of Homer Simpson pouring milk onto cereal and it erupting in flames

As of this writing, Bandit, CodeQL, and even our own Semgrep rule report yaml.load(...) as dangerous (when called without SafeLoader). However, it turns out that PyYAML began efforts to fix the vulnerable load(...) call in 2019 with version 5.1. This version printed a warning when yaml.load(...) was used, guiding users to the Loader= keyword argument. PyYAML 5.1 introduced FullLoader and UnsafeLoader classes, along with function call variants full_load and unsafe_load. FullLoader was used by default when calling yaml.load(...).

1<stdin>:1: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

Unfortunately, exploits CVE-2020-1747 and CVE-2020-14343 were discovered for FullLoader, resulting in patches which eventually went out in PyYAML version 5.4. At this point, I was very confused about the safety of the default API. Having read patch notes, PyYAML's load(...) deprecation notice, and pull request discussions on CVE patches, I really had no idea which PyYAML APIs were safe.

Sometimes, software documentation is unclear or out-of-date and as security professionals, we need to test for ourselves what actually happens under various conditions. A mentor of mine called this "putting the test leads to the system." Below, I'll describe the method I used to figure out which PyYAML APIs are vulnerable in which versions of the package. Similar methods can be used to check exploitation conditions in other software.

Testing APIs x versions

The Smash Bros. crossover logo with the Python icon

To test which combinations of APIs and PyYAML versions were vulnerable, I used Tox, a testing harness for Python that manages virtual environments for you while running a test suite. Typically used for compatibility testing, you can direct Tox to use specific versions of Python and Python packages for a given test suite. In this case, Tox enabled me to test a series of payloads against a list of PyYAML versions–perfect! 👌

Speaking of payloads, I collected four payload test cases by hunting for proof-of-concept exploits for PyYAML: a base case, and one for each of CVE-2017-18342, CVE-2020-1747, and CVE-2020-14343.

1!!python/object/new:tuple
2- !!python/object/new:map
3  - !!python/name:eval
4  - [ "__import__('os').system('echo {check}')" ]

The tests are configured in such a way that if any payload was activated, that API is considered vulnerable. The code for running these tests is here. This is the final results matrix:

Turns out that safe_load, safe_load_all, and the SafeLoader class have always been safe! 🥳 load, full_load, and their _all variants are safe against known payloads after version 5.4. (There may still be unknown payloads.) The Loader class is completely unsafe.

Given this outcome, we are updating the Python YAML deserialization rule to alert on APIs known to be vulnerable after version 5.4: unsafe_load, Loader, UnsafeLoader, and their C variants. Generally, we want Semgrep rules to produce actionable results, so we are opting to not have the rule alert on known vulnerable APIs prior to 5.4. A cursory glance on GitHub shows that only a handful of projects use PyYAML versions older than 5.4, which was released over a year-and-a-half ago, so we think it is appropriate to alert on vulnerable APIs for version 5.4 and later. If you are running older versions of PyYAML, consider updating to be safe, or ensure you are using safe_load everywhere. safe_load is the best option no matter the PyYAML version. If you can't update and must check your code for older vulnerable APIs, you can use this Semgrep snippet ($> semgrep -f https://semgrep.dev/s/Pwo3) or write your own rule which is tailored for your specific situation.

Before this update, the Semgrep rule looked like this:

Now, the rule looks like this. Once again, thanks to shivankar-madaan!

Conclusion

If you’re using a package that has a checkered history and you're unsure of the current safety of the API, you can design a similar test to check the actual behavior: find a testing harness that lets you test package versions, find payloads for the API, and write tests evaluating the behavior of the API. Stay safe_load out there! 😉 (Dad jokes, remember?)

Join the r2c Community Slack to say “hi” or ask questions — there’s a friendly and active community ready to help!