When I started working in DevSecOps, I hadn’t even done DevOps before (gasp!). I had been following the Waterfall software development methodology in the Canadian Public Service (with a tiny bit of agile thrown in) for over a decade before I joined Microsoft, and my entire world changed. I had been working on the OWASP DevSlop open-source project with my friend Nikky Becher at the time, muddling our way through, trying to figure out how to pentest in a DevOps environment, and suddenly, I was supposed to be a DevOps expert. I thought, “How can I catch up? And how can I do it FAST?”
I decided that I would create an app, using the Azure DevOps CI/CD, and I would ‘share’ my pipeline that I used to do it. Quickly, I figured out there wasn’t a way in Azure DevOps (at the time) to open-source or ‘share’ a pipeline, so I decided I would film myself live as I built. I made a basic app and a basic dev -> QA -> UAT -> Prod deployment, and I was off to the races. I started coding live on Twitch and adding every security tool to my pipeline that I could get my hands on.
Not every episode of the OWASP DevSlop show went well, and neither did every tool. The DAST I chose threw a false positive while I was presenting at Microsoft in Ottawa, Canada (live on stage, naturally). There’s one episode with Nancy Gariché (one of the project leaders), and I just FAILED the whole time, and after 3 hours, we never got the tool working at all. There was a 5-hour episode where I updated the .Net CORE framework and all my dependencies that had vulnerabilities in them, and that was exhausting. But I LEARNED. And I learned fast.
In 2018, I joined a company called IANS Research, doing ‘Ask an Expert’ calls helping clients with Azure and AppSec problems. As I learned more about DevOps and DevSecOps, I started helping clients with those topics as well. I would often have to do research to figure out the solution before a call, but I didn’t mind. All of their questions helped give direction to my learning. Over the years, I have helped hundreds of AppSec teams crush their problems and learned a LOT along the way.
In 2020, I started coaching two different companies with their AppSec programs, for a few hours a week, building out their DevSecOps programs. When contrasted with the short types of assistance I was giving to IANS clients, these long-term relationships helped me see programs over extended periods of time. And I got my hands dirty on a weekly basis, trying many different types of tools (for better or worse).
I also attended numerous conference talks and read lots of articles to see what others were doing, and learned best practices along the way.
All of this learning was A LOT of work. One day, an IANS client told me they were going to start in DevSecOps and asked if I could make them a “do not do” list. A list of pitfalls that they should work to avoid. Dear reader, I dove deep down this rabbit hole. It had never occurred to me to start with “what not to do” rather than “this is the way”. And how many headaches could be avoided…
With this in mind, I wrote a conference talk (video below) on this topic. This blog series will explore each of the 15 ‘worst practices’ that I cover in the talk.
The 15 items I will present in this series are as follows:
Breaking Builds on False Positives
Untested Tools
Artificial Gates
Missing Test Results
Runaway Tests
Impossible SLAs
Untrained Staff
Forgotten Bugs
No Positive Reinforcement
Only Worrying About Your Part
Multiple Bug Trackers
Insecure SDLC
Overly Permissive CI/CD
Automation Only in the CI/CD
Hiding Mistakes and Errors
I hope that by sharing mistakes I have seen and made, all of us can avoid these issues going forward.
The next article will be linked here when it is ready.
About
Semgrep lets security teams partner with developers and shift left organically, without introducing friction. Semgrep gives security teams confidence that they are only surfacing true, actionable issues to developers, and makes it easy for developers to fix these issues in their existing environments.
