The previous article in this series is Recognizing and Rewarding Your Security Champions.
If you've followed my conference talks, you likely saw my Security Metrics That Matter presentation, and understand that I absolutely love data. Here's a general list of security metrics that matter, if you don't want to read the whole article or watch the entire talk.
You may wonder, why are metrics important? The answer is twofold.
We can use data and metrics to report up to our bosses and show them we are succeeding. It's evidence that what we are doing is working, and how well it is working. You can then use that data again to ask for more resources (staff, tools, budget), a raise, or other changes.
The second reason is so that we, ourselves, can improve. We want to improve our program, ourselves, and our results. When we measure our activities and their impacts, we can see which activities or methods produce better results. We can then use that information to change our approach, for the better.
It is important, however, that we do not become fooled by vanity metrics. Vanity metrics are numbers that make us look good, but don't necessarily mean anything. My talk on this subject has several stories, but for now let's just tell one.
I used to work somewhere, and we all wrote blog posts. We were measured on how many “clicks” we got. A colleague of mine got 10X the number of clicks that I did, and I asked him how he did it. He explained he got the most clicks on Reddit. I was unfamiliar with the platform but thought I would give it a try. First though, I asked for extra data: I wanted to know _how lon_g people were staying on our articles. It turned out that people were staying on my articles approximately 1.5 minutes (which means they were reading the whole thing), and on his they were staying an average of 1.5 seconds (which means almost no one was reading the article, they were just clicking the link. This is commonly known as a “bounce”.) The purpose of our jobs was to write articles to help customers know how to use our products, and this means a bounce wasn't valuable. Armed with this new information, we started comparing different platforms, and it turned out almost all traffic from Reddit were ‘bounces'. I also noticed that my Twitter followers were significantly more likely to read the article when compared to LinkedIn, and LinkedIn got better results than Reddit. My colleague started focussing on sharing links on Twitter (he had more followers than I did), and I started trying to get more followers on the same platform. It turns out that measuring clicks was a vanity metric. The rest, as they say, is history.
Now for your security champion program metrics! Measure the following things so you can see what's working and what is not. Don't forget to report upwards about the ROI (return on investment) your champions program has produced!
How many new security champions you have attracted
Measuring program engagement: how many people attended an event, how many people reported issues to you, how many people asked questions,
Use the bug tracker for metrics on how many security bugs are being reported and fixed, especially if you have targeted a specific bug class. Also, count how many new instances of that type of bug appear, hopefully this number will be very low.
Instances where champions have told you about a security issue you would not have known about otherwise
If the champions report better work satisfaction and/or fewer missed days of work
Gather stories of your champs saving the day, providing help to their teammates, or anything else that makes for a good story-telling session for upper management.
Up next, I will share a few more tips that don't fit into any of the previous categories and conclude this series. Please feel free to email me with any questions!
