As the old saying goes, you can’t improve what you can’t measure. And when it comes to continually improving your organisation, security is no exception. Security metrics can help a company identify the strengths and weaknesses in its cybersecurity program, measuring its results and demonstrating how well the security program is doing.
But, as with most things, security metrics aren’t always an exact science. Security teams may face challenges such as being able to locate and find the right metrics, or have so much data to analyse, they don’t know where to begin. So, when it comes to being able to reap the business benefits of security metrics, what are the best metrics to use, and which should be avoided? Six security experts spoke to ITProPortal to share their experience of security metrics, along with their suggestions as to how to best get the most out of them.
The security metrics that just don’t work
According to Michael Scheffler, AVP EMEA at Bitglass, “When it comes to cloud security, one of the least useful metrics is to measure the number of cloud services that employees are using. This is because as much as IT security teams like to think they are blocking all cloud services not approved by them, the stark reality is that there are likely hundreds of cloud services being used by employees that the IT security team has no knowledge of. Referred to as shadow IT, this poses security and compliance risks since sensitive corporate data is stored in shadow IT cloud apps – yet the company has no control over that data.”
Scheffler goes on to explain that the second security metric that doesn’t work, in his opinion, is assuming “that traditional tools for safeguarding data on premises are equally capable of protecting data in the cloud. With more and more organisations storing sensitive information in the cloud – information like customer data (45 per cent), employee data (42 per cent) and intellectual property (24 per cent) – adopting proper cloud security measures is critical (source: Guardians of the Cloud, Bitglass’ 2019 Cloud Security Report).
“Over the last five years, cloud adoption has grown at an astonishing rate. Consequently, employees have been able to work more efficiently and flexibly, allowing organisations to enhance their operations in various ways. With that being said, the need for data protection is more vital than ever, and security strategies that organisations are implementing must be shaped around a cloud-first environment. Adoption rates of basic cloud security tools and practices are still far too low – and many organisations need to rethink their approach to protecting data in the cloud.”
Tim Bandos, VP of Cybersecurity at Digital Guardian, also has a favourite ineffective security metric - the Number of Threats Blocked by Security Controls. “Of course, it sounds amazing to report to the board that your controls blocked millions upon millions of threats at your perimeter firewall,” Bandos explains, “but anecdotally this is the absolute worst. It sends the wrong message in relation to the effectiveness of your cybersecurity program and doesn’t truly gauge how resilient your organisation is to an actual threat such as ransomware or a state-sponsored attack. A better metric in my opinion here is the mean cycle time from initial infection to detection or the duration to neutralise a successful threat, because at some point, they will get in!”
Another of Bandos’ favourite ineffective security metrics he’s seen, is merely showing the Number of Critical Vulnerabilities Patched. “Yes, it sounds great to say to your CIO that you’ve patched 100 critical vulnerabilities,” he says. “However, let’s say your environment still has 1,000 outstanding vulnerabilities that still need to be patched – and by the way, those vulnerabilities exist on some of your more critical infrastructure that house sensitive data. Don’t get me wrong; I still think it’s crucial to demonstrate your progress in patching, but we need to avoid only showing the positive and provide a more comprehensive view of risk and measurement of your program’s effectiveness."
Keeping an up-to-date approach
As Josh Flinn, Director of Product Strategy & Innovation at Cybera, suggests, “All information available in security metrics is useful.” But, as he goes on to elaborate, “Some metrics, like the number of threats blocked, seem less useful, however a spike in this metric can indicate an active attack against your network or a compromised endpoint.”
It’s important that organisations keep up-to-date with the latest cybersecurity technology, as Flinn explains: “Attacks are becoming more sophisticated so the more information you are armed with the better. The big problem with security metrics is the vast amount of information that is available now. Sifting through all the data and trying to correlate it is more than a person or team can reasonably do. The key for security going forward is AI and ML, so the security professionals can focus on the threats instead of the data."
Additionally, according to Matthew Buskell, Area Vice President at Skillsoft, treating technical certifications as the priority recruitment metric is also an outdated approach. He says, "It’s time for firms to demonstrate a greater willingness to diversify their workforce and assess what traits are required — lateral thinking, problem solving skills, an understanding of risk management — rather than narrowly focusing on technical certifications alone. This requires a depth and breadth of vision that goes beyond traditional thinking.”
Buskell explains that this approach can have a positive impact on reducing the gender gap in the security industry. He says, “When it comes to mining the potential of the female empowered workforce, numerous national programmes are encouraging women to acquire cyber-skills. The UK’s National Cyber Security Centre has created courses to encourage girls to consider studying the subject at A-level and university. Similarly, since 2013 the Code First: Girls organisation has been supporting young adult and working age women in the UK to further develop professional skills, such as coding and programming, and working with companies to help them capture top female tech talent."
Using the metrics in a beneficial way
Security metrics are only as good as the knowledge the IT team has of how to utilise them. As Richard Cassidy, Senior Director Security Strategy at Exabeam, points out, “MTTD (mean-time-to-detect) and MTTR (mean-time-to-response) far too often focus organisations on ‘alerts’ and how quickly security teams can triage, close or escalate them. In essence, this is a kind of ‘alert-whack-a-mole’, except it’s an infinite game that results in ‘alert-fatigue’, which (as the industry breach metrics prove time and time again), simply leads to even poorer security outcomes. We should turn our attention to metrics that tie security to business context; there’s a new concept to consider – mean-time-to-answer (MTTA).”
Cassidy continues, “Technology has now caught up to enable a much more context enriched story of the chain of events, as they relate to a user (be it an exec or privileged user) and a critical asset (be it a database, server or key host). We’ve got to start focusing security and GRC teams on how they can provide better ‘answers’ on the risk or threat context of an alert, so that we drive a far more relevant and business critical outcomes."
Additionally, Steve Nice, Chief Security Technologist at Node4, also believes that security metrics are beneficial for companies, explaining that from his perspective, “trying to turn qualitative evidence – how we ‘feel’ about a situation for example – into a number between X and Y is fine, but knowing what the ‘value’ actually is, is entirely arbitrary. The point of exercises and metrics like this are to simplify comparison – nothing more. But as soon as you try and perform anything more the most facile comparative analysis on them, their use and meaning become a hindrance rather than a help. So, terms such as ‘average’ in this context are actually deceptive.
“When it comes to ‘value’ in management terms, I think as long as we’re consistent with regards to how we arrive at it, and avoid the temptation to ‘over-reach’ on its statistical significance, it could be a useful metric. The crucial plot in all of this, is if we make a statement like ‘the threat has decreased from ‘X’ to ‘Y’’, we can explain unambiguously what it was in the various composite terms/situation that initiated this change.
“Essentially, I don’t think it’s really that important whether we have a security status set in ‘blue’ or ‘pink’ … it’s more to do with whether we’re ‘bluer’ or ‘pinker’ than we were last time we looked; why this is so, and what we need to do to make things better.”
There’s no doubt that security metrics are becoming essential to helping organisations continually improve their cybersecurity strategies. But in order to ensure that IT teams aren’t wasting valuable time trying to read metrics that aren’t that useful, it’s important to identify exactly which security metrics work, and which should be let go.