After 20 years of a security model built entirely around prevention, there is now a shift in focus towards detection technologies. Gartner, for example, predicts a 60/40 expenditure split in favour of detection by 2018.
This is due to the acknowledgement that, despite the latest technology and extensive investment in preventative security, network attackers are getting through perimeters: Major data breaches nearly a daily news item. Security prevention is still a necessity, but everyone knows that complete prevention is a fallacy.
Despite important advances in firewall technology, sandboxing solutions, and security intelligence, it is no longer possible to prevent a motivated attacker from getting into a network. Preventative measures may protect against 95 per cent or more attempted attacks, but it takes only one chance to be successful, perhaps through extremely clever spear phishing or social engineering, to compromise a valid user account. Once an attacker is inside, the challenge is detecting them quickly before theft or damage ensue.
There are plenty of security tools. So the question is what is it that a tool detects? Does it alert one to the ubiquitous presence of malware or is it actually calling out an attack that is currently in process? If a tool is intended to find an attacker, how well does it pinpoint the attack?
Security: The data overload
There is such a confusing array of terms and vendor claims about detection. One thing for certain is that the problem is not lack of data produced by security tools. In fact, most security organisations suffer from a data overload, particularly in terms of the sheer number of daily alerts generated by security systems. A Ponemon Institute survey of enterprises showed that the average enterprise receives 16,937 weekly alerts. Such a high volume makes it unlikely that a security operator would be able to find real indications of attack activity buried under so many alerts that are dominated by false positives and minimally valuable warnings about the presence of malware. It is a classic needle in the haystack problem.
Accuracy and efficiency
A swift remediation of marketing claims would be to get each vendor to report on two metrics: accuracy and efficiency. Efficiency is simply the number of alerts a security system produces in real customer deployments. Too many alerts overwhelm a security team and generally mean that each one is of low quality.
To make objective comparisons, the results from each system should be normalised to 1,000 endpoints per day. For instance, a system that produces 10 alerts in a network with 5,000 endpoints is far better than one with four alerts in a network with 500 endpoints. The first has two alerts per thousand endpoints per day, and the latter has eight alerts per thousand endpoints per day. Having this kind of data would help organisations make better decisions about the purchase and use of security tools. A system producing 400 alerts per 1,000 endpoints per day is not just inherently at a disadvantage over a system producing ten — it is likely simply not usable.
The other component to this assessment is the accuracy or usefulness of these alerts. According to the Ponemon study, only four percent of alerts are actually investigated when organisations are flooded with thousands of alerts. Alerts must point to specific threats with a high degree of accuracy so that they practically jump out in the face of the security operator. This means that there can only be a small, workable number of alerts and that they all must be valuable.
Usefulness can be a matter of definition, but it has to involve a specific action on the part of the security operator rather than a passive result. For an alert to be valuable it has to be investigated, remediated or resolved in some hands-on way. Simply auto-achieving or whitelisting are too passive a result. Usefulness as a metric can be expressed as ratio of useful alerts to total alerts expressed as a percentage. Here there is no need to normalise the results to the size of the environment.
It’s time for vendors to report accuracy and efficiency as averaged across their customer base. The metrics will be enormously helpful in knowing which system one should buy. It may also drive a new focus on making detection more accurate and efficient. Such a focus could drive product development and result in tools that are more effective in dealing with the most pressing security issues.
Fast, accurate detection is key
Accuracy and efficiency also work together to make a security group more productive. Studies show that many security or general IT organisations waste a considerable amount of time due to inefficient or inaccurate security alerts. As much as two-thirds of a day can be spent in a wild goose chase that does not bring a security operator any closer to finding an attack or dangerous threat. This is especially damaging when most organisations are either short-staffed, overworked or both. Too many alerts and alerts of poor quality also contribute to fatigue and low morale, which, in turn could affect turnover and promote sloppy, half-hearted work.
Finding network attacks requires a new standard for fast, accurate detection. We would welcome a new standard to ensure that security operators have the necessary tools that increase their effectiveness rather than reduce it.
Jason Matlof, Executive Vice President, LightCyber