Skip to main content

How big data is making security exciting again

As security attacks become both more persistent and more complex, the rules-based approach to network and data protection is becoming increasingly ineffective.

The traditional rules-based approach uses security information and event management (SIEM) systems designed to track and mitigate 'known threats'. However, network security is now entering a new age, where unknown threats pose the greatest danger.

Facing the challenge of emerging malware

So what happens when a new and previously unidentified piece of malware comes calling? Once it's inside the network, an attacker can ensure that evidence of the malware's presence is hidden in the massive amounts of 'normal' data that enterprise systems generate. Trying to locate it can thus be extremely difficult.

Furthermore, given that SIEMs tend to overlook a lot of data, and data quite often simply gets lost, organisations' ability to spot new threats can become extremely compromised - especially when data being dismissed could potentially provide valuable clues to spotting advanced emerging threats.

That's why the new frontier of enterprise security is statistical analysis and pattern recognition in big data – specifically, machine data.

In this new world of security, CSOs and IT teams have to unlearn their over-reliance on traditional data protection technologies such as antivirus software, firewalls, and security information and event management systems. The non-stop barrage of attacks that the enterprise faces has turned security into a reactive, administrative role.

Security should be an exciting industry to work in, but too often those in charge – both seasoned professionals and new entrants – are just responding to systems alerts rather than applying their knowledge and thinking more laterally about threats.

What can be done?

To address this, security professionals need to have much greater oversight of everything that happens inside the enterprise. How? By being able to quickly analyse and sift through the machine data generated by interactions with IT systems in order to identify unusual patterns and abnormal behaviours which could indicate that an attack is taking place.

While big data analysis technologies can help to identify possible anomalies, they still require human insight and intelligence to interpret what those anomalies might mean.

There are many examples of anomalies that could be crucial to spotting a new form of malware in your system:

  • The presence of URL strings that are four or five times longer than normal, indicating the possible presence of command-and-control malware attempting to launch a web protocol attack.
  • A network access password being entered 10 times faster than it's possible for a human to type.
  • An excessive amount of outbound DNS traffic or DNS requests, indicating that an employee's machine has become part of a botnet.

Correctly analysing the signs could be the difference between spotting a new form of malware and allowing it to run rampant and invisible throughout your company's network.

What machine data can tell you

Interrogating machine data is also an excellent way of spotting when a security threat is being created internally – not by a clever piece of malware, but by a malicious insider who may feel entitled to intellectual property and wants to take it with them to their new job. Questions you could ask of your data are:

  • Why is a user repeatedly trying to access a file they don't have permission to view?
  • Why is there a significant change in the mix of categories of websites they access?
  • Why has their ID card been used to enter the office when they're meant to be on holiday in the Bahamas?

Achieving this level of operational intelligence not only opens up new possibilities for how companies defend themselves against the myriad security threats they face – it also re-engages the interest and creativity of the IT teams entrusted with the task of risk mitigation.

While there may be no 'silver bullet' for advanced threat detection, big data represents a compelling way to change the tide of online warfare back in favour of the good guys.

Matt Davies is the Product Marketing Director for Europe, the Middle East and Africa at search analytics firm Splunk.

Image credit: Flickr (Victor1558)