I have to say that the presenters here at FIRST seem to be on a totally different brain level to the rest of the planet. I think the gist from listening to Stephano Zanero’s presentation on Flaws and Frauds in the evaluation of IDS/IPS technologies and reading his white paper is that we are probably just at the beginning of the evolution of current IDS/IPS (Intrusion Detection/Intrusion Prevention) systems.
Recently I have had a major argument with an on-line gaming provider that their system is producing too many false positives and that it is going to cause them a customer service problem in the long run. Of course, not realising my background in security – they blanked me. However after listening to Zanero’s talk, it became obvious why they have so many false positives.
Currently there are two strategies used by systems to detect intrusion Misuse based and anomaly based. Recently there has been a move towards anomaly based systems.
The table below from Stephano’s talk shows the major problem with anomaly based systems.
Now from the table you can see the problem of false positives that many users will have experienced due to the gaming site’s anomaly based system.
I have to say that there is absolutely no way that I can repeat all his calculations, graphs and formulas of the subject here but if you want to find out more check out his videocast here or get hold of him at his company here.