Top websites can't handle advanced bots

Pretty much every top website in retail, financial services, consumer services, OTA members (Online Trust Alliance), news and media, and top U.S. government agencies, is vulnerable to advanced bots, a new research says.

Bot detection and mitigation company Distil Networks, analysed 1,000 top websites in these verticals, and how they behave against crude, simple, evasive and advanced bots. All of the verticals performed quite well against crude bots, (75 per cent in consumer services, 70 per cent in government, 65 per cent in financial services, 64 per cent in news and media, 78 per cent in retail and 67 per cent in OTA members), but when it comes to advanced bots, 1 per cent is the best result found.

Bots are used, as Distil says, for competitive data mining, online fraud, account hijacking, data theft, vulnerability scans, spam, man-in-the-middle attacks, etc.

“Bots, especially Advanced Persistent Bots (APBs) are evolving in sophistication because of their polymorphic nature, and quick deployment to access sensitive information and reap monetary benefits. Our 2016 Bad Bot Landscape Report found over 88 per cent of all bad bot traffic last year was made by APBs, bots that mimic human behavior,” said Rami Essaid, CEO and co-founder of Distil Networks.

“OTA’s Trust Audit continues to set the bar for best practices, including evaluation of bot risk. We support OTA’s efforts to promote best practices in the industry and are troubled to find that most companies are failing to keep their defenses up to the sophistication level of today’s advanced and evasive bots. This is concerning, as bots can easily paralyze website infrastructure, pirate entire online directories, and destroy a company’s competitive advantage.”

Detection rates by vertical

Vertical Crude Simple Evasive Advanced
Consumer Services 75 per cent 18 per cent 4 per cent 1 per cent
Government 70 per cent 7 per cent 0 per cent 0 per cent
Financial Services 65 per cent 12 per cent 0 per cent 0 per cent
News and Media 64 per cent 7 per cent .09 per cent .09 per cent
Retailers 78 per cent 11 per cent 1.6 per cent .08 per cent
Members 67 per cent 13 per cent 1 per cent 1 per cent