Six Mistakes of Log Management

Top Log Mistakes are, in no particular order,

#1 not logging at all.

#2 not looking at the logs

#3 storing logs for too short a time

#4 prioritizing the log records before collection

#5 ignoring the logs from applications

#6 only looking at what they know is bad

Since I wrote my log mistakes paper a few years ago, the domain of log analysis changed a lot. Many factors affected it; among those are new regulatory compliance requirements, wider adoption of “best practice” and governance frameworks such as ISO, COBIT and ITIL as well as new technologies with their log files. New standards, such as NIST 800-92 Guide, have been created. Thus, I am updating this article with newly committed mistakes as well as new prospective on the old ones.

Thus, this article, just like its predecessor, again covers the typical mistakes organizations make while approaching management of computer logs and other records produced by IT infrastructure components.

As digital technology continues to spread (“A web-enabled fridge anybody? It is just eight (!) grand today, you know”) and computers start playing even more important role in our lives (I do have a penchant for the obvious, don’t I?), the records that they produce, a.k.a. logs, start to play bigger and bigger role. From firewalls and intrusion prevention systems to databases and enterprise applications to wireless access points and VOIP gateways, logs are being spewed forth at an every increasing pace.

Both security and other IT components not only increase in numbers, but often come with more logging enabled out of the box. Example of that trend include Linux systems as well as web servers that now ship with increased level of logging. All those systems, both legacy and novel, are known to generate copious amounts of logs, audit trails, records and alerts, that beg for constant attention. Thus, many companies and government agencies are trying to set up repeatable log collection, centralization and analysis processes and tools.