Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of A Pixel


Have you ever heard the expression “climbing trees to get to the moon”? This means that while one can make progress inching along (up a tree in this case) … if one stands back to look at the big picture … no matter how successful one is at climbing trees … it is quite obvious they are never going to get to the moon.

I have news … for those waiting for big breakthroughs in algorithms to better make sense of transactional data (e.g., to lower false positives/negatives), they are going to be waiting a long time (actually, it's worse, forever).

Using algorithms to analyze transactions is like analyzing a single pixel. No matter how much computing power, time and sophistication of algorithms … determining if the “red pixel” is a fire versus a fire engine simply isn’t a precision activity.

Next generation, real-world aware systems are going to FIRST apply inbound transactions (i.e., pixels) to earlier observations in order to construct context (i.e., pictures). Then, and only then, will algorithms be able to more accurately determine the relevance of new transactions. There are no short cuts.

Assembling pixels into pictures is the story of Context Accumulation.

But how do pixels get assembled into pictures? There is only one way to accomplish this and that starts with “feature extraction” and I wrote a bit about this in my post called “Context: A Must-Have and Thoughts on Getting Some …” Just to be clear, algorithms are, of course, required on pixels to perform “feature extraction” and there is room for extraordinary improvement in this area (e.g., see point #5 in the above blog post).

Persistent Context and Perpetual Analytics are the most significant hurdles necessary to deliver the next generation of intelligent systems.

AND FOR THE RECORD: Don’t expect Google’s search analytics to get much better this technology is near the end of its road as well. The reason being is; they are applying analytics to documents … which are really just context-less pixels. Quantum leaps forward in search are going to come from context accumulation.