Industrial Revolution of Data

Data-data-everywhere, was a special report on managing information, The Economist , 27-Feb-10. With the-data-deluge on the cover.

In The Petabyte Age , I have provocatively written from mathematics as a data compression field and mentioned new mapping techniques as new ways of visualizing data in I Saw It With My Eyes .

Monstrous amounts of data are produced by machines and the best way to deal with all that information is by machines.

Some say they need watching, I say most probably WE need watching when recognizing pattern and features. At one hand, we tend to oversimplify ( Beware Wind ) or if you torture data long enough often they show you everything you want to recognize.

Hal Varian, Google's chief economist, predicts that the job of statistician will become the "sexiest" around. Machine learning expert, I add.

If we want users to find the needle in the haystack, you need to create information on information, if you want to recognize structure in information you need to create models from data. Lucky if they are computational. Even more lucky if they are computational and interpretable, as most of our mlf models .