 |
| PHOTO | NMG |
"In his book ‘‘Numbersense’’, Kaiser Fung, a professional statistician and adjunct statistics professor at New York University, correctly emphasises on data analysis over big data mining.
The careful observation of data and good questions generated from careful observations, not the size of it - the ability to process, store and make sense out of the data. He gives the example where some years ago the Gates Foundation made a mistake of assuming that smaller schools are better for student achievement which was later proven to be untrue.
The unfortunate bit about the current hype on big data mining is that there is not enough attention paid into reporting the accuracy of data mining and processing procedures. Most of data being analyzed today is unstructured, poorly formatted, poorly documented and not designed with the data scientist in mind making it more difficult to process.
Second, the other problem Kaiser argues about big data is that it actually moves us backwards since more data results to more time spent in analyzing, arguing, validating and replicating results.
This same argument has also been posited by Nat Silver – America’s top statistics-driven writer and analyst - in his book ‘‘Signal and Noise’’, he talks about the need to reduce the big data collected to its essence which he referred to as pulling the signal out from all the noise (big data)."
Read more....
The unfortunate bit about the current hype on big data mining is that there is not enough attention paid into reporting the accuracy of data mining and processing procedures. Most of data being analyzed today is unstructured, poorly formatted, poorly documented and not designed with the data scientist in mind making it more difficult to process.
Second, the other problem Kaiser argues about big data is that it actually moves us backwards since more data results to more time spent in analyzing, arguing, validating and replicating results.
This same argument has also been posited by Nat Silver – America’s top statistics-driven writer and analyst - in his book ‘‘Signal and Noise’’, he talks about the need to reduce the big data collected to its essence which he referred to as pulling the signal out from all the noise (big data)."
Read more....
