söndag 11 oktober 2015

What is Big Data and How to Unlock its Full Potential

Figure 10. A system of statistical registers - by object
type and subject field, from Wallgren and Wallgren (2014).
From: AAPOR (press release) and AAPOR Report on Big Data (full report)
In recent years we have seen an increase in the amount of statistics in society describing different phenomena based on so called Big Data. The term Big Data is used for a variety of data as explained in the report, many of them characterized not just by their large volume, but also by their variety and velocity, the organic way in which they are created, and the new types of processes needed to analyze them and make inference from them. The change in the nature of the new types of data, their availability, the way in which they are collected, and disseminated are fundamental. The change constitutes a paradigm shift for survey research.

There is a great potential in Big Data but there are some fundamental challenges that have to be resolved before its full potential can be realized. In this report we give examples of different types of Big Data and their potential for survey research. We also describe the Big Data process and discuss its main challenges. The task force recommendations are summarized below.

Read more....