Or more so "will it be"? Big Data makes statistics and relevant knowledge all the more important but seems to underplay Sampling Theory.
I've seen this hype around 'Big Data' and can't help wonder that "why" would I want to analyze everything? Wasn't there a reason for "Sampling Theory" to be designed/implemented/invented/discovered? I don't get the point of analyzing the entire 'population' of the dataset. Just because you can do it doesn't mean you should (Stupidity is a privilege but you shouldn't abuse it :)
So my question is this: Is it statistically relevant to analyze the entire data set? The best you could do would be to minimize error if you did sampling. But is the cost of minimizing that error really worth it? Is the "value of information" really worth the effort, time cost etc. that goes in analyzing big data over massively parallel computers?
Even if one analyzes the entire population, the outcome would still be at best a guess with a higher probability of being right. Probably a bit higher than sampling (or would it be a lot more?) Would the insight gained from analyzing the population vs analyzing the sample differ widely?
Or should we accept it as "times have changed"? Sampling as an activity could become less important given enough computational power :)
Note: I'm not trying to start a debate but looking for an answer to understand the why big data does what it does (i.e. analyze everything) and disregard the theory of sampling (or it doesn't?)