Wherefore Big Data? 
*

*It's been many years since I've done any statistics (or any serious math), but I do remember that the sampling error decreases more slowly for larger sample sizes (like n^-1/2, at least for some statistics).

*I also remember (from numerical analysis) that for processes modeled as linear ODEs, errors in constant coefficients or initial conditions increase exponentially with time (and at least as bad for non-linear processes), i.e. further reduction of initial condition errors only buy us a logarithmic increase in predictive precision over time.

*While a lot has been said about Big Data and bias errors, one thing is certain: whatever data you can already collect (along with whatever biases it may contain), you can sample it randomly without introducing additional bias errors. In short: if you can store it -- you can sample it (without bias).
In light of these (and I could be wrong, of course), it seems that any additional sample we collect has diminishing returns on statistics as well as predictions (and the benefits diminish quite rapidly). It seems, then, that even if storing and analyzing Big Data is relatively cheap, it still doesn't pay. We get lots of data that buys us little if any additional knowledge: very little extra statistic accuracy, no less bias, and virtually no added predictive ability[1]. What, then, are the benefits of Big Data?
(This question is most not a duplicate of Is sampling relevant in the time of 'big data'?, and in any event, the answers to that question don't answer mine)
[1] This last point -- predictive ability -- seems the most pertinent, as that is what many commercial uses for Big Data are for. But user behavior changes all the time -- probably with some complicated feedback, and possibly non-linearly -- so whatever extra accuracy we get, say n^-1/2, this meager gain then becomes logarithmic when predictions are concerned. In fact, it can be argued that to get better predictions, it's preferable to reduce the time it takes to calculate the statistics (by sampling), than to enhance precision by increasing the sample size, because time has an exponential effect on "knowledge", while sample size has a mere polynomial effect.
 A: it isn't only the data that is big, also the problem is big. 
Indeed, the benefits of increasing the sampling size aren't great if you were computing the mean of terabytes of data. Only that nobody is interested in the 10th digit of the mean anyways...
More often than not,  big data problems are more like a big amount of problems to be solved at once. You have millions of users, thousands of products. The sample sizes for each of them aren't big data,  but you have a lot of them... similarly, in image recognition, you have lots of pixels,  lots of labels (imagenet has some 20000 categories or so),  so more often than not you don't even have a single training example that is really similar... 
When searching a large space of hypotheses, you also need to adjust for multiple testing problems. Say you are testing for problems with an $\alpha=0.999$ certainty. But you are testing just 100 hypotheses, then you end up with a certainty of only $\bar\alpha=0.90$ that the result is really correct. And this confidence drops quickly - at 1000 tests, there is a 2 in 3 chance of having a false positive. A (at least theoretical) way out is to use a larger $\alpha$ such as $\alpha=0.99999$ in the individual tests. But then you may need to get a larger sample to be able to reach such a confidence ever...
