2
$\begingroup$

Suppose you are getting a stock's prices at hourly intervals and you want to use those to model the stock's returns (relative to the day's starting price). When you get the stock price, you are also getting some features measured at the same time as the price. Say you build a regression model from the computed returns and the features. Now, your boss tells you, "variance of regression coefficients decreases if you increase the sample size, so why not just sample at minute intervals rather than hourly intervals? You'll get 60x data". Is this sound advice?

$\endgroup$
2
$\begingroup$

This sounds like an example of Goodhart’s law, where minimizing variance became aim by itself, rather than the metric.

The boss is technically correct, standard error is $\sigma/\sqrt n$, so the larger the sample, the smaller it would be. So the more data you have, the less the estimates would vary. You can achieve this by sampling your data more frequently, sure. You would achieve same effect with repeating every row in your data, to get ten-fold decrease. “Technically” this would work, but not necessary improve the results. For example, if you’re are doing daily forecasts, than usually daily data would be enough for your, and having data in minute frequency would not help much. Moreover, such data would be likely autocorrelated, so the minutely rows would be very similar to each other. With repeating rows this should be even more obvious, since it does not add any new information at all. Always ask yourself in such cases if you gain new information with the additional rows, maybe not?

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ The point about serial correlation is good, but the correlation that matters is among the errors, not in the raw data themselves (as your language might be misinterpreted to suggest). Certainly there's a possibility that residuals in a sufficiently dense (in time) sample will be positively correlated, but there may be circumstances in which they are not. $\endgroup$ – whuber Oct 9 at 20:51
  • 1
    $\begingroup$ Just a side comment : I think the value $\sigma/\sqrt{n}$ for the standard error only holds when data are independent and normally distributed. If data are correlated, the information might saturate with $n$, and adding new data points would not help reducing the variance of the estimator : en.wikipedia.org/wiki/Effective_sample_size $\endgroup$ – Camille Gontier Oct 10 at 16:05
  • 1
    $\begingroup$ @CamilleGontier agree. $\endgroup$ – Tim Oct 10 at 16:32
  • $\begingroup$ @Camille Yes, independence is needed for that formula to be accurate, but Normality is not. A sufficient condition is that the data are independent and have a common finite variance. $\endgroup$ – whuber Oct 11 at 14:47
1
$\begingroup$

I agree with the previous answer, with some additions : your boss's advice is not unsound. As you are only increasing the number of observations, and not their nature (i.e. what you observe) or the sampling protocol, the variance of your estimator can only decrease. How it will decrease depends on the correlation structure of your observations :

  • Best case scenario : all your observations are i.i.d., and the variance will tend to 0 as the number of observations $n$ increases (https://en.wikipedia.org/wiki/Effective_sample_size)
  • Worst case scenario : your new observations are perfectly correlated with the previous ones, and thus do not bring any information. Then the variance of the estimator is left unchanged.

Intuitively, I guess that observations sampled with a short time interval will be highly correlated.

An interesting reference on how correlations are linked to the information content is the following one : Moreno-Bote, R., Beck, J., Kanitscheider, I., Pitkow, X., Latham, P., & Pouget, A. (2014). Information-limiting correlations. Nature neuroscience, 17(10), 1410-1417.

One way to optimize your observation times so as to minimize the variance of your estimators is to perform Optimal Experiment Design (OED). Speaking of Goodhart's law, I was concerned that such an optimized protocol (in which you only try to reduce the variance of your estimates) might in return increase its bias. However, I could not come up with an example of a protocol that would reduce the variance of your estimates while increasing its bias : Biased estimator obtained by optimal experiment design

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.