Suppose I'm measuring an athlete's 1 mile run time once every week-end. At different times during each week I measure regressors such as the amount of water that she drank each day, the amount of food eaten during each meal, daily minutes of exercise etc.

I'm trying to fit a regression model to my data and forecast the athlete's week-end run time on a daily frequency so I can custom tailor a daily diet/fitness plan, but I run into a dilemma here.

Subsampling my feature matrix on a weekly frequency and extrapolating my prediction from last week has stronger guarantees of independence. However, I have more recent data by the end of the week and so my forecast usually improves by the end of the week, so just subsampling seems inferior to taking the most recent data point each day.

On the other hand, if I used the highest granularity, i.e. every 7~ daily observations of feature vectors that map to the same week-end height, my observations become non-independent and so it hurts my $p$-values for most choices of models.

What is the best way I can combat this issue?

  • 1
    $\begingroup$ I am not certain I understand your problem: Do you want to just estimate one set of coefficients? Or you want to do data assimilation, where you update the estimates as new data comes in? (Perhaps coupled with feedback-control, i.e. where you are simultaneously implementing the diet/fitness plan.) What is the purpose of trying to determine "p values"? (vs. just coefficients and/or prediction error) $\endgroup$ – GeoMatt22 Oct 12 '16 at 2:15
  • $\begingroup$ @GeoMatt22 My bad for being unclear. I'm open to both approaches: either estimating one set of coefficients or updating estimates as new data comes in. In my post, I'm saying that I thought of estimating a single set of coefficients for the regression of the week-end run time against daily observations, but I would run into the problem of non-independent observations. $\endgroup$ – elleciel Oct 12 '16 at 2:46
  • $\begingroup$ @GeoMatt22 Also let's assume my setting is open loop so there is no feedback control. In the example given, I'm just trying to determine static coefficients that minimize prediction error, but my understanding is that the significance of the estimated coefficients is worsened in the presence of non-independent observations. $\endgroup$ – elleciel Oct 12 '16 at 2:55
  • 1
    $\begingroup$ I was thinking that "p values" seem like more of an issue for a "once and done" inference problem. But if you have new data coming in, for the data assimilation problem, you can then check/update predictions. Now I think your concern is not "significance" per se, but reliability of estimated coefficients. If you want to decrease the number of coefficients (predictors), a simple approach would be to reduce the daily observations with PCA. $\endgroup$ – GeoMatt22 Oct 12 '16 at 3:00
  • 1
    $\begingroup$ More generally, Kalman filtering could be a natural framework for your problem, including both regularization* and (possible) control aspects. (*This would be the term for your concern I think, essentially too many predictors may overfit, predicting the noise rather than the signal.) $\endgroup$ – GeoMatt22 Oct 12 '16 at 3:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.