0
$\begingroup$

From a data stream i'm receiving a pair of measurements consisting of a current consumption and a current percentage every second. By accumulating the consumption over time it will represent eventually the maximum capacity when the percentage reaches from 100% to 0%.

I want to predict the maximum capacity in (almost) real time using linear regression with a small sample size window of two percent. However, when i compare the models of these local regressions of every two percent with the model of the whole data regression, i get very different results due to perhaps local fluctuation. (see figure)

Is there a way to bring the local regression models closer to the whole data model? (in a way that i can see the differences due to fluctuation but overall closer predictions to the whole data model)

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

First of all, if your fluctuations are caused mainly by randomness of your data you might try some regularization techniques: Lasso, Ridge, ElasticNet.

The other solution could be to remember previous model (the one you get at the previous step with data from the previous window) and take its coefficients into account when fitting model with data from a new window. You might get some inspiration from momentum gradient descent techniques.

Improve this answer
$\endgroup$
1
  • $\begingroup$ thx for your answer, i was looking into the solutions you mentioned and remembering a previous model sounds most promising. Although, i feel like manipulating the data a bit too much now. $\endgroup$
    – R. Doe
    Commented Feb 17, 2016 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.