1
$\begingroup$

I am working on a regression problem of load prediction (where I try to estimate next hour consumption using previuous consumption values).

At first I had relatively poor perfromance, however when I added features like consumption_rolling average and temprature) I managed to get an R^2 of 1 and RMSE value of zero using a linear regression model (A random forest regressor had a very close perfromance, R^2 almost 0.98)

Now obviously this is an unrealistically hight R squared bascially no error. But I am not sure where could I have went wrong to achieve an RMSE value of 0?

To double check my results I ran my experiment in two different environments

  1. Microsoft Machine learning Studio
  2. local machine using sklearn linear_model (python)

My sample size is 7640 record (with 0.75 non randomized split)

(P.S. I am coming from a CS background)

Update (In response to comments)

  • Scatter Plot: Energy consumption (label) vs. rolling_energy (window of 3)

enter image description here

  • Scatter Plot: Energy consumption vs rolling temprature (window of 3)

enter image description here

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ Perfect prediction can either stem from some error you made or from data that allows basically perfect prediction. Both will be hard to distinguish without a reproducible example. $\endgroup$
    – Bernhard
    Commented Jul 13, 2021 at 8:36
  • 1
    $\begingroup$ I feel that your rolling average also involves that time instant. Also, maybe you can share scatter plots of rolling avg vs target, and temp vs target to see their expressive power. $\endgroup$
    – gunes
    Commented Jul 13, 2021 at 8:38
  • $\begingroup$ @gunes I added both scatter plots as requested. But is the rolling average having the time instant a bad thing? Giving that in production I should have access to the past 3 hours in production. $\endgroup$
    – A.Shoman
    Commented Jul 13, 2021 at 9:34
  • 1
    $\begingroup$ What do you mean by a “non randomized split”? $\endgroup$
    – Dave
    Commented Jul 13, 2021 at 9:57
  • $\begingroup$ @Dave I meant no shuffling is done prior to splititng. So in case of sklearn, random_state is set to None. (or it can be manually, where training data is the first n rows and the rest is test) scikit-learn.org/stable/modules/generated/… $\endgroup$
    – A.Shoman
    Commented Jul 13, 2021 at 10:05

0

Reset to default