2
$\begingroup$

I have a time-series data that I want to model using machine learning models like Lasso Regression, Ridge, elastic net, etc.

However, in order to make it stationary, I difference the output variable, which is resulting in negative values being present now in the differenced data.

However, I think that differencing is having a negative effect on the preformance of my models:

With Differencing: (Yes I do realize the above-100 MAPE, however I think this is also due to the effect of having negative values in the differenced data. I know MAPE has many pitfalls, I also wish if someone could help me solve this issue)

Best Validation Scores:
R^2: 0.062
Adj R^2: -1.904
RMSE: 34.442
MSE: 1215.642
MAE: 26.633
MAPE: 231.714

Testing Scores:
R^2: 0.313
Adj R^2: -0.473
RMSE: 33.659
MSE: 1132.937
MAE: 23.688
MAPE: 215.160

Without Differencing:

Best Validation Scores:
R^2: 1.000
Adj R^2: 1.000
RMSE: 0.004
MSE: 0.000
MAE: 0.003
MAPE: 0.002

Testing Scores:
R^2: 1.000
Adj R^2: 1.000
RMSE: 0.004
MSE: 0.000
MAE: 0.003
MAPE: 0.002

I just want to know if

  • this is normal or not ?
  • and how to deal with differenced data ?
  • is avery low RMSE (around 0) and a very high R^2 (around 1 and some cases 1) a sign of overfitting ?

Note: My dataset is small in size (154 samples). However I tried to gather more data to make it bigger, but the same thing is happening.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

differencing is a form of filtering . Unwarranted filtering like differencing or uneeded power transformations can have a deleterious effect.

the original series does not have to be stationary. the residuals from a useful model must be stationary.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thank you @IrishStat, however, I am only afraid of the huge decrease of the error metrics when I don't difference (I mean, R^2 of 1 and RMSE of nearly zero!!) $\endgroup$
    – Perl Del Rey
    Sep 21, 2019 at 17:44
  • $\begingroup$ To be of further help .. I would need to look at the data . Please post it a csv format and I will try to look at it . If for some reason the data is proprietary simply code it by adding constant1 and dividing by constant2 ..this preserves the underlying model .. $\endgroup$
    – IrishStat
    Sep 21, 2019 at 17:50
  • $\begingroup$ I have no knowledge whatsoever about the assumptions using machine learning models like Lasso Regression, Ridge, elastic net, etc . If you review my most recent response comparing some assumed models stats.stackexchange.com/questions/427996/… with EDA you will better understand what I can do to help. $\endgroup$
    – IrishStat
    Sep 21, 2019 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.