I have a time-series data that I want to model using machine learning models like Lasso Regression, Ridge, elastic net, etc.
However, in order to make it stationary, I difference the output variable, which is resulting in negative values being present now in the differenced data.
However, I think that differencing is having a negative effect on the preformance of my models:
With Differencing: (Yes I do realize the above-100 MAPE, however I think this is also due to the effect of having negative values in the differenced data. I know MAPE has many pitfalls, I also wish if someone could help me solve this issue)
Best Validation Scores: R^2: 0.062 Adj R^2: -1.904 RMSE: 34.442 MSE: 1215.642 MAE: 26.633 MAPE: 231.714 Testing Scores: R^2: 0.313 Adj R^2: -0.473 RMSE: 33.659 MSE: 1132.937 MAE: 23.688 MAPE: 215.160
Best Validation Scores: R^2: 1.000 Adj R^2: 1.000 RMSE: 0.004 MSE: 0.000 MAE: 0.003 MAPE: 0.002 Testing Scores: R^2: 1.000 Adj R^2: 1.000 RMSE: 0.004 MSE: 0.000 MAE: 0.003 MAPE: 0.002
I just want to know if
- this is normal or not ?
- and how to deal with differenced data ?
- is avery low RMSE (around 0) and a very high R^2 (around 1 and some cases 1) a sign of overfitting ?
Note: My dataset is small in size (154 samples). However I tried to gather more data to make it bigger, but the same thing is happening.