I have to forecast future energy consumption. I decided to use ARIMA model. But my model's forecast shows the wrong trend. The blue line shows true value. And the orange line shows my model's forecast. The x-axis represents time index. The y-axis represents the quantity of energy consumption. My model's forecast represents upward trend. However, the true value doesn't show upward trend.

How can I remove my model's wrong trend?

model_1 = auto_arima(total['meter_reading'],exogenous = total.iloc[:,2:], max_p= 2, max_q=50,out_of_sample_size = 720)


enter image description here

fig = plt.figure(figsize=(50,10))
plt.title('site_id: 2 | Office' ,fontsize=30)
plt.xlabel("Date of measurement",fontsize=30)
plt.plot(total['meter_reading'][-720:].reset_index(drop = True))
plt.plot(model_1.predict(n_periods = 720, exogenous=total.iloc[-720:,2:]))

enter image description here

I post the head of my data.

enter image description here

And when I predict, my X variables(exogenous variables) are not forecasted values.

It's actual data.

With my real exogenous variables, I have to forecast target variable.

This is possible because I'm building my model with past data.

I also post my exogenous variables I use when I predict.

enter image description here


The 5 predictors future values may be the cause of your "increasing forecast" OR a trend coefficient tat is unwarranted . How are you specifying the values for these (X) series into the future ?

Additionally your arima model might be questionable as it has redundant arima structure . Also note that AUTOBOX used differences of all series as triggered by the arima structure where your software did not. This might be the most important point. Your graph strongly suggests something is awry with your current tool of choice..

If you post Your data and any future predictions for the 5 X's , I will try and help further. Ultimately you might want to question the author of auto.arima as to why you are getting "silly forecasts".

auto.arima is easily fooled when anomalies exist and are untreated. The 4 arima coefficients look self-cancelling (redundant) to me and are posssibly "significant" due to unnecessary differencing. The arima model should be identified/based upon the Y adjusted for the X's NOT the original Y which might be the cause of your dilemma. If auto.arima is identifying the arima structure using the original Y values then this is a questionable approach.

The data should usually be scaled to make all the variables similar in "size" .It appears that Y and X4 should be scaled . Another thought is that if the user-specified future values of the causals are inconsistent with the history of the causals thus might be root problem.

If you wish you can get help to actually add a csv file to your post or failing that email me the data and I will try to help further.


You sent 366 days of hourly (scaled to be commensurate) readings on 6 series thus 366x24 values . I dropped from the analysis the last 600 values and made a 600 period out forecast. My initial mission was to develop an equation similar to yours to assess possible issues or to replicate the results that you reported . I used AUTOBOX a piece of software that I have helped to develop and restricted the automatic process in a number of ways see https://autobox.com/pdfs/SARMAX.pdf for the general equation.

Here is the data enter image description here

Here is the forecast plot with 95% confidence intervals enter image description here

Here is the model ( X5 was found to be not-significant and dropped from the model enter image description here

Here is the equation enter image description here

Note that the arima model is (1,1,0) which is both statistically significant AND sufficient.

The residuals form the model are suspect and here enter image description here

When I executed the analysis I needed to restrict AUTOBOX to your level of analysis:

1) No outlier detection ( pulses,level/step shifts,local time trends)

2) No augmentation tests for any error variance fixup ( Note clear change in error variance over time)

3) No day-of-the-week effects

4) No day-of-the-month effects

5) no month-of-the-year effects

6) No holiday effects

7) No long weekend effects

8) No hourly effects

9) No week-in-month effects

For you to adequately model your data you need to consider possible model augmentation using one or more the 9 feature extraction options that I surpressed .

I think you should address your next question to the author of your software as there appears to be something anomalous about your results . I also suggest that you browse https://stats.stackexchange.com/search?tab=newest&q=user%3a3382%20hourly to examine other discussions about the analysis of hourly data.

  • $\begingroup$ I edited my post. I don't know how to post the entire data. So I uploaded head of my data. As I mentioned in my post, 5 X's are not future predictions. 5 X's are actual value. I'm sorry for the confusion caused. $\endgroup$
    – jjw
    Nov 21 '19 at 5:50
  • $\begingroup$ I found that i cannot upload data file into stackexchange(meta.stackexchange.com/questions/240242/…). In addition, I'm cautious to upload data into public space. Could I email you? If you let me know of your email, I will email you. $\endgroup$
    – jjw
    Nov 21 '19 at 10:58
  • $\begingroup$ Look at my profile ...you can get it there or consider file sharing services like DropBox or Google Drive $\endgroup$
    – IrishStat
    Nov 21 '19 at 11:04
  • $\begingroup$ I sent an email to you Thanks a lot!!. $\endgroup$
    – jjw
    Nov 21 '19 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.