1
$\begingroup$

Good Morning I’m trying to study a set of samples. In the original set there are several samples along the day. Samples time is not regular. But I can get average of that sample and fill the gaps. So I’ve made a new set containing average values per hour. I’ve 24 samples per day on a complete year. My target is to forecast next hour before that data is measured. Due nature of sample metrics, I think they cannot be 0 but they can be extremely little compared to the rest of samples so Naan and ceros have been reemplaced by 0.05 Quantile value. This is the graphical look Plot original data set

Histogram for that values (200 bins) is: Original Hist

Next I’ve taken a log of every sample. There is no ceros and taking a log helps on remove trend. It does not seem having trend , but I think it could not be bad. At least peeks will be smoothed. Another way to smooth is normalizing. I do not what is better, so I got this one. Once log is taken Histogram is Bimodal hist It seems it is a bimodal distribution. I need to find how it affects to an arima model… But in this moment this question is parked, to keep on thinking….As far I know it could be related to a seasonality but I cannot figure out how. But any comment about it is welcome. And now the real trouble. This is autocorrelation_plot of the serie. Autocorrelation Remember there is a sample every hour So 24 per day and 8760 Samples in a year. I’m new on this field, completely, so I ignored that result. It looks very nice.. But that’s all Next thing I did is the make an adfuller test. Pvalue is 2.7959091995853143e-22 <<<< 0.05 So it is Stationary. Isn’t It?

Well not only newbe. I’am a little bit nervous too. So I try some arima models playing with values of pdq , (Every new element is included in history to calculate new forecast) . My best bet was 202 model:

Prediction (red) Vs Original (blue)

Blue is original data and red forecasting. Both are closed and MSE for this model (2,0,2) is 0.3303230211924562.

Residuals Plot is enter image description here

Sligthy biased but centered. I do not know whether that bias is significative. Nother thing to inform about.

As far 202 is an ARMA these are model results

Looks Nice… But… After that, I kept reading… and from several articles I do not remember, it seems Prediction Vs Original plot show a periodic behavior… And I’m concerned for the damped oscillator plot that acf looks. I do not Understad at all

So I got my acf plot manually Acp & Pacf plot

First thing: I got better MSE with 202 rather than 101 as I think it is suggested by those graphs. ¿I’m wrong about 101?

Second, Yes I’ve have Stacionality, a 24 period to be exact. Because I’ve a hourly sample on daily basis. So clear as water, I need to diff the serie to avoid that seasonality.

Third: Every cols in Pacf are out of the blue confident area (it is no good true?)

If I understood correctly the way to remove seasonality is to diff, and then make the model, forecast and undiff the final result . This is the way I wanted to took

But when I differed on a lag of 24…. THERE IS NO AUTOCORRELATION. Both plots acf and pacf are plane. If I’m right, That implies I can forecast one value looking at the last day same hour but it is not true !!!. So I can try the first way but in the meantime, Why I get this result and what means. I have review every lines and it seems there no typhos, so I think there are some things, concepts, ideas I do not get completely along this travel…

¿Who can kindly help me?

$\endgroup$

1 Answer 1

1
$\begingroup$

After several days... I know what happened... So I'll answer myself for helping somebody were in similar situation.

1.- How to convert n modal to a normal distribution. 2.- Why I lost my correlation?

First the easiest and hardest question. The second. I lost correlation because I'M a COMPLETELY ROOKIE. I got the main serie differenced once by 24 lag... This implies that the 24th first elements are NAN. Plot acf and plt pacf do not work with nan's so the plot was empty. I realize that several days ago working on a different problem. Suddenly the plot was there!!!

Second question. The binormal distribution above is a 3-normal distribution. I found that result applying transforms. After that I normalized. Then I got distribution too pointy to be gaussian but with a single maximun. I think it is not a key result but it worked and I could have an arima model converged.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.