0
$\begingroup$

I have made a prediction using an ANN. After that, I have created an error array, which values are the difference between the targets or real values and the predictions.

Once this is made, I want to validate my model. My idea is to do it by using the ACF and the PACF. So far, everything is ok and I obtain the folloging autocorrelation functions: ACFPACF

I think this is not a bad result (ok, there are some important residuals for the first lags, but he rest seem to be within the limits).

Trouble begins when I try to make the Box-Pearce test. All the P-values are really low, under the p-value (0,05). So, I could not consider my prediction is valid.

The code below show the result of Box-Pierce test. The first Matrix shows the statistic values, and the next one shows the p-values (I understand that they are estimated by establishing similarity with the X squared function). So, they are (enormously) under the limits, as I said.

array([ 272.110743 , 405.31175231, 439.75638084, 445.73745259, 451.0971869 , 451.75916969, 452.3519487 , 452.89218161, 452.95798727, 453.48704933, 453.62826167, 453.79590874, 453.88635655, 453.89157604, 454.25958408, 454.26605408, 454.64673648, 454.7396868 , 455.01564022, 455.23178931, 455.39501516, 456.35546367, 456.64498206, 456.73760066, 456.73775197, 456.76631803, 456.78216617, 457.02755731, 457.0666677 , 457.10870398, 457.12327263, 457.13057787, 457.18305857, 457.22595984, 457.44212799, 457.58639875, 457.70087134, 457.7026865 , 457.72151729, 457.83367567, 457.84085643, 458.05775687, 458.34272957, 458.35522179, 458.43899971, 458.83023313, 458.92382469, 459.19209704, 459.30614482, 459.37937575, 459.50949545, 459.60762723, 459.95879364, 460.42219166, 460.96503055, 461.83552692, 461.8417434 , 462.37303705, 462.4111623 , 462.44848182, 462.45085587, 462.52814275, 462.89330548, 463.19514212, 463.21142978, 463.2740221 , 463.27477873, 463.27499291, 463.44599611, 463.56698722, 463.78137471, 463.82198247, 463.83472034, 464.08570346, 464.20285156, 464.21131114, 464.74779748, 464.98531181, 465.01226428, 465.04772004, 465.08171724, 465.09383896, 465.09536112, 465.26019974, 465.29071963, 465.35609093, 465.36409458, 465.3643095 , 465.44287389, 465.44294162, 465.98331855, 465.98752041, 466.05838656, 466.2727417 , 466.27373189, 466.40820921, 466.49389354, 466.62259119, 466.80657811, 466.87438758, 466.87738561, 466.88799094, 466.88934776, 466.98057218, 466.99017527, 467.67866287, 467.70313436, 467.74555521, 467.86168569, 468.51407921, 468.75513049, 468.89706442, 469.25969911, 469.33234657, 469.34241964, 469.38624359, 469.38842281, 469.40307366, 469.51403827, 469.51420302, 469.51864191, 469.52036341, 469.54766635, 469.54783568, 469.55055976, 469.58065015, 469.62841149, 469.64405012, 469.64581608, 469.64732833, 469.68438062, 469.71349646, 469.776721 , 469.85017379, 469.8618585 , 469.86482696, 469.86727215, 469.87906817, 469.87921847, 470.03059261, 470.14087279, 470.14093203, 470.30519246, 470.321453 , 470.38251885, 470.49461275, 470.56533691, 470.69619433, 470.69704975, 470.7253107 , 470.83405561, 470.85297236, 471.28296371, 471.29249131, 471.30610246, 471.3144643 , 471.38383622, 471.40879785, 471.44070629, 471.4483675 , 471.45094532, 471.54371279, 471.54758225, 471.66917668, 471.67299552, 471.78189924, 471.92764016, 472.26295012, 472.48249884, 472.83264735, 472.88645481, 472.902301 , 472.9318239 , 473.15098257, 473.40471359, 473.6354701 , 473.63612789, 473.63695487, 473.69504236, 473.71545137, 473.74522537, 473.75902572, 473.76663223, 473.77684943, 473.77947599, 473.77950094, 473.78420699, 473.79000749, 473.84358855, 473.95988891, 473.98297885, 474.10741401, 474.25230545, 474.41629502, 474.4863328 , 474.69115645, 474.86070165, 474.87653637, 474.98513961, 474.98533702, 475.10368542, 475.11420696, 475.28946066, 475.29303731, 475.29351975, 475.29435603, 475.29448767, 475.30147089, 475.31299415, 475.33162323, 475.37251441, 475.39703623, 475.39845353]), array([ 3.93449071e-61, 9.72011179e-89, 5.40313202e-95, 3.62522870e-95, 2.84839023e-95, 2.05263630e-94, 1.38793137e-93, 8.87609116e-93, 6.68513169e-92, 3.76844232e-91, 2.43330830e-90, 1.47596379e-89, 8.88712539e-89, 5.35155942e-88, 2.60007571e-87, 1.45359733e-86, 6.55927073e-86, 3.29881877e-85, 1.47545308e-84, 6.60956752e-84, 2.95950172e-83, 8.83104705e-83, 3.55130700e-82, 1.53418723e-81, 6.77786031e-81, 2.89345428e-80, 1.21814096e-79, 4.51465591e-79, 1.81093920e-78, 7.12643120e-78, 2.79244008e-77, 1.07981416e-76, 4.02320583e-76, 1.48251071e-75, 4.96551971e-75, 1.69515176e-74, 5.78582240e-74, 2.05186551e-73, 7.12313276e-73, 2.33834223e-72, 7.95332666e-72, 2.42766897e-71, 7.10025320e-71, 2.32396833e-70, 7.27975796e-70, 1.96227929e-69, 5.98784386e-69, 1.67109429e-68, 4.94814786e-68, 1.47710577e-67, 4.25576996e-67, 1.23174311e-66, 3.15513295e-66, 7.62138858e-66, 1.76239471e-65, 3.49766709e-65, 1.00864087e-64, 2.28852909e-64, 6.39750609e-64, 1.77383197e-63, 4.95217856e-63, 1.32708539e-62, 3.11318217e-62, 7.45182706e-62, 2.00341018e-61, 5.23877001e-61, 1.39624593e-60, 3.69410298e-60, 9.01711271e-60, 2.23289252e-59, 5.27663834e-59, 1.33356538e-58, 3.38675350e-58, 7.72333994e-58, 1.85182224e-57, 4.61772874e-57, 9.16502214e-57, 2.05035133e-56, 4.97867896e-56, 1.19703134e-55, 2.86185175e-55, 6.86192719e-55, 1.64240888e-54, 3.65281892e-54, 8.53697650e-54, 1.95531743e-53, 4.55861430e-53, 1.06002041e-52, 2.37384999e-52, 5.45807862e-52, 1.00269083e-51, 2.27736258e-51, 5.00787940e-51, 1.03404999e-50, 2.31437154e-50, 4.88476817e-50, 1.04602971e-49, 2.19096283e-49, 4.46769567e-49, 9.49396073e-49, 2.05962798e-48, 4.43266170e-48, 9.52751333e-48, 1.96741236e-47, 4.17470494e-47, 6.76846474e-47, 1.41544677e-46, 2.92586926e-46, 5.85145710e-46, 9.47621400e-46, 1.79162426e-45, 3.50433771e-45, 6.27324486e-45, 1.24966175e-44, 2.53848888e-44, 5.06882251e-44, 1.02384444e-43, 2.04951748e-43, 3.94002494e-43, 7.86425584e-43, 1.56063339e-42, 3.08732409e-42, 6.02479771e-42, 1.18282664e-41, 2.31061552e-41, 4.45048042e-41, 8.48317233e-41, 1.62985185e-40, 3.13507931e-40, 6.00755625e-40, 1.13203241e-39, 2.13125851e-39, 3.94850669e-39, 7.26181236e-39, 1.36060911e-38, 2.54783746e-38, 4.75432845e-38, 8.80987406e-38, 1.63333642e-37, 2.85995820e-37, 5.06432903e-37, 9.29151554e-37, 1.60356672e-36, 2.90538172e-36, 5.16452537e-36, 8.98875322e-36, 1.58200455e-35, 2.71809641e-35, 4.86893706e-35, 8.61092309e-35, 1.47658413e-34, 2.60286316e-34, 3.97558599e-34, 6.98782832e-34, 1.22257688e-33, 2.13592664e-33, 3.64408610e-33, 6.29126594e-33, 1.08024429e-32, 1.86413128e-32, 3.21225809e-32, 5.35566453e-32, 9.16875664e-32, 1.50534787e-31, 2.56175677e-31, 4.19939163e-31, 6.78266693e-31, 1.02720095e-30, 1.61143383e-30, 2.41728017e-30, 3.98077543e-30, 6.61733600e-30, 1.09205554e-29, 1.69149628e-29, 2.58497509e-29, 3.96996081e-29, 6.54199200e-29, 1.07493467e-28, 1.72987386e-28, 2.80938745e-28, 4.53678913e-28, 7.34294648e-28, 1.18752913e-27, 1.91375433e-27, 3.08298577e-27, 4.95713772e-27, 7.93778548e-27, 1.26725639e-26, 1.98870385e-26, 3.05435889e-26, 4.81343167e-26, 7.33880391e-26, 1.10945200e-25, 1.66384536e-25, 2.56036793e-25, 3.77617199e-25, 5.61574285e-25, 8.71983245e-25, 1.31425282e-24, 2.04010147e-24, 3.05178795e-24, 4.70013029e-24, 6.88416735e-24, 1.05737058e-23, 1.62154511e-23, 2.48043736e-23, 3.78583817e-23, 5.75307181e-23, 8.71045462e-23, 1.31304980e-22, 1.96236933e-22, 2.93957477e-22, 4.42174505e-22]))

This has been made by calling:

from statsmodels.stats.diagnostic import acorr_ljungbox box_pierce=acorr_ljungbox(error, lags=int((len(error)-1)/5),boxpierce=True)

I do not understand why I get discordance between the ACF's and the Box-Pearce test. Can anybody say what's happening?

EDIT

I have used this function to calculate the Box-Pierce test.

It seems to be that I have used it in the wrong way, but I cannot find the mistake.

If I have understood well, I have written the function properly(?). But if the ACF and PACF look not so bad (correlations seem to be within the limits of 95%), why the results of the Box-Pierce test are so bad?

I add more information. What you can see below is an image of a significative part of the predicted time series vs reference time series. By seeing this, I would say predictions are not bad. The problem is that the errors are big because the variable I'm predicting has big values. Has this something to do with the (bad) results of residuals? prediction

And I think I got a good R coefficent: R= 0.9762correlation coefficent

$\endgroup$
4
  • $\begingroup$ There is no way your array of values around $450$ corresponds to the Box-Pierce statistics for the data shown in the graphs. Could you clarify what those values actually are? $\endgroup$ – whuber Jun 12 '17 at 14:40
  • $\begingroup$ @whuber I answer because I've just created an account and I am not able to make comments until I have 50 of reputation. Those values are obtained with this function: > box_pierce=acorr_ljungbox(error, > lags=int((len(error)-1)/5),boxpierce=True) as I said in the post. The number of lags is the error array length divided by 5 (this is a criteria I have read about). Maybe I'm doing something wrong, but I can't find the error. $\endgroup$ – Jvr Jun 12 '17 at 14:48
  • $\begingroup$ Please visit stats.stackexchange.com/help/merging-accounts to merge your accounts: as the original poster of this question, you will be able to edit it and comment on it. Your comment indicates you may be confusing your data with the error argument. The help page says that this argument is supposed to be "regression residuals when used as diagnostic test," but since these values don't balance out around zero, they're obviously not residuals. $\endgroup$ – whuber Jun 12 '17 at 15:22
  • $\begingroup$ I don't see it clear when you say I migth be confusing my data with the error argument. You suggest I am not using residuals? The error array is the difference between the reference data and the prediction. I understand these differences are the residuals. Am I rigth? $\endgroup$ – Jvr Jun 12 '17 at 17:02
1
$\begingroup$

The autocorrelation at the first couple of lags is too strong for the Box-Pierce test to tell you it is not statistically significant. Also the lag six is statistically significant and lags three, four and seven are borderline-significant. Pretty pronounced autocorrelation, I would say.

Normally the first lags are the ones that matter the most. We can think that autocorrelation at some further lags may spike due to chance (except for the lags corresponding to the seasonal period(s), which can sometimes be long), while the ones at the first few lags are often treated as genuine as we can find subject-matter or model-specific explanations why they could be there.

In short, both the ACF and the Box-Pierce test tells you the residuals are autocorrelated.

$\endgroup$
1
  • $\begingroup$ Thank you Richard, Now I see that clearly it's not a good prediction. I must try to make another model. $\endgroup$ – Jvr Jun 12 '17 at 20:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.