Akaike Information Criterion I cannot interpret the result Maybe is a silly question, or maybe I'm doing something wrong.
I've tried to implement AIC criterion to estimate the optimum number of parameters using Auto Regressive (AR) linear models using white noise as input.
$$AIC(p) = N\log(RSS) + 2p,$$
where $N$ is the number of observations and $RSS$ the residuals sum of squares.
The results does not really make sense to me: since the predictors variables are completely indipendent, I would expect an increasing values of aic, so why do I see such a large oscillations?     
x = randn(100000,1);  % Generate white noise
for p = 1:30; 
    Z = [];
    for z = 1:p;                            % Here I build the predictors variables
        Z = horzcat(Z, x(z:end-p-1+z));
    end
    [b,bint,r] = regress(x(p+1:end),Z);     % Linear Regression
    aic(p) = N*log(sum(r.^2)/N) + 2*(p+2);  
    
end

 A: The most probable reason is that the AIC is not suited for this task. The Akaike Information Criterion and the Bayesian Information Criterion are two criteria for model comparison and selection, which are respectively defined as
$$
AIC  = -2\log( \hat{L} )+ 2p
$$
and
$$
BIC  = -2\log( \hat{L} )+ \log(N)p
$$
where $N$ is the number of data points. Although they differ in the way they are derived, we can consider, for this specific application, that they only differ in the way they penalize the number of free parameters $p$ in a model. A good reference on the differences between the AIC and the BIC is the following one :
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: understanding AIC and BIC in model selection. Sociological methods & research, 33(2), 261-304.
In the tapering-effects context (that is to say when your family of models shows a continuous increase of complexity), which is illustrated in Fig. 1 of the paper, AIC will perform better than the BIC. On the other hand, when you only have a few big effects (i.e. when there is a big jump in complexity when you rank your models by increasing complexity), which is illustrated in Fig. 2, the BIC should be favored.
Here, you are clearly in the latter case. An autoregressive model with $p=0$ implies that your data are uncorrelated and corresponds to white noise. Any model with $p > 0$ implies a temporal correlation between your data. So there is a big jump in complexity between $p = 0$ and $p=1$, but not so much between $p=1$ and any other strictly positive value of $p$.
Using the BIC instead of the AIC thus allows to meaningfully compare your models :

