New answers tagged aic
1
Stepwise model selection, particularly forward stepwise, is not very reliable. This page provides much general discussion.
With Cox models the problem is even worse than for ordinary linear regression.
In ordinary linear regression there is no bias if you omit a predictor that is uncorrelated to the predictors you include. In other types of regressions ...
1
Consider scalar parameters $\theta_0$ and the corresponding scalar estimate $\hat \theta$ for simplicity.
I will answer Q1 and Q3 which are essentially asking why is the mean of the score function $\Bbb{E}_{\theta}(s(\theta)) =0 $. This is a widely known result.. To put it simply, Notice that score function $s(\theta)$ depends of the random observations $X$...
4
The auto.arima() algorithm from the forecast package for R does look at the characteristic roots and will not return a model with near-unit-roots even if it has a low AICc. This is explained at the very end of the link you cite.
1
Why should the above formula for AIC be the same regardless of the dimensionality of the data? Shouldn't there be some way (e.g., an additional term) dependent on the number of dimensions [of the data]?
There is no need for an additional term, since the maximised value of the likelihood function (which appears in the AIC formula) already maximises over all ...
0
Overfitting: Predicitng much worse on test data than training data. i.e. Does not generalize outside of particular data used to train model. Which means if it can't predict outside of data used to train, it probably doesn't explain much about the process used to generate either datasets.
I suggest you read/watch Statistical Rethinking by McElreath. He ...
5
Why can't algorithms avoid overfitting themselves?
They can. If you design an algorithm that implements model selection based on cross-validation or information criteria, you should achieve a good balance between overfitting and underfitting.
What I don't understand is why this requires a human to hide data from the algorithm. Doesn't this imply that ...
1
No.
The simplest way to investigate such a question is to simulate. Here is R code with 100 observations and 20 predictors, and we compare forward and backward stepwise model building:
set.seed(1)
library(MASS)
dataset <- as.data.frame(matrix(rnorm(2100),nrow=100,dimnames=list(NULL,c("y",LETTERS[1:20]))))
stepAIC(lm(y~1,dataset),scope=y~.,direction="...
answered Oct 29 at 19:15
S. Kolassa - Reinstate Monica
58.9k10 gold badges113 silver badges216 bronze badges
1
Since you are evaluating the error on the holdout subsample, there is no reason to adjust twice the estimated negative log-likelihood for overfitting by adding $2p$. This adjustment would introduce a bias in your estimated twice the negative log-likelihood without adding any benefits to the estimate. Hence, I would not do as you do.
However, you could ...
0
Determining optimal lags is a double-edged sword. I agree with the procedure mentionend in the comments in principle. The advantage is that you will save degrees of freedom.
Other suggetions:
-You could try to seasonal adjust your data. This might reduce the "needed" lags.
Some generel comments on your Var:
-Do you checked for cointegration? Might worth ...
5
Yes it is a valid approach in the sense that it's been proposed and used in the published literature:
See for example this review paper on model selection and model averaging where the specific technique you propose is discussed on pages 16 (for the weights) and 18 for the single parameter estimation based on AIC weights:
http://byrneslab.net/classes/...
1
The AIC is a creature of information theory. It began when Akaike realized that the maximum likelihood estimator is biased upward in $k$. It also happens that the bias is approximately equal to $k$ under a set of highly stylized circumstances. The AIC could be thought of as a constant minus the expected information. It is also approximately equal to the ...
Top 50 recent answers are included
