# What is the outline for the procedure of model selection, with different models based on likelihood functions?

I have two basic models $M_1$ and $M_2$. They each have a likelihood function; $L_{M_1} = f(\mathbf{X}|\mathbf{\theta_1})$ and $L_{M_2} = f(\mathbf{X}|\mathbf{\theta_2})$ (here $\mathbf{X}$ is the data set, and $\mathbf{\theta_i}$ are the parameters for the model). The models are nested since model $M_1$ parameter's are all present in model $M_2$. Specifically, $\mathbf{\theta_1}$ has 2 parameters and $\mathbf{\theta_2}$ has 3 parameters.

QUESTION 1)The model $M_1$ is termed a 'null model' in this case as it is the more simple in the nested case?

QUESTION 2)I want to do model selection. The more complicated model, $M_2$, has a greater likelihood but I would like it to be statistically justified. Is it correct to assume that the likelihood-ratio-test is the suitable choice to apply?

QUESTION 3)For the likelihood ratio test, I am supposed to use the parameter values which maximize the likelihood. In the case of model 1 $M_1$ I can do this analytically, but for model 2 $M_2$ I must approximate it via sampling. Is it suitable that I use $\mathbf{\theta_1}^{max}$ and $\mathbf{\theta_1}^{approx.max}$ in the same mechanism for comparison?

I applied the likelihood-ratio-test, and get this result (in logs):

LogLikModel1 = -61159.4324
LogLikModel2 = -30577.2917
LogLikRationTest_D = 61164.2813


QUESTION 4)To estimate the significance the chi-squared, $\chi^2$ test must be used?

QUESTION 5)If the answer to the question 4) is yes, then must I use this table (link to chisquared and p-values) for mapping the degrees of freedom and chi values to p-values? Are the numbers that the likelihood-ratio-test produce chi values? How do I use these values in logs with the table for chi-squared p-values because the probabilities are too low to not use logs?

(1) Yes, that's correct. The null hypothesis is that the sub-model is sufficient and the extra parameters in the larger model are unnecessary.

(2) Yes, as long as the null model is a sub-model of the alternative model, the likelihood ratio test is justified.

(3) The theory of the likelihood ratio test is based on the exact value of the maximized log-likelihood, so this could be a concern. However, the approximate optimized log-likelihood could be an underestimate (but not an overestimate, since it cannot exceed the true maximum), so your observed test statistic will be too small, if anything, so the test result will be conservative (i.e. if you observe a significant result, you can be confident that it would still be significant if you had the true likelihood).

(4) Yes. The likelihood ratio test statistic has a $\chi^2$ distribution under the null hypothesis.

(5) Use degrees of freedom equal to the number of parameters you had to delete to produce the submodel (1 in your case, apparently). You can use a table but unless you test statistic value is right on the table, you won't be able to calculate the exact $p$-value. You could use, for example, the pchisq function in R to calculate $p$-values. For example,

 1-pchisq(LRT,df=D)


will give you the $p$-value when the observed test statistic is the variable LRT and the degrees of freedom is D. However, in your case your observed test statistic is $>60000$ when you add only one parameter, resulting in a $p$-value of essentially 0. A jump of over 30000 log-likelihood points indicates either a very large effect and/or a very large sample size. So, you don't need a table to see that the additional parameter is very important.

Please note that the model $$M_2$$ must be correctly specified So that it contains the distribution which generated the data. If it does not then the likelihood ratio test statistic has a weighted chi- square distribution ( e.g. Vuong 1989 econometrica)