# Comparison between Log-likelihood ratios and beta coefficients

I was asked a question recently which I could not find an answer for and was hoping someone could enlighten me.

The question was regarding the significance of a single variable in a linear model.

What is the difference between a traditional p-value which you would obtain from the t-statistic formula

$(\beta_1 - \beta_0) / \sigma(\beta_1)$

and log-likelihood ratio between a model with the variable in question, and a model without.

In the log-likelihood case, would a significant LLR suggest that the variable had a significant impact on the model? Assuming the LL for the full model was better than the subset model.

Is this result comparable to a traditional t-distribution p-value one would normally get from a linear regression? If not, why not?

What is the difference between a traditional p-value {(b1 - b0) / se(b1)} and log-likelihood ratio between a model with the variable in question, and a model without.

Depends on what you do with the likelihood ratio. If - as is sometimes done - you work out the LR test statistic and then manipulate it in order to get a statistic whose small sample distribution can be worked out, you're rejecting the same cases as the LR test should, without using asymptotic results. But usually when people apply an LR test they usually base their likelihood ratio test on applying Wilk's theorem, yielding an asymptotic chi-square approximation.

If you take the t-statistic you mentioned, it's asymptotically normal... if you square that, you have what is (that same) asymptotic chi-square test statistic.

As a result, in small samples from normal distributions, the two tests aren't identical, but as n increases they become more similar (reject exactly the same cases, as the significance level of the approximation becomes exact).

In the log-likelihood case, would a significant LLR suggest that the variable had a significant impact on the model?

Yes, up to the fact that your test is not exactly at the desired significance level.

Assuming the LL for the full model was better than the subset model

It's always at least as good.

Here's an example in R. I am going to use the glm function because I can get the t-ratio and the asymptotic chi-square test statistic from the output. I will use the cars data available in R (a few less interesting lines have been cut out).

summary(glm(dist~speed,cars,family=gaussian))

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -17.5791     6.7584  -2.601   0.0123 *
speed         3.9324     0.4155   9.464 1.49e-12 ***

(Dispersion parameter for gaussian family taken to be 236.5317)

Null deviance: 32539  on 49  degrees of freedom
Residual deviance: 11354  on 48  degrees of freedom
AIC: 419.16

Number of Fisher Scoring iterations: 2


Note that the $236.5317$ in the output is simply $11354/48$ (up to rounding error, since the actual residual deviation is closer to 11353.52)

LR test statistic = $\frac{(32539-11353.54)}{236.5317} = 89.565$

t-test statistic = $9.464$

$9.464^2 = 89.567$ (difference in last figure is simply rounding error in both calculations)

So the difference is basically the same as the difference between looking up 9.464 in $t_{48}$ tables and in normal tables.

• Thanks Glen. An very thorough answer. +1. I have one follow up question. I am actually using this in regard to spatial auto regressive parameters, but feared no one would respond if i was more specific. Do you think that we could make the same conclusions? That the test statistic for an auto correlation parameter will be approximately the same as a LLR test statistic between a model with and without the parameter? Thanks again Commented Jan 30, 2014 at 9:37
• I guess it depends on which test statistic you mean. For example, the z-test for an autocorrelation will likely be almost identical to the LR test (though I haven't worked out the LR test, I think from the form of the likelihood for an AR(p) (which apart from a term involving the first p observations, looks the same as a regression) I think it should work out. Commented Jan 30, 2014 at 9:46