Skip to main content
31 votes

AIC versus Likelihood Ratio Test in Model Variable Selection

AIC and likelihood ratio test (LRT) have different purposes. AIC tells you whether it pays to have a richer model when your goal is to approximate the underlying data generating process the best you ...
Richard Hardy's user avatar
20 votes
Accepted

Is the exact value of any likelihood meaningless?

It's “meaningless” in the sense that it's very hard to interpret, it's just “the bigger, the better”. That is the case because the likelihood is not probability and it is calculated without ...
Tim's user avatar
  • 138k
17 votes

Likelihood ratio vs. score vs. Wald test: Different p values, which to use?

First, I disagree somewhat with jsakaluk's answer that the two tests are testing different things - they are both testing whether the coefficient in the larger model is zero. They are just testing ...
Jonathan Bartlett's user avatar
17 votes
Accepted

What happens to the likelihood ratio as more and more data is gathered?

If one takes the logarithm of this product, $${\mathfrak{r}}=\log \prod_{i=1}^n \frac{f(x_i)}{g(x_i)} = \sum_{i=1}^n \log\frac{f(x_i)}{g(x_i)}$$and turns it into an average $$\bar{\mathfrak{r}}_n=\...
Xi'an's user avatar
  • 106k
16 votes

Neyman-Pearson lemma

I recently wrote an entry in a linkedin blog stating Neyman Pearson lemma in plain words and providing an example. I found the example eye opening in the sense of providing a clear intuition on the ...
Ignasi's user avatar
  • 161
14 votes
Accepted

What is a *likelihood ratio test* for a specific distribution, and how does it relate to hypothesis tests?

A likelihood ratio test is just a particular type of hypothesis test where the test statistic is obtained in a specific way. They arise out of Neyman and Pearson's attempt to find a way to obtain &...
Glen_b's user avatar
  • 283k
13 votes
Accepted

Bayes optimal classifier vs Likelihood Ratio

They are not the same, but in you case they could be used for the same purpose. Optimal Bayes classifier is $$ \DeclareMathOperator*{\argmax}{arg\,max} \argmax_{c \in C} p(c|X) $$ i.e., among all ...
Tim's user avatar
  • 138k
12 votes

Why is the chi-square test more popular than the G-test?

The Pearson test is popular because it's simple to compute - it's amenable to hand-calculation even without a calculator (or historically, even without log-tables) - and yet generally has good power ...
Glen_b's user avatar
  • 283k
11 votes

Neyman-Pearson lemma

The Context (In this section I'm just going to explain hypothesis testing, type one and two errors, etc, in my own style. If you're comfortable with this material, skip to the next section) The Neyman-...
Jack M's user avatar
  • 439
10 votes

Why is a likelihood-ratio test distributed chi-squared?

As other commentators have pointed out, Wilks' theorem (Wilks 1938) only shows that, under various regularity conditions, this statistic is asymptotically chi-squared distributed. The asymptotic ...
Ben's user avatar
  • 125k
10 votes
Accepted

LRT comparing a random effects model and nested logistic regression model

Yes, they are nested: the mixed model reduces to the simpler model if $\sigma^2_1=\sigma^2_{x_2}=0$. (This is the same as $G=0$, because the covariances must be zero if the variances are, but stating ...
Ben Bolker's user avatar
  • 43.7k
9 votes

Is the exact value of any likelihood meaningless?

When we use likelihood then we are comparing probability (density) of the data given a certain hypothesis/theory. The actual probability is not important. It can actually become extremely small. ...
Sextus Empiricus's user avatar
8 votes
Accepted

Difference between pointwise mutual information and log likelihood ratio

They're very closely related. "Log-likelihood" is not a very useful name, because it sounds like it's going to be a log probability. But it's not that at all. "Log likelihood ratio"...
senderle's user avatar
  • 306
8 votes
Accepted

Why use the Wald test in logistic regression?

In logistic regression (& other generalized linear models with canonical link functions), the coefficient estimates $\hat\theta$ are arrived at by Fisher Scoring: iterating $$\vec\theta_{k+1} = \...
Scortchi - Reinstate Monica's user avatar
8 votes
Accepted

GLM tests involving deviance and likelihood ratios

The confusion probably comes from the fact that there are three models involved, and the term "deviance" refers to twice the log or the likelihood ratio between two of them. The models are: ...
Christian Hennig's user avatar
7 votes
Accepted

Likelihood Ratio Test statistic for the exponential distribution

This is one of the cases that an exact test may be obtained and hence there is no reason to appeal to the asymptotic distribution of the LRT. To see this, begin by writing down the definition of an ...
JohnK's user avatar
  • 20.4k
7 votes

Does likelihood ratio test control for overfitting?

Your reasoning is too pessimistic. Given the $K$ additional features, the LR test statistic will follow an asymptotic $\chi^2$ distribution with $K$ degrees of freedom if the null is true (and other ...
Christoph Hanck's user avatar
7 votes

Comparing models using the deviance and log-likelihood ratio tests

The residual deviance is twice the difference between the likelihood in the log scale of the saturated model and that of your proposed model: $$ResidualDeviance=2\times(ll(SaturatedModel)-ll(Proposed ...
一个锅's user avatar
7 votes

AIC and BIC criterion for Model selection, how is it used in this paper?

In my answer here I show that in a case like the present one, in which we test nested models against each other, the minimum AIC rule selects the larger model (i.e., rejects the null) if the ...
Christoph Hanck's user avatar
7 votes

Is the exact value of any likelihood meaningless?

The likelihood function is usually taken to be the PDF viewed as as a function of parameters for known data. For example, if I have a coin with Heads probability $\theta$ and toss it $n = 10$ times, ...
BruceET's user avatar
  • 56.4k
7 votes
Accepted

Likelihood ratio test vs p value for Poisson regression

Assuming that you're using glm(), which is the standard/most-used tool for fitting Poisson regressions in R: the $p$-values in the table printed by ...
Ben Bolker's user avatar
  • 43.7k
6 votes
Accepted

Likelihood ratio test confusion

As the likelihood ratio is a function of the data $x$, it is a statistic. You calculate the statistic and test a hypothesis under $H_0$, so here we assume $p_0$ is the true value of the parameter. ...
Dilly Minch's user avatar
6 votes

Compare two Weibull distributions

My question is, what is the null hypothesis in this likelihood ratio test? Under the null for that particular test as described, all the parameters are the same (which is why you were fitting a ...
Glen_b's user avatar
  • 283k
6 votes
Accepted

Likelihood modification in Metropolis Hastings ratio for transformed parameter

You should notice that what you denote $p(y|f(\theta))$ is actually the same as $p(y|\theta)$ [if you overlook the terrible abuse of notations]. As you mention, changing the parameterisation does not ...
Xi'an's user avatar
  • 106k
6 votes
Accepted

Likelihood ratio test for mixed effects model

My first hint is to use lmer from the lme4 package, not lme. When the variance components ...
Robert Long's user avatar
  • 60.9k
6 votes

Applying Wilks' theorem to uniform distribution

From the Wikipedia page for Wilks' theorem: The theorem no longer applies when any one of the estimated parameters is at its upper or lower limit: Wilks’ theorem assumes that the ‘true’ but unknown ...
Dan Phillips's user avatar
6 votes
Accepted

Does ( P(B|A) - P(B|~A) ) / P(B|A) have a name?

At least in epidemiology, the term is Relative Risk reduction: the relative risk reduction (RRR) or efficacy is the relative decrease in the risk of an adverse event in the exposed group compared to ...
seanv507's user avatar
  • 6,777
6 votes
Accepted

Likelihood Ratio Test Equivalent with $t$ test: Difference of Two Means from Constant Variance Normal Distributions

Under $H_0$, the MLEs are: $$\hat{\mu}=\frac{n_1\bar{X}+n_2\bar{Y}}{n}, \qquad \hat{\sigma}^2_0=\frac{1}{n}\left( \sum_{i=1}^{n_1}{(X_i-\hat{\mu})^2} + \sum_{i=1}^{n_2}{(Y_i-\hat{\mu})^2} \right)$$ ...
Spätzle's user avatar
  • 3,930
6 votes
Accepted

Likelihood ratio test for random intercept (MATLAB)

Is there a way of generating a LinearMixedModel object which does not include any random effects? I am not a MATLAB expert, and programming questions are off-topic here anyway (so you might try ...
Robert Long's user avatar
  • 60.9k
6 votes
Accepted

Compare linear regression slopes between non-nested models with differing dataset sizes

Welcome here. A simple solution is to define a new binary variables which takes the value $1$ for the subset of the data and $0$ for all other observations not included in the subset. Then you can ...
Arne Jonas Warnke's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible