9
$\begingroup$

I am using the glmfit function in MATLAB. The function only returns the deviance and not the log likelihood. I understand that the deviance is basically twice the difference between the log likelihoods of the models but what I don't get is I am only using glmfit to create one model, but somehow I am getting a deviance.

  • Doesn't calculation of the -2 Log likelihood require 2 models?
  • How can the deviance be analysed when there is only one model?

Another question I am having is say I did have two models and that I was comparing them using log likelihood test. The null hypothesis would be the first model and the alternative hypothesis would be the second model. After getting the log likelihood test statistic would I check it against the chi squared cdf to determine the p-value? Am I right that if it is less than the alpha level I would reject the null and if it is greater I would fail to reject the null?

$\endgroup$
2
  • 2
    $\begingroup$ To your first question. Yes there are 2 models. The other one is a perfect model with log likelihood = 0. In this way your deviance is just equal to your model's log likelihood. $\endgroup$
    – FMZ
    Mar 16, 2012 at 1:51
  • 1
    $\begingroup$ would it be perfect model - my model, or my model - perfect model? And would dividing it by -2 really give me the log likelihood of the model and I could use that to do the log likelihood test? $\endgroup$
    – shiu6rewgu
    Mar 16, 2012 at 17:47

1 Answer 1

13
$\begingroup$

The statistical term deviance is thrown around a bit too much. Most of the time, programs return the deviance $$ D(y) = -2 \log{\{p(y | \hat{\theta})\}},$$ where $\hat{\theta}$ is your estimated parameter(s) from model fitting and $y$ is some potentially observed/observable occurrence of the random quantity in question.

The more common deviance that you refer to would treat the deviance above as a function of two variables, both the data and the fitted parameters: $$ D(y,\hat{\theta}) = -2\log{\{p(y|\hat{\theta})\}}$$ and so if you had one $y$ value but two competing, fitted parameter values, $\hat{\theta}_{1}$ and $\hat{\theta}_{2}$, then you'd get the deviance you mentioned from $$-2(\log{\{p(y|\hat{\theta}_{1})\}} - \log{\{p(y|\hat{\theta}_{2})\}}). $$ You can read about the Matlab function that you mentioned, glmfit(), linked here. A more fruitful, though shorter, discussion of the deviance is linked here.

The deviance statistic implicitly assumes two models: the first is your fitted model, returned by glmfit(), call this parameter vector $\hat{\theta}_{1}$. The second is the "full-model" (also called the "saturated model"), which is a model in which there is a free variable for every data point, call this parameter vector $\hat{\theta}_{s}$. Having so many free variables is obviously a stupid thing to do, but it does allow you to fit to that data exactly.

So then, the deviance statistics is computed as the difference between the log likelihood computed at the fitted model and the saturated model. Let $Y=\{y_{1}, y_{2}, \cdots, y_{N}\}$ be the collection of the N data points. Then:

$$DEV(\hat{\theta}_{1},Y) = -2\biggl[\log{p(Y|\hat{\theta}_{1})} - \log{p(Y|\hat{\theta}_{s})} \biggr]. $$ The terms above will be expanded into summations over the individual data points $y_{i}$ by the independence assumption. If you want to use this computation to calculate the log-likelihood of the model, then you'll need to first calculate the log-likelihood of the saturated model. Here is a link that explains some ideas for computing this... but the catch is that in any case, you're going to need to write down a function that computes the log-likelihood for your type of data, and in that case it's probably just better to create your own function that computes the log-likelihood yourself, rather than backtracking it out of a deviance calculation.

See Chapter 6 of Bayesian Data Analysis for some good discussion of deviance.

As for your second point about the likelihood test statistic, yes it sounds like you basically know the right thing to do. But in many cases, you'll consider the null hypothesis to be something that expert, external knowledge lets you guess ahead of time (like some coefficient being equal to zero). It's not necessarily something that comes as the result of doing model fitting.

$\endgroup$
6
  • $\begingroup$ Thank you EMS! You really helped me understand what deviance is a lot! I still have a few questions, but I am not sure how to ask them. Once I figure out how to word it I will definitely reply here. $\endgroup$
    – shiu6rewgu
    Mar 16, 2012 at 17:47
  • $\begingroup$ Ok first question, how would I extract the log likelihood for the model I created from the deviance considering that matlab only gives me the deviance? Also, (I know this make me look fairly stupid but) for p(y |θˆ2) would that be the probability of getting a certain y value from the outcome data set or the independent variables given the fitted parameter $\endgroup$
    – shiu6rewgu
    Mar 16, 2012 at 17:51
  • $\begingroup$ It appears I was mistaken about Matlab's method. It computes deviance by looking at two models, and I have edited the answer above to reflect this. $\endgroup$
    – ely
    Mar 16, 2012 at 23:44
  • $\begingroup$ +1, this is a really nice answer. I hope to see more of them in the future. $\endgroup$ Mar 18, 2012 at 18:17
  • 1
    $\begingroup$ @SibbsGambling In this link there is an example with coolibah tree data showing a "full" or "saturated" model where the log-likelihood is not zero. I believe there are certain situations where the saturated model must have a likelihood of one by definition, but not in all situations. $\endgroup$
    – ely
    Jun 9, 2015 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.