2
$\begingroup$

I am studying GLMs and I am struggling a bit with some of the concepts. I think mine is more of a theoretical issue.

Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance.

I know they probably have different meanings, but I thought the would be estimated in the same way, I thought they measure the same thing, but now i am wondering what actually each of them measures.

I know that the variance of $y_i=a(\phi)* V(\mu)$ $\rightarrow$ the product of the dispersion parameter and the variance function.

But then I get lost when it's about estimating $\phi$ (the dispersion parameter).

I also know that the concept of deviance, $D$, is related to $\phi$, because $D$ converges in distribution to $\phi*\chi^2$ (with $n-p$ degrees of freedom).

Would someone please specify the meaning of the three terms? :)

$\endgroup$
  • $\begingroup$ which three terms? It would also help if linked to a source with the same notation as your post $\endgroup$ – Taylor Jan 24 '17 at 3:57
1
$\begingroup$

An exponential family has a density of the form

$$ f(y\vert \mu, \phi) = h(y,\phi)\exp \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right) $$

Above, $\phi$ is the dispersion parameter. Typically you look at models of the form

$$ f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right) $$

where $\mu_i = \vec{x}^\top_i\vec{\beta}$ where $\vec{x}_i$ are your covariates and $\vec{\beta}$ are slopes. Thus, all observations share the dispersion parameter. Now to

Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance.

...

Would someone please specify the meaning of the three terms? :)

then the dispersion parameter, $\phi$, is used as the variance is

$$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), \qquad V(\mu)=A''(\mu_i)$$

so the dispersion parameter is a parameter used to scale the variance function $V$. Notice that it is used to scale the observations', $i = 1,\dots,n$, variances by the same factor $d(\phi)$.

The variance of $y_i$ is $\text{Var}(y_i\vert\mu_i;\phi)$. It is the variance you expect given a $\mu_i$ and dispersion parameter $\phi$.

The deviance, as Carl writes, is a generalization of the idea of using the sum of squares of residuals. The deviance is given by

$$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$

were $\tilde\mu_i$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $b(\mu) = \mu$ and $T(y) = y$. In this case

$$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$

A linear model has

$$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$

so

$$\begin{split} D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\ &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\ &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2 \end{split}$$

which is the usual residual sum of squares.

$\endgroup$
0
$\begingroup$

Statistical dispersion is a generic term that includes variance, standard deviation, and interquartile range. Statistical deviance could mean a criterion for "abnormal" occurrence or behaviour based on an infrequency of occurrence in members of a population (talking to yourself happens infrequently etc.) However, as pointed out below by @EdM statistical deviance has a different definition as well. In statistics, deviance is a quality-of-fit statistic for a model that is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals in ordinary least squares to cases where model-fitting is achieved by maximum likelihood.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.