What do the terms dispersion parameter - deviance - and variance of $y_i$ mean? I am studying GLMs and I am struggling a bit with some of the concepts. I think mine is more of a theoretical issue.
Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance.
I know they probably have different meanings, but I thought the would be estimated in the same way, I thought they measure the same thing, but now i am wondering what actually each of them measures.
I know that the variance of $y_i=a(\phi)* V(\mu)$ $\rightarrow$ the product of the dispersion parameter and the variance function. 
But then I get lost when it's about estimating $\phi$ (the dispersion parameter).
I also know that the concept of deviance, $D$, is related to $\phi$, because $D$ converges in distribution to $\phi*\chi^2$ (with $n-p$ degrees of freedom).
Would someone please specify the meaning of the three terms? :)
 A: An exponential family has a density of the form 
$$
f(y\vert \mu, \phi) = h(y,\phi)\exp
   \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right)
$$
Above, $\phi$ is the dispersion parameter. Typically you look at models of the form 
$$
f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp
   \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right)
$$
where $\mu_i = \vec{x}^\top_i\vec{\beta}$ where $\vec{x}_i$ are your covariates and $\vec{\beta}$ are slopes. Thus, all observations share the dispersion parameter. Now to 

Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance.
...
Would someone please specify the meaning of the three terms? :)

then the dispersion parameter, $\phi$, is used as the variance is 
$$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), 
    \qquad V(\mu)=A''(\mu_i)$$
so the dispersion parameter is a parameter used to scale the variance function $V$. Notice that it is used to scale the observations', $i = 1,\dots,n$, variances by the same factor $d(\phi)$. 
The variance of $y_i$ is  $\text{Var}(y_i\vert\mu_i;\phi)$. It is the variance you expect given a $\mu_i$ and dispersion parameter $\phi$.
The deviance, as Carl writes, is a generalization of the idea of using the sum of squares of residuals. The deviance is given by
$$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$
were $\tilde\mu_i$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $b(\mu) = \mu$ and $T(y) = y$. In this case 
$$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$
A linear model has 
$$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$
so 
$$\begin{split}
D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\
  &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\
  &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2
\end{split}$$
which is the usual residual sum of squares.
A: Statistical dispersion is a generic term that includes variance, standard deviation, and interquartile range. Statistical deviance could mean a criterion for "abnormal" occurrence or behaviour based on an infrequency of occurrence in members of a population (talking to yourself happens infrequently etc.) However, as pointed out below by @EdM statistical deviance has a different definition as well. In statistics, deviance is a quality-of-fit statistic for a model that is often used for statistical hypothesis testing. It is a generalization of the idea of using the sum of squares of residuals in ordinary least squares to cases where model-fitting is achieved by maximum likelihood.
