Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

 2 added 4 characters in body edited Jul 14 '18 at 9:03 Benjamin Christoffersen 1,66166 silver badges2424 bronze badges An exponential family has a density of the form $$f(y\vert \mu, \phi) = h(y,\phi)\exp \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right)$$ Above, $$\phi$$ is the dispersion parameter. Typically you look at models of the form $$f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right)$$ where $$\mu_i = \vec{x}^\top_i\vec{\beta}$$ where $$\vec{x}_i$$ are your covariates and $$\vec{\beta}$$ are slopes. Thus, all observations share the dispersion parameter. Now to Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance. ... Would someone please specify the meaning of the three terms? :) then the dispersion parameter, $$\phi$$, is used as the variance is $$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), \qquad V(\mu)=A''(\mu_i)$$ so the dispersion parameter is a parameter used to scale the variance function $$V$$. Notice that it is used to scale the observations', $$i = 1,\dots,n$$, variances by the same factor $$d(\phi)$$. The variance of $$y_i$$ is $$\text{Var}(y_i\vert\mu_i;\phi)$$. It is the variance you expect given a $$\mu_i$$ and dispersion parameter $$\phi$$. The deviance of $$y_i$$deviance, as Carl writes, is a generalization of the idea of using the sum of squares of residuals. The deviance is given by $$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$ were $$\tilde\mu_i$$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $$b(\mu) = \mu$$ and $$T(y) = y$$. In this case $$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$ A linear model has $$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$ so $$\begin{split} D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\ &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\ &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2 \end{split}$$ which is the usual residual sum of squares. An exponential family has a density of the form $$f(y\vert \mu, \phi) = h(y,\phi)\exp \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right)$$ Above, $$\phi$$ is the dispersion parameter. Typically you look at models of the form $$f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right)$$ where $$\mu_i = \vec{x}^\top_i\vec{\beta}$$ where $$\vec{x}_i$$ are your covariates and $$\vec{\beta}$$ are slopes. Thus, all observations share the dispersion parameter. Now to Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance. ... Would someone please specify the meaning of the three terms? :) then the dispersion parameter, $$\phi$$, is used as the variance is $$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), \qquad V(\mu)=A''(\mu_i)$$ so the dispersion parameter is a parameter used to scale the variance function $$V$$. Notice that it is used to scale the observations', $$i = 1,\dots,n$$, variances by the same factor $$d(\phi)$$. The variance of $$y_i$$ is $$\text{Var}(y_i\vert\mu_i;\phi)$$. It is the variance you expect given a $$\mu_i$$ and dispersion parameter $$\phi$$. The deviance of $$y_i$$ as Carl writes is a generalization of the idea of using the sum of squares of residuals. The deviance is given by $$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$ were $$\tilde\mu_i$$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $$b(\mu) = \mu$$ and $$T(y) = y$$. In this case $$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$ A linear model has $$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$ so $$\begin{split} D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\ &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\ &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2 \end{split}$$ which is the usual residual sum of squares. An exponential family has a density of the form $$f(y\vert \mu, \phi) = h(y,\phi)\exp \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right)$$ Above, $$\phi$$ is the dispersion parameter. Typically you look at models of the form $$f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right)$$ where $$\mu_i = \vec{x}^\top_i\vec{\beta}$$ where $$\vec{x}_i$$ are your covariates and $$\vec{\beta}$$ are slopes. Thus, all observations share the dispersion parameter. Now to Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance. ... Would someone please specify the meaning of the three terms? :) then the dispersion parameter, $$\phi$$, is used as the variance is $$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), \qquad V(\mu)=A''(\mu_i)$$ so the dispersion parameter is a parameter used to scale the variance function $$V$$. Notice that it is used to scale the observations', $$i = 1,\dots,n$$, variances by the same factor $$d(\phi)$$. The variance of $$y_i$$ is $$\text{Var}(y_i\vert\mu_i;\phi)$$. It is the variance you expect given a $$\mu_i$$ and dispersion parameter $$\phi$$. The deviance, as Carl writes, is a generalization of the idea of using the sum of squares of residuals. The deviance is given by $$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$ were $$\tilde\mu_i$$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $$b(\mu) = \mu$$ and $$T(y) = y$$. In this case $$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$ A linear model has $$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$ so $$\begin{split} D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\ &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\ &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2 \end{split}$$ which is the usual residual sum of squares. 1 answered Jul 14 '18 at 8:50 Benjamin Christoffersen 1,66166 silver badges2424 bronze badges An exponential family has a density of the form $$f(y\vert \mu, \phi) = h(y,\phi)\exp \left(\frac{b(\mu)T(y) - A(\mu)}{d(\phi)}\right)$$ Above, $$\phi$$ is the dispersion parameter. Typically you look at models of the form $$f(y_i\vert \mu_i, \phi) = h(y_i,\phi)\exp \left(\frac{b(\mu_i)T(y_i) - A(\mu_i)}{d(\phi)}\right)$$ where $$\mu_i = \vec{x}^\top_i\vec{\beta}$$ where $$\vec{x}_i$$ are your covariates and $$\vec{\beta}$$ are slopes. Thus, all observations share the dispersion parameter. Now to Basically I am a bit confused by the meaning of these three concepts: the variance of y (the response), the dispersion parameter and the deviance. ... Would someone please specify the meaning of the three terms? :) then the dispersion parameter, $$\phi$$, is used as the variance is $$\text{Var}(y_i\vert\mu_i,\phi)=A''(\mu_i)d(\phi) = V(\mu_i)d(\phi), \qquad V(\mu)=A''(\mu_i)$$ so the dispersion parameter is a parameter used to scale the variance function $$V$$. Notice that it is used to scale the observations', $$i = 1,\dots,n$$, variances by the same factor $$d(\phi)$$. The variance of $$y_i$$ is $$\text{Var}(y_i\vert\mu_i;\phi)$$. It is the variance you expect given a $$\mu_i$$ and dispersion parameter $$\phi$$. The deviance of $$y_i$$ as Carl writes is a generalization of the idea of using the sum of squares of residuals. The deviance is given by $$D = 2\phi\sum_{i=1}^n\left(\log f(y_i\vert \tilde\mu_i) - \log f(y_i\vert \mu_i) \right)$$ were $$\tilde\mu_i$$ are the linear predictors in a saturated model. To see that this is a generalization of the sum of squares of residuals, we focus on the distribution of canonical form where $$b(\mu) = \mu$$ and $$T(y) = y$$. In this case $$D = 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right)$$ A linear model has $$A(\mu) = \mu^2/2,\quad \tilde\mu_i = y_i$$ so $$\begin{split} D &= 2\sum_{i=1}^n\left(y_i(\tilde\mu_i - \mu_i) - A(\tilde\mu_i) + A(\mu_i)\right) \\ &= 2\sum_{i=1}^n\left(y_i(y_i - \mu_i) - \frac{y_i^2}{2} + \frac{\mu_i^2}{2}\right) \\ &= \sum_{i=1}^n\left(y_i - \mu_i\right)^2 \end{split}$$ which is the usual residual sum of squares.