The response variable $Y_i$ is normally distributed,has a mean of $\mu_i$ and a variance of $\sigma^2$ in an example I am given of a linear model. Hence $Y_i ∼ N(\mu_i,\sigma^2)$. I do not understand why $Y_i$ does not have a variance of $\sigma_i^2$ instead of $\sigma^2$. Can someone explain please?

  • 2
    $\begingroup$ The lack of $i$ subscript on $\sigma^2$ in the notation means that the variance is assumed constant ... the same for every $Y_i$. Various models (the usual forms of anova, regression, additive models, for example) use such an assumption. $\endgroup$ – Glen_b Apr 20 '15 at 0:06

All this says is that we are assuming that all of the observations have the same variance.

This is a general notation for something being shared over a range of subscripts. Because you mention a linear model I'll give some details regarding the linear model formulation, but, as Tim mentioned in his comment, the idea that a parameter is shared if it doesn't have a subscript is far more general than just the linear model.

Recall that the model for OLS (ordinary least squares) is typically written as $$ Y = X \beta + \varepsilon $$ where $\varepsilon \sim N(0, \sigma^2 I_n)$, meaning that the errors are all independent and identically distributed Normal random variables with mean 0 and shared variance $\sigma^2$.

This means that $E(Y) = X\beta$ which can be written as $E(Y_i) = \mu_i$ for the appropriate $\mu_i$ (specifically $X_i^T\beta := \mu_i$). Similarly $Var(Y) = Var(\varepsilon) = \sigma^2 I_n \implies Var(Y_i) = \sigma^2$ and all $Y_i$ are independent. The point of this is that the standard OLS model is exactly what you have when, for each $i$, $Y_i \sim N(\mu_i, \sigma^2)$ with independence.

We certainly can do linear regression where $Var(Y_i) = \sigma^2_i$, but (still assuming independence) this would be weighted least squares, not OLS. What changes is that we now have $\varepsilon \sim N(0, \Sigma)$ where $\Sigma$ is a diagonal matrix with the $i$th element being $\sigma_i^2$.

  • 1
    $\begingroup$ I would make it more clear in your answer that the OLS example is just an example, while this notation could be used in many different contexts. $\endgroup$ – Tim Apr 20 '15 at 5:12
  • $\begingroup$ Good point. I've edited to hopefully make that clear. $\endgroup$ – jld Apr 20 '15 at 13:32
  • $\begingroup$ This answer doesn't mention that we are talking about normal (Gaussian) distributions. Best not to assume that to be evident! $\endgroup$ – Nick Cox Apr 20 '15 at 15:31
  • $\begingroup$ @NickCox I don't follow what you mean. I mention that the model is $Y = X \beta + \varepsilon$ with $\varepsilon \sim N(0, \sigma^2 I_n)$. $\endgroup$ – jld Apr 20 '15 at 17:28
  • $\begingroup$ You're assuming that the notation $N(,)$ is understood. Whether the OP understands this (I guess so), the thread here is likely to be consulted by people who don't know that. So, for completeness, and for fuller comprehensibility, you should spell it out. $\endgroup$ – Nick Cox Apr 20 '15 at 17:39

I do not understand why $Y_i$ does not have a variance of $\sigma_i^2$ instead of $\sigma^2$.

It's a convenient assumption used in most introductory texts. You don't want to deal with all issues at once when teaching a new subject. So, they start with this assumption to streamline the exposition.

In real application you may want to loosen it up to various degrees. For instance, in physics and engineering experimental data the constant variance is often a reasonable assumption and holds very well empirically. In economics applications you may need to account for changing variance, which is called heteroscedasticity. It's an involved topic and there are many ways of dealing with this conditions, such as GLS.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.