I'm trying to understand my lecture notes but am a bit stuck on the concept of identifiability. In one-way ANOVA, could someone please explain the reason for the constraint $\sum_{i=1}^{m} \beta_{j} = 0$ where we have m groups of observations, each group consisting of k observations with $Y_{ij}$ as the jth observation from the ith group, $E(Y_{ij}) = \mu + \beta_{i}, i = 1,...,m; j = 1,...,k, \text{Var}(Y_{ij}) = \sigma^{2}$and$ H_{0} : \beta_{1} = \beta_{2} = ... = \beta_{m}$? I don't quite get the identifiability reason.

  • 1
    $\begingroup$ Imagine someone asks you "Which two numbers add to 11?" $\endgroup$ – Glen_b Jan 24 '17 at 7:43
  • $\begingroup$ @Glen_b I get the idea but do you mind elaborating in the context of ANOVA? $\endgroup$ – python_learner Jan 24 '17 at 17:12

Consider for simplicity that $m=2$ and compare the models

  • $\mu=0,\beta_1=0,\beta_2=2$,

  • $\mu=1,\beta_1=-1,\beta_2=1$,

  • $\mu=2,\beta_1=-2,\beta_2=0$.

These models are all special cases of $(\mu,\beta_1,\beta_2)=(\mu,-\mu,2-\mu)$. You can see that whatever $\mu$ we choose, $\mu+\beta_1=0$ and $\mu+\beta_2=2$, so there's an infinite set of parameter-triples that match $E(Y_{1j})=0$ and $E(Y_{2j})=2$, and no way to distinguish between them.

Consequently, while data will allow you to estimate the two group-means, those two pieces of information (two df) - no matter how precisely estimated - are not going to be enough to estimate the three parameters (three df) in the model -- there's an extra degree of freedom that allows you to move all three parameters in particular ways relative to each other while keeping the group-means the same.

You need to restrict/constrain/regularize the situation in some way so that the model doesn't have more things to estimate than the design has the ability to identify.

| cite | improve this answer | |
  • 1
    $\begingroup$ Ah, this answer gave me a light bulb moment - thank you! $\endgroup$ – python_learner Jan 25 '17 at 2:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.