# General Linear Models, categorical data, intercept and reference class

We were studying linear regression:

$$Y = \beta_0 + \sum_i \beta_ix_i + \varepsilon.$$

We were then doing hypothesis tests: $$H_0\text{:}\; C\beta=0 \\H_1\text{:}\;C\beta \neq0.$$

Under the assumptions that:

• Data follows GLM $$\mathbf{Y} = \mathbf{\tilde{X}}\beta + \varepsilon$$, with rank$$\left(\mathbf{\tilde{X}}\right)=p+1$$.
• $$\mathbb{E}\left[\varepsilon\right]=\mathbf{0}$$, and $$\text{Cov}\left[\varepsilon\right]=\sigma^2\text{Id}$$ with $$\sigma^2 < \infty.$$
• $$\varepsilon \sim N(\mathbf{0}, \sigma^2\text{Id}).$$

Then, we separate the data into $$g$$ categories. We then introduce $$g-1$$ dummy variables (to avoid linear dependence on the design matrix), so that now the model is:

$$Y = \beta_0 + \sum_i \beta_ix_i + \sum_{j}^{g-1}\gamma_jx_{\text{type j}} + \varepsilon.$$

And our $$C$$ matrix would then be of dimension $$g-1 \times (p+1+g-1)$$, with a $$g-1 \times g-1$$ identity matrix on the right and zeroes everywhere else (under $$H_0$$, all $$\gamma_j$$ are zero). Essentially, we are testing whether or not the classes contribute any effect to $$Y$$.

However, since we only introduced $$g-1$$ dummy variables, we need to leave one out, the reference class. When we run the regression, we get back some coefficients. In this case we are interested in the intercept ($$\beta_0$$) and the coefficients of $$\gamma_j$$-s. We then compare $$\gamma_j$$ against the intercept, whether or not the value of $$\gamma_j$$ is large enough compared to the intercept.

I don't understand the last part, why are we comparing against the intercept? How does the intercept capture any information about the reference class?

As for my background, I'm doing my masters in math, with a pure math bachelors, so hit me.

• As a heads up, GLM typically refers to generalized linear models, which are different from the general linear model. In fact, once you get into more advanced linear regression modeling, you will start to call it all linear regression or linear modeling.
– Dave
Commented Dec 19, 2022 at 4:48

When all of the “other” groups have their indicator variables set to zero, only the intercept contributes to the outcome ($$Y$$). Since this observation does not correspond to any of the “other” groups, it must correspond to the group that was omitted. In that regard, the intercept represents something related to the group that was omitted from the indicator variables.