How useful are linear hypotheses?

Question

In a linear model $Y=X\beta + \varepsilon$, one can easily test linear hypotheses of the form $H_0: C\beta = \gamma, $ where $C$ is a matrix and $\gamma$ is a vector with dimension equal to the number of rows in $C$. Namely, one can derive a test statistic which has an F distribution under the null and go from there.

Theoretically, these tests are very interesting to me and seem quite flexible, as $C$ and $\gamma$ can be anything.

However, I'm interested to know how useful these hypotheses are in practical applications and what are some interesting examples of these applications? (besides testing if a single coefficient is 0 or all coefficients of the model being zero, which is included in every lm call in R for example)

Answer 1

These linear hypotheses on the coefficient vector have three main uses:

• Testing the existence of relationships: We can test the existence of relationships between some subset of the explanatory variables and the response variable. To do this, let $\mathbf{e}_\mathcal{S}$ denote the indicator vector for the subset $\mathcal{S}$ and test the linear hypotheses:

$$H_0: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} = 0 \quad \quad \quad H_A: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} \neq 0.$$

• Testing a specified magnitude for the relationship: We can test the magnitude of a relationship between an explanatory variables and the response variable using some specified value of interest. This is often useful when a particular specified magnitude has some practical significance (e.g., it is often useful to test if the true coefficient is equal to one). To test $\beta_k = b$ we use the linear hypotheses:

$$H_0: \mathbf{e}_k \boldsymbol{\beta} = b \quad \quad \quad H_A: \mathbf{e}_k \boldsymbol{\beta} \neq b.$$

• Testing the expected responses of new explanatory variables: We can test the values expected values of responses corresponding to a new set of explanatory variables. Taking new explanatory data $\boldsymbol{X}_\text{new}$ we get corresponding expected values $\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{X}_\text{new} \boldsymbol{\beta}$. This means that we can test the hypothesis $\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{y} $ via the hypotheses:

$$H_0: \boldsymbol{X}_\text{new} \boldsymbol{\beta} = \boldsymbol{y} \quad \quad \quad H_A: \boldsymbol{X}_\text{new} \boldsymbol{\beta} \neq \boldsymbol{y}.$$

As you can see, the first use is to test for whether some of the coefficients are zero, which is a test of whether those explanatory variables are related to the response in the model. However, you can also undertake more general tests of a specific magnitude for the relationship. You can also use the linear test to test the expected response for new data.

Answer 2

When you fit a linear model, the statistical softwares give you the point estimate, confidence interval, test statistics, and p-values of the $\beta$_s. If you are just interesting in these $\beta$s, you can stop here (for example, the simple linear regression just have on intercept and one slope, so $\beta$s themselves are enough). But for little complicated model, you will not satisfied by $\beta$s, and you want to estimate, test the linear combinations of $\beta$s. At this time The importance of C matrix is obvious. For complicated model, such as model with interactions, C matrix must needs be constructed.

Example 1: For ANOVA, one categorical covariate has 3 level. Suppose level 1 is reference. Two $\beta$ will give you the difference between level 2 vs 1 and level 3 vs 1. If you want the difference between level 2 and 3, you need C matrix (0 1 -1). (the first 0 is for intercept). If you want to estimate the means for level 3, C=(1 0 1) is needed.

Example 2: If you want to the multiple hypothesis simultaneously, see Testing the general linear hypothesis: $H_0: \beta_1 = \beta_2 = \beta_3 = \beta_4 = \beta$. Here T=C.

Example 3: If the interactions exist, we need to have the linear relation for each combination (cell) of interaction. Here is 16x16 C matrix to get 8 intercepts and slopes. How to understand the coefficients of a three-way interaction in a regression?

You can find more example on the internet, textbooks.

In summary, for linear model, constructing C matrix is equal to half of theory of linear model.