# How useful are linear hypotheses?

In a linear model $$Y=X\beta + \varepsilon$$, one can easily test linear hypotheses of the form $$H_0: C\beta = \gamma,$$ where $$C$$ is a matrix and $$\gamma$$ is a vector with dimension equal to the number of rows in $$C$$. Namely, one can derive a test statistic which has an F distribution under the null and go from there.

Theoretically, these tests are very interesting to me and seem quite flexible, as $$C$$ and $$\gamma$$ can be anything.

However, I'm interested to know how useful these hypotheses are in practical applications and what are some interesting examples of these applications? (besides testing if a single coefficient is 0 or all coefficients of the model being zero, which is included in every lm call in R for example)

• $\gamma$ is the vector with dimension equal to the number of rows in matrix C. (not size of $\beta$) – user158565 Oct 29 '18 at 23:24
• @a_statistician Yes, sorry, I'll edit that. – Blaza Oct 29 '18 at 23:26

These linear hypotheses on the coefficient vector have three main uses:

• Testing the existence of relationships: We can test the existence of relationships between some subset of the explanatory variables and the response variable. To do this, let $$\mathbf{e}_\mathcal{S}$$ denote the indicator vector for the subset $$\mathcal{S}$$ and test the linear hypotheses:

$$H_0: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} = 0 \quad \quad \quad H_A: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} \neq 0.$$

• Testing a specified magnitude for the relationship: We can test the magnitude of a relationship between an explanatory variables and the response variable using some specified value of interest. This is often useful when a particular specified magnitude has some practical significance (e.g., it is often useful to test if the true coefficient is equal to one). To test $$\beta_k = b$$ we use the linear hypotheses:

$$H_0: \mathbf{e}_k \boldsymbol{\beta} = b \quad \quad \quad H_A: \mathbf{e}_k \boldsymbol{\beta} \neq b.$$

• Testing the expected responses of new explanatory variables: We can test the values expected values of responses corresponding to a new set of explanatory variables. Taking new explanatory data $$\boldsymbol{X}_\text{new}$$ we get corresponding expected values $$\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{X}_\text{new} \boldsymbol{\beta}$$. This means that we can test the hypothesis $$\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{y}$$ via the hypotheses:

$$H_0: \boldsymbol{X}_\text{new} \boldsymbol{\beta} = \boldsymbol{y} \quad \quad \quad H_A: \boldsymbol{X}_\text{new} \boldsymbol{\beta} \neq \boldsymbol{y}.$$

As you can see, the first use is to test for whether some of the coefficients are zero, which is a test of whether those explanatory variables are related to the response in the model. However, you can also undertake more general tests of a specific magnitude for the relationship. You can also use the linear test to test the expected response for new data.

When you fit a linear model, the statistical softwares give you the point estimate, confidence interval, test statistics, and p-values of the $$\beta$$_s. If you are just interesting in these $$\beta$$s, you can stop here (for example, the simple linear regression just have on intercept and one slope, so $$\beta$$s themselves are enough). But for little complicated model, you will not satisfied by $$\beta$$s, and you want to estimate, test the linear combinations of $$\beta$$s. At this time The importance of C matrix is obvious. For complicated model, such as model with interactions, C matrix must needs be constructed.

Example 1: For ANOVA, one categorical covariate has 3 level. Suppose level 1 is reference. Two $$\beta$$ will give you the difference between level 2 vs 1 and level 3 vs 1. If you want the difference between level 2 and 3, you need C matrix (0 1 -1). (the first 0 is for intercept). If you want to estimate the means for level 3, C=(1 0 1) is needed.

Example 2: If you want to the multiple hypothesis simultaneously, see Testing the general linear hypothesis: $$H_0: \beta_1 = \beta_2 = \beta_3 = \beta_4 = \beta$$. Here T=C.

Example 3: If the interactions exist, we need to have the linear relation for each combination (cell) of interaction. Here is 16x16 C matrix to get 8 intercepts and slopes. How to understand the coefficients of a three-way interaction in a regression?

You can find more example on the internet, textbooks.

In summary, for linear model, constructing C matrix is equal to half of theory of linear model.