4
$\begingroup$

In a linear model $Y=X\beta + \varepsilon$, one can easily test linear hypotheses of the form $H_0: C\beta = \gamma, $ where $C$ is a matrix and $\gamma$ is a vector with dimension equal to the number of rows in $C$. Namely, one can derive a test statistic which has an F distribution under the null and go from there.

Theoretically, these tests are very interesting to me and seem quite flexible, as $C$ and $\gamma$ can be anything.

However, I'm interested to know how useful these hypotheses are in practical applications and what are some interesting examples of these applications? (besides testing if a single coefficient is 0 or all coefficients of the model being zero, which is included in every lm call in R for example)

$\endgroup$
  • $\begingroup$ $\gamma$ is the vector with dimension equal to the number of rows in matrix C. (not size of $\beta$) $\endgroup$ – user158565 Oct 29 '18 at 23:24
  • $\begingroup$ @a_statistician Yes, sorry, I'll edit that. $\endgroup$ – Blaza Oct 29 '18 at 23:26
5
$\begingroup$

These linear hypotheses on the coefficient vector have three main uses:

  • Testing the existence of relationships: We can test the existence of relationships between some subset of the explanatory variables and the response variable. To do this, let $\mathbf{e}_\mathcal{S}$ denote the indicator vector for the subset $\mathcal{S}$ and test the linear hypotheses:

$$H_0: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} = 0 \quad \quad \quad H_A: \mathbf{e}_\mathcal{S} \boldsymbol{\beta} \neq 0.$$

  • Testing a specified magnitude for the relationship: We can test the magnitude of a relationship between an explanatory variables and the response variable using some specified value of interest. This is often useful when a particular specified magnitude has some practical significance (e.g., it is often useful to test if the true coefficient is equal to one). To test $\beta_k = b$ we use the linear hypotheses:

$$H_0: \mathbf{e}_k \boldsymbol{\beta} = b \quad \quad \quad H_A: \mathbf{e}_k \boldsymbol{\beta} \neq b.$$

  • Testing the expected responses of new explanatory variables: We can test the values expected values of responses corresponding to a new set of explanatory variables. Taking new explanatory data $\boldsymbol{X}_\text{new}$ we get corresponding expected values $\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{X}_\text{new} \boldsymbol{\beta}$. This means that we can test the hypothesis $\mathbb{E}(\boldsymbol{Y}_\text{new}) = \boldsymbol{y} $ via the hypotheses:

$$H_0: \boldsymbol{X}_\text{new} \boldsymbol{\beta} = \boldsymbol{y} \quad \quad \quad H_A: \boldsymbol{X}_\text{new} \boldsymbol{\beta} \neq \boldsymbol{y}.$$

As you can see, the first use is to test for whether some of the coefficients are zero, which is a test of whether those explanatory variables are related to the response in the model. However, you can also undertake more general tests of a specific magnitude for the relationship. You can also use the linear test to test the expected response for new data.

$\endgroup$
4
$\begingroup$

When you fit a linear model, the statistical softwares give you the point estimate, confidence interval, test statistics, and p-values of the $\beta$_s. If you are just interesting in these $\beta$s, you can stop here (for example, the simple linear regression just have on intercept and one slope, so $\beta$s themselves are enough). But for little complicated model, you will not satisfied by $\beta$s, and you want to estimate, test the linear combinations of $\beta$s. At this time The importance of C matrix is obvious. For complicated model, such as model with interactions, C matrix must needs be constructed.

Example 1: For ANOVA, one categorical covariate has 3 level. Suppose level 1 is reference. Two $\beta$ will give you the difference between level 2 vs 1 and level 3 vs 1. If you want the difference between level 2 and 3, you need C matrix (0 1 -1). (the first 0 is for intercept). If you want to estimate the means for level 3, C=(1 0 1) is needed.

Example 2: If you want to the multiple hypothesis simultaneously, see Testing the general linear hypothesis: $H_0: \beta_1 = \beta_2 = \beta_3 = \beta_4 = \beta$. Here T=C.

Example 3: If the interactions exist, we need to have the linear relation for each combination (cell) of interaction. Here is 16x16 C matrix to get 8 intercepts and slopes. How to understand the coefficients of a three-way interaction in a regression?

You can find more example on the internet, textbooks.

In summary, for linear model, constructing C matrix is equal to half of theory of linear model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.