8
$\begingroup$

I have a linear regression model for a dependent variable $Y$ based on two independent variables, $X1$ and $X2$, so I have a general form of a regression equation

$Y = A + B_1 \cdot X_1 + B_2 \cdot X_2 + \epsilon$,

where $A$ is the intercept, $\epsilon$ is the error term, and $B_1$ and $B_2$ are the respective coefficients of $X_1$ and $X_2$. I perform a multiple regression with software (statsmodel in Python) and I get coefficients for the model: $A = a, B_1 = b_1, B_2 = b_2$. The model also gives me $p$ values for each coefficient: $p_a$, $p_1$, and $p_2$. My question is: What is the null hypothesis for those individual $p$ values? For example, to obtain $p_1$ I know that the null hypothesis entails a 0 coefficient for $B_1$, but what about the other variables? In other words, If the null hypothesis is $Y = A + 0 \cdot X_1 + B_2 \cdot X_2$, what are the values of $A$ and $B_2$ for the null hypothesis from which the $p$-value for $B_1$ is derived?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Your model is missing an error term. $\endgroup$ – Andreas Dzemski Dec 30 '18 at 19:48
6
$\begingroup$

The null hypothesis is $$ H_0: B1 = 0 \: \text{and} \: B2 \in \mathbb{R} \: \text{and} \: A \in \mathbb{R}, $$ which basically means that the null hypothesis does not restrict B2 and A. The alternative hypothesis is $$ H_1: B1 \neq 0 \: \text{and} \: B2 \in \mathbb{R} \: \text{and} \: A \in \mathbb{R}. $$ In a way, the null hypothesis in the multiple regression model is a composite hypothesis. It is "fortunate" that we can construct a pivotal test statistic that does not depend on the true value of B2 and A, so that we do not suffer a penalty from testing a composite null hypothesis.

In other words, there are a lot of different distributions of $(Y, X1, X2)$ that are compatible with the null hypothesis $H_0$. However, all of these distributions lead to the same behavior of the the test statistic that is used to test $H_0$.

In my answer, I have not addressed the distribution of $\epsilon$ and implicitly assumed that it is an independent centered normal random variable. If we only assume something like $$ E[\epsilon \mid X1, X2] = 0 $$ then a similar conclusion holds asymptotically (under regularity assumptions).

Improve this answer
$\endgroup$
10
  • $\begingroup$ But as I understand it, doesn't the null hypothesis have to be a probability distribution? If I have specific values for the coefficients, I can generate a probability distribution by adding noise (epsilon) to the regression equation. But if I don't have specific values for coefficients, how would I generate the null probability distribution? $\endgroup$ – tmldwn Dec 30 '18 at 20:35
  • $\begingroup$ A composite null hypothesis is a whole set of possible probability measures. $\endgroup$ – Andreas Dzemski Dec 30 '18 at 20:52
  • $\begingroup$ I have edited my answer to emphasize this point. $\endgroup$ – Andreas Dzemski Dec 30 '18 at 21:28
  • 2
    $\begingroup$ @tmldwn: Here, the marginal distribution of the t-statistic does indeed not depend on where we are in the null. If you find this hard to understand then I suggest you go carefully through the derivation of the distribution of the t-statistic. Note that the t-statistic depends on the LS estimator. In a way this automatically adjusts the test statistic correctly for the "true" hypothesis in the null space (we don't have to take a stand on what A, B2 are because we don't need them to compute the test statistic). $\endgroup$ – Andreas Dzemski Dec 31 '18 at 11:02
  • 1
    $\begingroup$ This answer is completely wrong. As explained in this document, there is anova for the whole regression, but a t-test for each coeffieicnt: reliawiki.org/index.php/… $\endgroup$ – Josh Dec 31 '18 at 15:47
0
$\begingroup$

You can make the same assupmtions for the other variables as the X1. The ANOVA table of the regression gives specific information about each variable significance and the overall significance as well.As far as regression analysis is concerned, the acceptance of null hypothesis implies that the coefficient of the variable is zero, given a certain level of significance.

If you want to acquire a more intuitive aspect of the issue, you can study more about Hypothesis testing.

Improve this answer
$\endgroup$
0
$\begingroup$

The $p$-values are the result of a series of $t$-tests. The null hypothesis is that $B_j=0$, while the alternative hypothesis (again, for each coefficient) is, $B_j\ne0$

(see here for more details: http://reliawiki.org/index.php/Multiple_Linear_Regression_Analysis#Test_on_Individual_Regression_Coefficients_.28t__Test.29)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.