Skip to main content
3 of 3
added 359 characters in body

How to interpret a 1% significant relation when adding a control diminishes the significance level at 5%?

In rough terms, I am given a t-statistic of 3.00 of the coefficient of variable $X$, $\beta_1$, after regressing the independent variable $X$ on the dependent $Y$:

$$ Y = \alpha + \beta_1 X.$$

I have previously formed the null hypothesis that $X$ explains $Y$. The normal interpretation would be:

"At the significance level of 1% I cannot reject the null hypothesis that $X$ explains $Y$".

However, adding some controls in different multiple regressions, say $Z_1$ in one specification and $Z_2$ in another, the t-statistic of $\beta_1$ drops to roughly 2.00, significant at 5% only, for only a couple of the tested control specifications.

What do I say then for interpreting the null hypothesis?

"I cannot reject the null hypothesis that $X$ explains $Y$ at the significant level %1 or 5%?

I understand that the null hypothesis considers one specification only, but what is the common approach?

Added clarification

In a study, consider I form the variables based on answers on questions of a specific group of people. I then run $$Y=\alpha+\beta_1X ~~~~~H_0: \beta_1=0$$ and $$Y=\alpha+\beta_1X+\gamma Z ~~~~~H_0: \beta_1=0$$

Next day, as an added robustness check and assuming nothing should/has change/changed regarding the answers for forming the variables $Y$, $X$ and $Z$, I ask the same group of people the same questions, and I run the same regression on the new dataset. Now, $\beta_1$ of the simple model is still significant at 1%, but it is significant at 5% only for the multiple regression model.

How do I treat this case? In rough terms, the two dataset should give the same results. But now, the new dataset is not exactly replicating the result of the former dataset, in terms of statistical significance. How do I interpret this result in my study as a concluding sentence? I say "I cannot reject the null hypothesis that $X$ explains $Y$ at the significant level %1 or 5%?