Interpreting when a regression coefficient is significant

Question

Consider the following regression model:

$y_i=\beta_1+\beta_2x_{i,2}+\beta_3x_{i,3}+\beta_4x_{i,2}x_{i,3}+\epsilon_i,$

where $\epsilon_i\sim N(0,\sigma^2).$ Here, $x_2$ is binary variable

$$X_2 = \begin{cases} 0, & \text{if method 1} \\ 1, & \text{if method 2} \end{cases}$$

and $x_3$ is continuous variable.

If I find $\beta_2$ is significant what does it mean? Does it mean $X_2$ variable is significant? what does it mean by $X_2$ variable is significant? is it there is evidence of association between $Y$ and $X_2$? How is to know whether there is evidence of difference between two methods?

Also, what is the interpretation when I find $\beta_4$ significant?

Answer 1

Without an interaction (product) term X2X3, the p-value for β2 provides a test of the null hypothesis that the adjusted mean difference between the two groups (controlling for X3) is zero in the population. In other words, a small p-value would indicate that the groups differ in their (adjusted) means (method 2 - method 1), controlling for X3 (this is equivalent to an analysis of covariance).

With the interaction (product) term X2X3 included, the interpretation is different. In that case, β2 gives the mean difference (method 2 - method 1) when X3 = 0 (which may or may not be meaningful, depending on whether X3 has a meaningful zero point; centering X3 prior to the analysis could help with the interpretation if raw X3 does not have a meaningful zero point). As a consequence, the p-value for β2 with interaction term provides a test of whether the groups differ in their means at the point X3 = 0.

See Chapters 8 and 9 in:

Cohen, J., Cohen, P., West, S. G. & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Erlbaum.

Answer 2

First of all “significance” means very little and is entirely arbitrary. So interpret regression coefficients whether or not they are “significant”. To say that $\beta_{i} \neq 0$ is to say that $X_i$ is associated with $Y$ after accounting for the effects of the other $X$s if linearity, additivity, and possibly other model assumptions hold. One might also say that a nonzero coefficient implies that the corresponding $X$ predicts $Y$ to some extent, and that this prediction is different from what one would get had the model not included this $X$. Note that that language is unclear about true vs. estimated effects. An intro to regression may be found here.