Interpreting Z-Scores of Linear Regression Coefficients The table below shows the coefficients obtained from Ordinary Least Squares (OLS) Regression. The Z-Score $Z_k$ is defined to have the null hypothesis: a  given coefficient $\beta_k=0$ and $Z_k$ has $t$-distribution:
$$
Z_k=\frac{\beta_k}{\hat \sigma \sqrt{(\mathbf X^\top \mathbf X)^{-1}_{k,k} }}
$$

I am quite new to hypothesis testing and I want to confirm:

*

*A Z-Score $|Z_k|>2$ is said to be at significant at the $5\%$ level. Does this mean that the coefficient $P(\beta_k≠0)=5\%$ ?


*The ESL book stated that "A large (absolute) Z-Score $Z_k$ will lead to the rejection of the null hypothesis $\beta_k=0$". Why? Shouldn't a large Z-Score imply that $P(\beta_k≠0)$ is much higher given that it is the area to the left or right of the boundary formed by $Z_k$?


*The statement of the book on (2) contradicts the findings on the book (1)?
 A: The Z-score is a measure of how extreme the observed regression coefficient is under the hypothetical scenario that the true regression coefficient is equal to 0. A large Z score means that the observed regression coefficient is extreme, and therefore unlikely, in this hypothetical scenario. Getting such an extreme coefficient under this scenario makes one doubt the validity of that scenario. That is hypothesis testing, with this hypothetical scenario often called the "null hypothesis".
How do we decide what Z-score counts as too extreme? Well, under the hypothetical scenario that the true regression coefficient is equal to 0, statisticians have figured out how likely a given Z-score is (using the normal distribution curve). Z-scores greater than 2 (in absolute value) only occur about 5% of the time when the true regression coefficient is equal to 0. If we actually witness an event that occurs only only 5% of the time in some hypothetical scenario, we say that result is incompatible with the assumptions of that scenario.
For example, if you were wondering whether a coin was fair (i.e., 50-50 head/tails) and you flipped it 6 times and got the same face each time, such an event would occur about 3% of the time if the coin was actually fair. Such an usual result under the hypothetical scenario that the coin is fair makes us doubt that the coin is fair, so we reject the assumptions of that scenario and claim the coin must not be fair.
So, observing a Z-score greater than 2 would be rare under the assumption that the true regression coefficient is equal to 0. Therefore, we reject this assumption (the null hypothesis) and claim that the true regression coefficient is different from 0.
With this in mind, let's answer your questions directly.

*

*No. What this means is that a Z-score greater than 2 (in absolute value) occurs less than 5% of the time under the assumption that the true regression coefficient is equal to 0. This would be a very unusual event under this assumption, which makes us doubt the assumption. Observations that make us doubt our assumption are described as "significant". If you wanted to use a different standard for what counts as "too extreme" not to doubt our assumption, e.g., you only doubt the assumption if an event occurs that would only occur 1% of the time if the assumption were true, we would use a different critical Z-score to judge our observed Z-scores by (in the case of 1%, that would be 2.58).


*A large Z score leads to rejection of the null hypothesis because large Z scores are very unusual when the null hypothesis is true, making us doubt the null hypothesis. The area under the normal distribution curve that is more "extreme" than a Z score of 2 is quite small, representing less than 5% of the total area under the curve, indicating a very unusual result under the assumption that the true regression coefficient is equal to 0.


*These statements do not contradict. They are describing the same methodology of hypothesis testing.
