# Does computing the test statistic for $H_{0}\text{: }\beta = c$, for $c \ne 0$ in a regression require a funky distribution?

Regression coefficients! Everybody loves 'em.[Citation needed]

You know what else everybody loves? Test statistics for regression coefficients![Disputed, quite disputed, but with pushback]

Test statistics for regression coefficients are standard in statistical software output (typically t or z test statistics, depending on the nature of the estimator), and are just dandy for testing null hypotheses of the form $$H_{0}\text{: }\beta = 0$$ vs $$H_{A}\text{: }\beta\ne 0$$. [Hot damn!]

However… what if we want to test null hypotheses of the form $$H_{0}\text{: }\beta = c$$ for $$c \ne 0$$?

I know that a t test for a Pearson's correlation coefficient is pretty straightforward when the null hypothesis being tested is of the form $$H_{0}\text{: }\rho =0$$ vs $$H_{A}\text{: }\rho \ne 0$$, but that the math gets wonkier when the null hypothesis being tested is of the form $$H_{0}\text{: }\rho =c$$ for $$c\ne 0$$. So I wonder…

How to calculate the t or z test statistic for the null hypothesis $$\boldsymbol{H_{0}\textbf{: }\beta = c}$$ for $$\boldsymbol{c \ne 0}$$? Is it as simple as $$(\hat{\beta}-c)/\hat{\sigma}_{\beta}$$ or does it get funky?

Note: This is not a question about a test of one regression coefficient equaling another regression coefficient (i.e. not a test of $$H_{0}\text{: }\beta_{A} = \beta_{B}$$).

Edit: I bring up Pearson's $$\rho$$ in motivating my question, because (in the OLS case) $$\rho$$ and $$\beta$$ share the same numerator ($$\rho$$ is just the numerator normalized over the sample size), but calculating a t statistic for $$\rho$$ gets funky as noted above. I assume this is because the standard error no longer has the form under the $$c=0$$ null, and wonder why the standard error of $$\beta$$ does not likewise get funky.

• I know for sure that this has been covered a few times before in answers but for some reason I am having trouble turning them up at the moment. – Glen_b Mar 29 '19 at 0:02
• @Glen_b Well... but have they ever been asked with such zesty panache? – Alexis Mar 29 '19 at 15:12

When testing $$H_0:\rho=\rho_0\,(\ne0)$$ against any suitable alternative, one usually resorts to the variance stabilising Fisher transformation of the sample correlation coefficient $$r$$.

The usual t-test you are referring to is mainly reserved for the case $$\rho_0=0$$.

For moderately large $$n$$ (for example when $$n\ge 25$$), we have

$$\sqrt{n-3}(\tanh^{-1}(r)-\tanh^{-1}(\rho))\stackrel{a}\sim N(0,1)$$

So the appropriate test statistic under $$H_0$$ would be

$$T=\sqrt{n-3}(\tanh^{-1}(r)-\tanh^{-1}(\rho_0))\stackrel{a}\sim N(0,1)$$

In simple linear regression with normality assumption of errors (having variances $$\sigma^2$$), we have the exact distribution of the least square estimator $$\hat\beta$$ of the slope $$\beta$$, given by $$\frac{(\hat\beta-\beta)\sqrt{s_{xx}}}{\sigma}\sim N(0,1)$$ where $$s_{xx}=\sum (x_i-\bar x)^2$$.

Now if $$\sigma$$ is known, we test $$H_0:\beta=\beta_0$$ using the statistic $$T_1=\frac{(\hat\beta-\beta_0)\sqrt{s_{xx}}}{\sigma}\stackrel{H_0}\sim N(0,1)$$

If $$\sigma$$ is not known we estimate $$\sigma$$ by the residual sd $$s$$, where $$s^2=\frac{SSE}{n-2}$$. The test statistic is now

$$T_2=\frac{(\hat\beta-\beta_0)\sqrt{s_{xx}}}{s}\stackrel{H_0}\sim t_{n-2}$$

The variance of $$\hat\beta$$ in both these cases is independent of $$\beta$$ (the parameter of interest), so the variance stabilising transformation on $$r$$ while testing '$$\rho=\rho_0$$' is not required here. Not to mention that the test for '$$\rho=\rho_0$$' is a large sample test, while those involving $$\beta$$ are not.

• Thank you, StubborAtom. Can you help me understand why the variance of $\beta$ doesn't also need some kind of stabilizing transformation under a null such as the one I present, given that, for example in the OLS case, it shares the same numerator as $\rho$? – Alexis Mar 28 '19 at 5:43
• @Alexis Are you talking about the slope $\beta$ in simple linear regression (with the normality assumption of residuals)? If that is the case, the test statistic involving $\beta$ has an exact distribution and not a large sample test as the one involving $\rho$. – StubbornAtom Mar 28 '19 at 6:02
• Had the test been $H_0:\rho=0$ then it would be comparable with the test $:H_0:\beta=0$ because the former implies the latter under null. – StubbornAtom Mar 28 '19 at 6:08
• "The variance of $\beta$ in both these cases is independent of $\beta$ (the parameter of interest), so the variance stabilising transformation on $r$ while testing '$\rho=\rho_{0}$' is not required here." Nailed it. Thank you! – Alexis Mar 28 '19 at 16:37
• @Alexis Yes I just used $T_1,T_2$ to mean test statistic #1 for $\sigma$ known and test statistic #2 for $\sigma$ unknown. – StubbornAtom Jun 5 '20 at 16:44

If one is using something like a Wald statistic, a very simple way to do this to test if

$$H_o: \beta - c = 0$$ vs $$H_a: \beta - c \neq 0$$

Since we know that $$\hat \beta \text{ } \dot \sim N(\beta, se)$$, then $$\hat \beta -c \text{ } \dot \sim N(\beta - c, se)$$, since $$c$$ is just a constant. This gives us a Wald statistic of $$\frac{\hat \beta - c}{ se }$$.

In the case that we are not using a Wald statistic, we can (hopefully, if the software allows) include an offset in our model. That is, we can simply add $$cx^*$$, where $$x^*$$ is the covariate of interest, directly into the linear predictor. Then, if we wanted to do something like a likelihood ratio test, we could have one model with no free parameter associated with $$x^*$$ and another one that includes a free $$x^*\beta^*$$ to be estimated.

• Thanks Cliff AB! What does the dot above the $\sim$ mean? – Alexis Mar 28 '19 at 0:18
• @Alexis: "Approximately distributed as", rather than "distributed as". Since Wald statistics rely on asymptotics. – Cliff AB Mar 28 '19 at 0:18
• Thank you again! Ok, next question (possibly rolled into your answer). Why does the same logic not apply to the t test of Pearson's $\hat{\rho}$? Is it because $\rho$ only goes from $-1$ to $1$? – Alexis Mar 28 '19 at 0:21
• @Alexis: ha, that's actually the much harder question, in my opinion. I think it's because one is not using a Wald test statistic (i.e., normal approximation)? I'm not super familiar with the ins and outs of Pearson tests. – Cliff AB Mar 28 '19 at 0:27
• What is tripping me up and why I pitched Pearson's $\rho$ vs. $\beta$ is that they have the same numerators! For $H_{0}:\rho = 0$ the test statistic $t = \frac{\hat{\rho}}{\sigma_{\rho}}$, where the latter $= \sqrt{\frac{1-\hat{r}}{n-2}}$. I guess under the null, that SE changes dramatically for $\rho$, and am wondering why it does not change dramatically for $\beta$? – Alexis Mar 28 '19 at 0:29