1
$\begingroup$

I'm trying to figure out a way to test hypotheses about variance in the linear regression model. The hypotheses I want to test are: \begin{align*} &H_0:\sigma^2=1\\ &H_1:\sigma^2<1 \end{align*}

The linear regression model is given by $$Y_i=aX_i+b+\varepsilon_i,\quad i=\overline{1,n}$$ where $\varepsilon_i\in\mathcal{N}(0,\sigma^2)$ are independent random variables. If $\hat{a},\hat{b}$ are least-square estimates of $a,b$, then the least-square estimate of $\sigma^2$ is: $$\hat{\sigma^2}=\frac{1}{n-2}\sum_{j=1}^n(Y_j-\hat{a}X_j-\hat{b})^2$$ Based on the conversation in this thread, $$Z(X)=\frac{(n-2)\hat{\sigma^2}}{\sigma^2}\in\chi^2({n-2})$$ but it depends on $\sigma^2$ as a function and is not a statistic. However, its distribution is $\chi^2_{n-2}$ no matter the value of $\sigma^2$. I try to find a confidence interval for $\sigma^2$ and a given statistical significance $\alpha$.

I need to find $\sigma_a^2,\sigma_b^2:$ $$\mathbb{P}_0(\sigma_a^2\leq\sigma^2\leq\sigma_b^2)=\mathbb{P}_0(\chi^2_{\alpha}({n-2})\leq Z\leq\chi^2_{(1-\alpha)}(n-2))=1-\alpha$$ Using the expression for $Z$ to transform the inequalities I get: $$\mathbb{P}_0(\sigma_a^2\leq\sigma^2\leq\sigma_b^2)=\mathbb{P}_0(\frac{(n-2)\hat{\sigma^2}}{\chi^2_{(1-\alpha)}}\leq \sigma^2\leq\frac{(n-2)\hat{\sigma^2}}{\chi^2_{\alpha}})=1-\alpha$$ which explicitly gives the borders of an interval.

My question is, can I use this for interval for hypothesis testing? If not, how do I test hypotheses for $\sigma^2$? Is there a better way to do it?

$\endgroup$
3
  • 2
    $\begingroup$ Under your null hypothesis, you do have a value for $\sigma,$ which leaves no doubt about the distribution of $\hat\sigma^2.$ $\endgroup$
    – whuber
    Jun 15, 2022 at 16:07
  • $\begingroup$ @whuber I don't understand what it means in relation to the CI for $\sigma^2$. The true value of $\sigma^2$ is fixed for any given model, so the distribution of $Z$ doesn't depend on it. I am trying to use this fact to obtain the CI and my question is whether it's a valid way to test $H_0$. $\endgroup$ Jun 15, 2022 at 16:24
  • 1
    $\begingroup$ You test $H_0$ by using the distribution the test statistic will have when $H_0$ is true. You can confirm such a test is equivalent to your confidence interval (as is usually the case). $\endgroup$
    – whuber
    Jun 15, 2022 at 17:25

1 Answer 1

4
$\begingroup$

As pointed out in the comments, the result you give for the pivotal quantity in this case is already enough to perform the hypothesis test. (Remember that when you do a classical hypothesis test you look at the distribution of the test statistic assuming that the null hypothesis is true.) If you assume normality of the error term, the standard way to test this is using the pivotal quantity you have identified:$^\dagger$

$$(n-2) \cdot \frac{\hat{\sigma}^2}{\sigma^2} \sim \text{ChiSq}(n-2) = \text{Ga}(\tfrac{n-2}{2}, \tfrac{1}{2}).$$

Consequently, taking $\sigma=1$ and scaling gives the null distribution for your test, which is:

$$\hat{\sigma}^2 \sim \text{Ga}(\tfrac{n-2}{2}, \tfrac{n-2}{2}).$$

Usually you would implement the test by computing the two-sided p-value:

$$p(\hat{\sigma}^2) = \int \limits_0^\infty \text{Ga}(r | \tfrac{n-2}{2}, \tfrac{n-2}{2}) \cdot \mathbb{I} \Big( \text{Ga}(r | \tfrac{n-2}{2}, \tfrac{n-2}{2}) \leqslant \text{Ga}(\hat{\sigma}^2 | \tfrac{n-2}{2}, \tfrac{n-2}{2}) \Big) \ dr.$$

Of course, you can proceed using the alternative method of computing a confidence interval for $\sigma^2$ at a particular confidence level $1-\alpha$ and then use this as a means of making inferences about the error variance. (Indeed, most statisticians prefer confidence intervals to hypothesis tests, due to a number of well-known limitations on the latter.) If you form a confidence interval then I would recommend you do so using the HDR criterion (i.e., setting the endpoints so that the density of your pivotal quantity is the same at each endpoint). For more information on this, see e.g., O'Neill (2022).


A warning: Although the above is the standard method for the Gaussian linear regression model, you should bear in mind that this test/interval is highly sensitive to the assumption of a mesokurtic error distribution. This is true both for the hypothesis test and the corresponding confidence interval. In either case, the kurtosis of the error term strongly affects the variability of the error variance estimator $\hat{\sigma}^2$, so it is usually best to use more robust tests that account for the sample kurtosis in the residual distribution (as an estimate of the true kurtosis of the error distribution). If you would like to learn more about how to adjust your test/interval to account for the kurtosis of the error distribution, see e.g., O'Neill (2014).


$^\dagger$ To avoid ambiguity, we note that in the gamma distribution we use the parameterisation of shape then rate.

$\endgroup$
4
  • $\begingroup$ (+1). I'm having trouble understanding the notation for the $p$-value. Intuitively, I would double the smaller one-sided $p$-value, calculated as the area under the null density from the observed residual variance into the tail. $\endgroup$ Jun 15, 2022 at 21:41
  • 1
    $\begingroup$ @COOLSerdash: That is an alternative approach that is not too bad, but it is not optimal (in my opinion). Usually we would measure outcomes as more or less conducive to the alternative hypothesis based on the likelihood-ratio statistic, which is proportionate to the density, so we would usually want the lower and upper cut-offs to occur at the same density level for the pivotal quantity. $\endgroup$
    – Ben
    Jun 15, 2022 at 22:00
  • $\begingroup$ @Ben, does $Ga$ denote the gamma distribution? $\endgroup$ Jun 16, 2022 at 14:04
  • $\begingroup$ @pieinthesky: Yes; see the footnote for parameterisation. $\endgroup$
    – Ben
    Jun 16, 2022 at 22:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.