9
$\begingroup$

Suppose we have a simple linear regression model $Z = aX + bY$ and would like to test the null hypothesis $H_0: a=b=\frac{1}{2}$ against the general alternative.

I think one can use the estimate of $\hat{a}$ and $SE(\hat{a})$ and further apply a $Z$-test to get the confidence interval around $\frac{1}{2}$. Is this ok?

The other question is strongly related to this one. Suppose that we have a sample $\{(x_1,y_1,z_1),\ldots ,(x_n,y_n,z_n) \}$ and we compute $\chi^2$ statistics

\begin{equation} \sum_{i=1}^n \frac{(z_i-\frac{x_i+y_i}{2})^2}{\frac{x_i+y_i}{2}}. \end{equation} Can these statistics be used to test the same null hypothesis?

$\endgroup$
8
$\begingroup$

In linear regression the assumption is that $X$ and $Y$ are not random variables. Therefore, the model

$$Z = a X + b Y + \epsilon$$

is algebraically the same as

$$Z - \frac{1}{2} X - \frac{1}{2} Y = (a - \frac{1}{2})X + (b - \frac{1}{2})Y + \epsilon = \alpha X + \beta Y + \epsilon.$$

Here, $\alpha = a - \frac{1}{2}$ and $\beta =b - \frac{1}{2}$. The error term $\epsilon$ is unaffected. Fit this model, estimating the coefficients as $\hat{\alpha}$ and $\hat{\beta}$, respectively, and test the hypothesis $\alpha = \beta = 0$ in the usual way.


The statistic written at the end of the question is not a chi-squared statistic, despite its formal similarity to one. A chi-squared statistic involves counts, not data values, and must have expected values in its denominator, not covariates. It's possible for one or more of the denominators $\frac{x_i+y_i}{2}$ to be zero (or close to it), showing that something is seriously wrong with this formulation. If even that isn't convincing, consider that the units of measurement of $Z$, $X$, and $Y$ could be anything (such as drams, parsecs, and pecks), so that a linear combination like $z_i - (x_i+y_i)/2$ is (in general) meaningless. It doesn't test anything.

$\endgroup$
  • 1
    $\begingroup$ Thanks for your answer. It was very useful. Actually, I wasn't very precise in the formulation of the second part of the question. Imagine, that x-s and y-s are positive numbers, measured in the same units. The z-s (observed outcome) somehow measure the "interaction" in that sense that if there is no interaction the z-s should be (x+y)/2 (expected outcome). So from my point of view it was the same to use regression with the null hypothesis a=b=1/2 or to compare goodness of fit using Pearson's chi^2 statistics. Does this make any sense? Thanks! $\endgroup$ – Lan Nov 30 '10 at 20:28
  • 1
    $\begingroup$ @Lan I think Wolfgang's answer nicely illustrates how to make the test you are proposing. It is an example of what meant by testing a hypothesis "in the usual way." $\endgroup$ – whuber Nov 30 '10 at 20:56
9
$\begingroup$

You can test this hypothesis with a full versus reduced model test. Here is how you do this. First, fit the model $Z = aX + bY$ and get the residuals from that model. Square the residuals and sum them up. This is the sum of square error for the full model. Let's call this $SSE_f$. Next, calculate $Z - \hat{Z}$ where $\hat{Z} = 1/2*X + 1/2*Y$. These are your residuals under the null hypothesis. Square them and sum them up. This is the sum of square error for the reduced model. Let's call this $SSE_r$.

Now compute:

F = $((SSE_r - SSE_f)/2) / (SSE_f / (n-2))$,

where $n$ is the sample size. Under $H_0$, this F-statistic follows an F-distribution with $2$ and $n-2$ degrees of freedom.

Here is an example using R:

x <- rnorm(n)
y <- rnorm(n)
z <- 1/2*x + 1/2*y + rnorm(n) ### note I am simulating under H0 here

res <- lm(z ~ x + y - 1)
summary(res)
SSE.f <- sum(resid(res)^2)

zhat  <- 1/2*x + 1/2*y
SSE.r <- sum((z-zhat)^2)

F <- ((SSE.r - SSE.f) / 2) / (SSE.f / (n-2))
pf(F, 2, n-2, lower.tail=FALSE) ### this is the p-value

Reject the null if the p-value is below .05 (if your $\alpha$ is indeed .05).

I assume you really meant for your model not to contain an intercept. In other words, I assume you are really working with the model $Z = aX + bY$ and not $Z = c + aX + bY$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.