Linear Regression with Dummy Predictor

Say there are two groups, each with n=500, with y=weight in pounds. The sample mean and sample standard deviation of weight are given:

Exercise(X=1): Mean=170, SD=20
Non-Exercise(X=0): Mean=190, SD=20

a. Would the regression equation be: y=b0+b1x, so y=190-20x?

And the interpretation: The mean different in weight between the two groups is 20 pounds.

b. Suppose that the SD for non-exercisers was 40, but the same model is fit to the data. How does this affect inferences on b1?

• Is the violation of assumption of equal variance violated, so inferences may not be valid?
• Any hints on how I could be more specific? The CI for b1 I think would be wider, with a different standard error.

(I am trying to review my regression concepts.)

• Just to let you know, we welcome questions like this but we treat them differently. Instead of giving you answers, we provide hints to help you figure it out for yourself. For more about our policy here, read this. – gung - Reinstate Monica Aug 31 '14 at 15:29
• Thanks, yes, I am just looking for hints to make sure I am on the right path to answering the question – user3731561 Aug 31 '14 at 15:59

For that data, the estimated regression equation would be either

$\,\hat{y}=190-20x\,,$ or

$\; y=190-20x+e$.

It can't be $y=190-20x$ because the observed $y$ values won't actually be $190-20x$, just near it.

b. Suppose that the SD for non-exercisers was 40, but the same model is fit to the data. How does this affect inferences on b1?

Is the violation of assumption of equal variance violated, so inferences may not be valid?

At n=500, the assumption of constant variance is seems sure to be violated. But even at small samples (where you wouldn't pretty confidently say they were from populations with different variance) the issue would be the difficulty in being reasonably confident that it wasn't violated.

There are a number of possibilities for dealing with inference if you don't assume constant variance; here are four of them (these are discussing hypothesis testing, but CIs would be based on the same calculations):

1) if you knew that one population sd was twice the other, you could use weighted regression. This knowledge is unlikely, but does sometimes happen.

You can then do hypothesis tests or CIs off that.

2) Since you're effectively just doing a two sample t-test you could use a Welch-Satterthwaite approximation. The numerator of the test statistic doesn't change, but the variance of the difference is based on the sum of the individual group estimates of their variances rather than a pooled variance, and (unless variances are identical), the degrees of freedom will be smaller.

You can generate a CI in similar fashion, using the df and standard error for the mean difference from the Welch-Satterthwaite calculation.

3) in large samples, you can basically treat $\hat{\sigma}$ as $\sigma$ and do a z-test for the difference. (given n=500, I'd probably just do this.)

This would look like a Welch-Satterthwaite t-test, but we'd just use z-tables. (It's also equivalent to assuming that you know the ratio of population sd's as in (1).)

The numerator is $\bar{y}_1-\bar{y}_0$, the denominator is the standard error of that difference (the square root of the variance of the difference in means, which variance is the sum of the individual variances of means)

Again, we can generate a CI from the same calculations - using the standard error from the denominator and the Z critical values for the desired coverage

This is all simple calculation. I am presuming you can do this from the information in your question. It should come out quite close to the answer I generated below (a different way).

4) you might look at some form of bootstrap interval where the resampling respected the fact that the two groups had different variance

In this case (nice big $n$, relatively modest change in sd), (1), (2) and (3) are going to give essentially the same inferences:

    mean sd   n
y0   190 20 500
y1   170 20 500
y1a  170 40 500


First, note that a regression CI for the case in your (a) should be the same as the CI here:

 t.test(y0,y1,var.equal=TRUE)

Two Sample t-test

data:  y0 and y1
t = 15.8114, df = 998, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
17.51781 22.48219
sample estimates:
mean of x mean of y
190       170


While the case in your (b) will yield a larger interval because we're now less certain about the mean of one of the groups:

> t.test(y0,y1a)

Welch Two Sample t-test

data:  y0 and y1a
t = 10, df = 733.824, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
16.0736 23.9264
sample estimates:
mean of x mean of y
190       170


You should get essentially the same interval under my first three options (and the fourth shouldn't be terribly different from that).

• Thank you- excellent, precise answer. You gave me a lot to think about/go over :) – user3731561 Sep 1 '14 at 14:26