0
$\begingroup$

Students are assigned to group A or B based on two aspects: a grade from an exam and an evaluation of their teacher. Students with both a high grade and a positive evaluation get into group A, other students in group B. I know that the grade is much more influential than the teacher’s evaluation.

I have the exam grades of 1,000 students. I also know whether students are assigned to group A or B.

I want to know whether the average grade of students in group A is higher than the average grade in group B.

I am thinking about this regression (which I believe is equivalent to a direct t-test):

$\text{Grade}_i=\alpha+\beta*\text{DumA}_i+\epsilon_i$

where $\text{Grade}_i$ is the grade of student i. $\text{DumA}_i$ equals 1 if student i is in group A and 0 otherwise.

Do students in group A have a higher grade if Alfa is significant according to a standard t-test? Note that students in group A tend to have a higher grade since assignment is mostly based on the grade. It seems to me that the dummy is endogenous and thus the estimate of Alfa is incorrect. Is my interpretation correct?

Does this mean that it is not possible to compare the mean of the two groups? Are there any alternatives?

Any thoughts are very much appreciated!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You can compare the means of the two groups using an ordinary t-test. But, if you want to generalize the results, it has to be to groups that are divided similarly to the ones in your sample.

You could also use your regression, but the means would be significantly different if $\beta_1$ is significant. $\alpha$ would be the mean of students in the group that was coded 0.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Dear Peter, Thank you very much for your helpful answer! Do you think that the endogeneity is not a problem? That is, the two groups of which I want to compare the mean grade are partly formed via the grades. $\endgroup$
    – f3000
    Commented Dec 1, 2013 at 15:11
  • $\begingroup$ That depends on what you mean by a "problem". E.g. it is perfectly fine to (say) test whether jockeys and basketball players are the same height. But does it tell you anything useful when you find out that they are not? This is not a problem with the t-test, nor can it be solved by using another test. It's a question of whether it makes any sense to do the comparison. $\endgroup$
    – Peter Flom
    Commented Dec 1, 2013 at 15:19
  • $\begingroup$ Peter, thank you very much! Two points: 1) The comments of Peter imply that the size of the correlation coefficient and the p-value of the t-test on whether this coefficient differs from zero are valid in the regression above. Is the regression above only not valid if I claim causality between the assignment to a group and the grade? $\endgroup$
    – f3000
    Commented Dec 1, 2013 at 21:49
  • $\begingroup$ 2) Basically, I would like to know whether the difference that I find between the mean of the grades in group A and group B is substantial. I believe that by using the t-test, I define substantial as that the difference between the two means (which exists almost by definition) cannot be attributed to coincidence, where coincidence is defined as the possibility that the grades of students in group A are generated from the same distribution as the grades of students in group B. Is my application of the t-test sensible and is my interpretation correct? $\endgroup$
    – f3000
    Commented Dec 1, 2013 at 21:49
  • $\begingroup$ 1) Causality has nothing to do with it. 2) Be careful about "substantial" vs. "significant". With a large enough N, very small differences can be significant; with a small N, even large differences may not be. $\endgroup$
    – Peter Flom
    Commented Dec 1, 2013 at 23:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.