A data set contains 5 columns: enter image description here

There are 16 people in the dataset(using subjectID to identify them). Each year, when these people were ages 11, 12, 13, 14, and 15, they provided us a score $y$. We now want to know whether there is a significant difference between male and female people (in their average $y$ score). How could I do? I found that this dataset does not meet the normality assumption. And in "Male" group, say subject 45 will appear 5 times, subject 268 will appear 5 times as well... Could I just use the mean of this "Male" group to make comparison with the mean of "Female" group?

And I also wonder if I can use a fixed effect model to compare these two groups:

enter image description here

We can see the difference between "Male" group and "Female" group is 0.13521, and it is not significant at 5% level as its p-value=0.228 (cannot reject $H_0$:"genderM"=0). So could I just say the difference is not significant?

  • $\begingroup$ Sampling the same subjects each year is a form of pseudo-replication, so your sample size is not as large as you think it is. You should only count each subject once $\endgroup$ – gardenhead Mar 26 at 21:16

Each measurement occasion is nested with an individual. There are a variety of ways to handle this issue: mixed effects models and cluster-robust standard errors are a few of them.

In a mixed effect model (aka multilevel model), you model each outcome value as a function of an intercept, (centered) age, and a measurement residual. You then model the intercept (and maybe also the slope of age) as a function of gender and an intercept (and slope) residual. The coefficient on gender is the effect of gender on the average outcome for an individual. The models look as follows:

$$y_{it} = \beta_{0t}+\beta_{1i}X^{age}_{it}+\epsilon_{it}\\ \beta_{0i}=\gamma_{00}+\gamma_{01}X^{gender}_{i}+u_{0i}\\ \beta_{1i}=\gamma_{10}+\gamma_{11}X^{gender}_{i}+u_{1i} $$ where $i$ indexes unit-ID and $t$ indexes measurement occasion for individual $i$. The third line can be replaced by $\beta_{1i}=\gamma_{10}+u_{1i}$ if you don't think the effect of age depends on gender but varies for each individual or $\beta_{1i}=\gamma_{10}$ if you think the effect of age is constant for all individuals.

To use cluster-robust standard errors, simply run the model $y_j=\beta_0+\beta_1X^{age}_{j}+\beta_2X^{gender}_{j}+\beta_3X^{age}_{j}X^{gender}_{j}+\epsilon_j$ where $j$ indexes each row in the data set, and then request a cluster-robust standard error with unit-ID as the clustering variable. If age is centered, $\beta_2$ will be the effect of gender on the outcome.

  • 2
    $\begingroup$ With 16 subjects, clustered SEs may perform poorly since 16 is pretty far from Asymptopia. $\endgroup$ – Dimitriy V. Masterov Mar 27 at 2:13
  • 1
    $\begingroup$ Good point. There might not be enough subjects for a multilevel model either actually. $\endgroup$ – Noah Mar 27 at 5:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.