# Estimate mean of a population with multiple samples when the individual sample mean is biased

I am working with datasets of grades going ~15 years back for different classes. I am trying to determine if there is a difference in the average grade for odd years compared to even years. There is a big event going on for several weeks during the exam period every even year close to the university, and I want to see if this affects the students grades.

I am only experienced with working with single samples from a population and estimating mean, variance etc.

My initial though was to find the mean/variance for each year, then estimate the total mean/variance using an unbiased linear estimator weighted by sample size (number of students taking the class that year).

However, I suspect this is not correct because the relative difficulty of the exam changes, this would of course affect every grade for that year. How do I adjust for this bias in the individual sample mean, which I assume is independent from the fact that it is an odd/even year?

• sounds like what you're looking for is a random effects model Commented May 15, 2019 at 0:50

Here is one elementary way to view this question. [It assumes scores in the past few odd years have not differed greatly over time, and similarly for the even numbered years. If you think exams year-to-year may have differed substantially, you could check that by doing a one-way ANOVA for Odd years (and separately) another ANOVA for Even years. If those differences are larger than the difference between Odd and Even, then you'd need to decide what more complicated analysis might untangle the two effects.]

Data. Suppose you have 8 odd years and 7 even years and about 50 students each year. Make a list of grades $$X_i$$ in odd years, for $$i = 1, 2,\dots, n_1;$$ also a list of grades $$Y_i$$ in even years, for $$i = 1, 2, \dots, n_2.$$

Here are summaries of some fake data generated in R to use as an example:

summary(x); length(x); sd(x)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
77.00   83.00   85.00   84.99   87.00   95.00
## 401      # n_1 number of students in odd years
## 3.326394 # SD of odd-year scores

summary(y); length(y); sd(y)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
64.00   71.00   73.00   73.64   76.00   85.00
## 348      # n_2 nr in even yrs
## 3.897763 # SD for even years


Two-sample Welch t test. Then do a Welch (separate-variances) t test to see whether the mean $$\mu_1$$ for odd years is equal to the mean $$\mu_2$$ for even years (null hypothesis) or not equal (alternative hypothesis). You can find the formula for the Welch t statistic and for its degrees of freedom in a basic statistics text or online.

Here results of the Welch test from R:

t.test(x, y)

Welch Two Sample t-test

data:  x and y
t = 42.507, df = 686.39, p-value < 2.2e-16
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
10.82226 11.87044
sample estimates:
mean of x mean of y
84.99002  73.64368


For my fake data, there is a highly significant difference between the sample means $$\bar X = 84.99$$ and $$\bar Y = 73.64,$$ as shown by the very small P-value.

[That is because I simulated the data to have different population means $$\mu_1 = 85, \mu_2 = 74,$$ as shown below---also, to have somewhat different standard deviations.]

set.seed(514)  # for reproducibility
x = round(rnorm(401, 85, 3))
y = round(rnorm(348, 74, 4))