3
$\begingroup$

I would pretty much appreciate if someone could explain the discrepancy detailed below.

Let's do a theoretical test first.

# make some test data with three levels and a vector of random numbers
test <- data.frame(ind = gl(3, 10), val = runif(30))
# calculate mean by "ind" (stored to a variable for pedagogic purposes
ind.mean <- with(test, tapply(X = val, INDEX = ind, FUN = mean))
# calculating mean of means by "ind" and a global mean yields same results - FINE
mean(ind.mean)
mean(test$val)

Here's for some tricky data.

rld <- structure(list(date = c("2011", "2011", "2011", "2011", "2011", 
"2011", "2011", "2011", "2011", "2010", "2010", "2010", "2010", 
"2010", "2010", "2010", "2010", "2010", "2010", "2010", "2010", 
"2010", "2010", "2010", "2010", "2010", "2010", "2010", "2010", 
"2010", "2010", "2010", "2010", "2010", "2010", "2009", "2009", 
"2009", "2009", "2009", "2009", "2009", "2009", "2009", "2009", 
"2009"), pm = c(1, 0.80952380952381, 0.75, 1, 0.866666666666667, 
0.8, 1, 0.8, 0.833333333333333, 0.666666666666667, 0.666666666666667, 
1, 1, 1, 1, 1, 1, 1, 1, 0.666666666666667, 0.75, 0.75, 1, 1, 
0.833333333333333, 0.833333333333333, 0.714285714285714, 0.833333333333333, 
0.818181818181818, 0.8, 0.666666666666667, 1, 0.5, 1, 1, 0.166666666666667, 
0.166666666666667, 1, 1, 0.8, 0.5, 0.5, 0.25, 0.333333333333333, 
0, 0.909090909090909)), .Names = c("date", "pm"), class = "data.frame", row.names = c(NA, 
-46L))

> head(rld)
  date        pm
1 2011 1.0000000
2 2011 0.8095238
3 2011 0.7500000
4 2011 1.0000000
5 2011 0.8666667
6 2011 0.8000000

One would expect the results to be identical (sensu above test case), alas...

> mean(rld$pm)
[1] 0.7822699
> mean(tapply(X = rld$pm, INDEX = rld$date, FUN = mean))
[1] 0.7500214

EDIT

Chase nailed it. Here's proof:

test3 <- data.frame(ind = rep(1:3, times = c(5, 15, 10)), val = runif(30))
> mean(test3$val)
[1] 0.4972464
> mean(tapply(X = test3$val, INDEX = test3$ind, FUN = mean))
[1] 0.468399
$\endgroup$
2
  • 1
    $\begingroup$ This really is a basic math question and not an R question. Perhaps someone with 'power' can redirect the category? $\endgroup$
    – Carl Witthoft
    Sep 5, 2011 at 20:05
  • $\begingroup$ I was thinking of getting it moved to either math.SE or crossvalidated. Didn't think hard enough before I posted. $\endgroup$ Sep 5, 2011 at 20:07

2 Answers 2

12
$\begingroup$

The problem is that you are trying to take the average of a set of averages where each component made up of a different number of rows:

> table(rld$date)
2009 2010 2011 
  11   26    9  

When you try and take the average of those groups, you assign equal weight to the average from 2009, 2010, and 2011.

I think an easy parallel is thinking about a set of quizzes and tests. Quizzes are more frequent and worth fewer points, so having a 100% average from a set of five quizzes worth 10 points each carries less weight than the midterm which is worth 100 points.

$\endgroup$
6
$\begingroup$

Only when the counts in all groups are the same will the mean of means equal the global mean,

$\endgroup$
3
  • $\begingroup$ By "the same" you mean in the same approximate range with a similar distribution? $\endgroup$ Sep 5, 2011 at 19:38
  • $\begingroup$ No. The same. As in identical. $\endgroup$
    – DWin
    Sep 5, 2011 at 19:50
  • $\begingroup$ I guess I should make explicit that I meant the counts of values, not identical values. $\endgroup$
    – DWin
    Sep 5, 2011 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.