1
$\begingroup$

I've had trouble finding a clear answer elsewhere on the internet and thought I'd put it to the XV community.

Problem Description

Suppose I have $N$ samples, each on a different subject. Each sample involves of $n_i$ measurements on a single subject, with each measurement yielding a value $x_j$. Each sample has a mean $\mu_i$ and standard deviation $\sigma_i$. I wish to combine the means into a single mean $\mu_T$ and test if $\mu_T$ is statistically different from a particular value $y$.

I have access to all $x_j$, but don't want to compute mean and standard error of all $x_j$, as different subjects may be more or less reliable.

To compute the combined mean, I am using the equation $\mu_T = \frac{\sum_i^N n_i \mu_i}{\sum_i^N n_i}$.

Question 1:

What is the correct formula for the standard error of $\mu_T$? I've thought about using the pooled standard error, \begin{equation} \sigma_{ErrT} = \sqrt{\frac{\sum_i^N (n_i-1)\sigma_i^2}{\sum_i^N (n_i-1)}\sum_i^n \frac{1}{n_i}} \end{equation} but is this an appropriate estimate of the standard error of $\mu_T$?

Question 2:

Once the standard error of $\mu_T$ is found, is it appropriate to use a t-test to compare if $\mu_T$ and $y$ are statistically different? What would be the d.o.f. of the test statistic?

Improve this question
$\endgroup$
5
  • $\begingroup$ So am I correct in saying that this is a weighted mean among $N$ subjects, with the weights being the number of measurements per subject? Is there any reason to think that the measurement variance would differ among individuals (that is, that there are differences among the $\sigma_i^2$ values)? $\endgroup$
    – EdM
    Commented Aug 21, 2015 at 20:38
  • $\begingroup$ @EdM Yes, it is a weighted mean based on measurements per subject. (In the practical example I'm thinking of, however, the number of measurements are all equal, i.e. $n_1=n_2=...=n_N$.) There are differences between the $\sigma^2$ values. $\endgroup$
    – Delyle
    Commented Aug 21, 2015 at 21:36
  • $\begingroup$ If you think that there are differences among the variances, have you considered weighting by the (inverse) variances in some way instead of by numbers of observations? And how confident are you that the differences among variances are real as opposed to just being from different samples having the same underlying variance? $\endgroup$
    – EdM
    Commented Aug 21, 2015 at 21:56
  • $\begingroup$ @EdM I had considered weighting by the inverse variance, but I can't justify it to myself. For example, let's say you want to combine the reaction times of two people, and the first has consistent reactions times while the other has highly variable reactions. If you took the weighted average with weights$=1/\sigma_i$, the first person's reaction time would contribute more, but is that person really a better representative of the parent population just because they were more consistent? $\endgroup$
    – Delyle
    Commented Aug 21, 2015 at 22:12
  • $\begingroup$ @EdM For your second question, I am not confident that the differences among variances do not arise simply because different samples have the same underlying variance. $\endgroup$
    – Delyle
    Commented Aug 21, 2015 at 22:12

1 Answer 1

Reset to default
1
$\begingroup$

Possibly the simplest way to accomplish what you want is to set up the problem as a linear model, where each of your observations is the difference $(x_{ij} - y)$ and the individuals are taken as independent variables, maybe better as a random effect. You then test whether the intercept of the linear model is different from zero, which is usually a direct output from a standard statistical program. This would essentially take into account, in a reasonably standard way, all of your issues about numbers of observations, differences among individuals, etc., and might provide useful estimates of the differences among individuals, particularly if the variances may be the same among individuals.

To answer the question as you posed it, I've rewritten your formula with hats for the estimated values so that we don't confuse them with population values.

$$\hat\mu_T = \frac{\sum_i^N n_i \hat\mu_i}{\sum_i^N n_i}.$$

If the observations are independent, then the basic properties of variances give the variance of your estimate $\hat\mu_T$ as:

$$\frac{\sum_i^N n_i^2 Var(\hat\mu_i)}{(\sum_i^N n_i)^2} = \frac{\sum_i^N n_i^2 (\frac{s_i^2}{n_i})}{(\sum_i^N n_i)^2} = \frac{\sum_i^N n_i s_i^2}{(\sum_i^N n_i)^2}.$$

This is based on the formula for the square of the standard error of the mean $$Var(\hat\mu_i) = \frac{s_i^2}{n_i},$$

where $s_i^2$ is the unbiased sample estimate of the variance. The square root $$\sqrt{\frac{\sum_i^N n_i s_i^2}{(\sum_i^N n_i)^2}}=\frac{\sqrt{\sum_i^N n_i s_i^2}}{\sum_i^N n_i}$$

gives you an estimate of the standard error of $\hat\mu_T$. This seems a bit different from your formula. I think that, out of the total number of observations, you have used up one degree of freedom for each of the $\hat\mu_i$ when doing it this way, which is somewhat analogous to treating the individuals as fixed effects in a linear model. But I get confused sometimes when I try to reason through degrees of freedom, which is one of many reasons why I try to use well-vetted statistical routines whenever possible.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for this. Using a linear model makes sense to me, just need some clarification. If the individuals are set up as random effects, we'd get different intercepts every time depending on how the individuals are arranged around 0. So should the individuals be set up as random effects and the intercept computed multiple times? $\endgroup$
    – Delyle
    Commented Aug 22, 2015 at 1:04
  • $\begingroup$ A random-effects treatment of intercepts among individuals should provide an estimate of the distribution of differences of intercepts among individuals from the overall intercept of the model. Then the question is whether the overall model intercept is different from 0, in the way I suggested that you code the data. I don't see that the ordering of the individuals would make a difference in that analysis. $\endgroup$
    – EdM
    Commented Aug 22, 2015 at 3:09
  • $\begingroup$ In R the model could be written lmer(xdiff ~ 1 + (1|individual)), where xdiff represents the difference of each observation from your hypothesized baseline value of $y$. Supply one data point per row, coded as to individual. This returns an estimate of the variance of intercepts among individuals and of the residual variance, which is then used to test whether the overall model intercept is different from 0. See here for one example. Fancier treatments are possible, but my guess is that this will do. $\endgroup$
    – EdM
    Commented Aug 22, 2015 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.