10
$\begingroup$

Given that I have $C$ to spend on my experiment, what is the optimal number of individuals $n$ of my experiment?

Say that I want to estimate the mean $\mu$ of distribution of individuals

$$X_i \sim N(\mu, \sigma)$$

and I can sample several values from each individual which is distributed as

$$Y_{ij} \sim N(X_{i}, \tau)$$

Say that

  • the costs of recruiting an individual are $a$
  • and the cost of obtaining a sample are $b$

For example, when we sample Alice, Bob and Carol twice each and Eve once, then we sampled 4 persons 7 times and the cost will be $4a+7b$. (So sampling the same person another time only costs $b$ whereas sampling a new person will cost $a+b$)

then what is the ideal number of individuals $n$ and samples $k$, within the costs $C$ to get the highest precision of estimate (in terms of lowest variance).

Let's assume that we know $\sigma$ and $\tau$. But $\mu$, which we want to estimate, is unknown.

Let's assume that we use a weighted least squares to estimate $\mu$. And I consider optimal to be the lowest standard error of the estimate (which needs to be unbiased).


Motivations for this question are

How to fix the problem in this XKCD comic?

Balancing effect of sample size and amount of trials on power

$\endgroup$
14
  • $\begingroup$ Just to clarify, if I sample Alice, Bob and Carol twice each and Eve once, I will pay $3a + a + (2\cdot 3)b + b$? $\endgroup$
    – usεr11852
    Feb 24, 2023 at 15:45
  • 1
    $\begingroup$ I'm not going to provide a complete answer, I just tell you that as far as I can see, the variance of the overall mean (assuming that your notation uses unsquared standard deviations) will be $\frac{\sigma^2}{n}+\frac{\tau^2}{kn}$. Obviously $k\to\infty$ will reduce the second term to 0, but increasing $n$ is the only way to get the first part down (and this will reduce the second part as well of course). I haven't checked whether the plain mean is the optimal estimator though. I am however sure that no estimator can get the variance to zero for fixed $n$ and $k\to\infty$. $\endgroup$ Feb 24, 2023 at 15:52
  • $\begingroup$ For $k=1$ the plain mean achieves $\frac{\sigma^2+\tau^2}{n}$ and this is optimal. For fixed $n$ and $k\to\infty$ there is no way to do better than $\frac{\sigma^2}{n}$ as this would assume we know the $X_i$ precisely, so whatever optimal GLS estimator will be between these. $n\to\infty$ will reduce the variance to zero in all cases, $k\to\infty$ never will. (Investing a certain amount into $k$ may be OK if $\tau^2>\sigma^2$.) $\endgroup$ Feb 24, 2023 at 16:04
  • $\begingroup$ I agree with @ChristianHennig (+1) Intuitively, small $k$ and high $n$ should be a better solution than high $k$ and low $n$. But i think this is where the costs come into play (that's why I asked) if $a >> b$ for example we accrue a lot of cost very quickly. $\endgroup$
    – usεr11852
    Feb 24, 2023 at 16:07
  • $\begingroup$ @usεr11852 if you sample 3 people and sample 2+2+1=5 times, then you will pay $3a+5b$. Obviously inreasing $n$ is better but it comes at a higher cost because it is not just adding another sample, but also adding an additional individual on the pool of sampled individuals. $\endgroup$ Feb 24, 2023 at 16:15

1 Answer 1

6
$\begingroup$

The mean per individual will be distributed as

$$\bar{Y}_i = \frac{1}{n_i} \sum_{j = 1}^{n_i} Y_{ij} \sim N\left(\mu, \sigma^2 + \tau^2/n_i\right)$$

where $n_i \geq 1$ are the number of observations for individual $i$ (we need at least 1 measurement for a participant).

The estimate will be

$$\hat{\mu} = \sum_{i=1}^n w_i \bar{Y}_i$$

with $$w_i = \frac{(\sigma^2 + \tau^2/n_i)^{-1}}{ \sum_{l=1}^n (\sigma^2 + \tau^2/n_l)^{-1}} $$

and the variance will be

$$\text{VAR}(\hat{\mu}) = \frac{1}{\sum_{l=i}^n (\sigma^2 + \tau^2/n_i)^{-1}} \approx \frac{\sigma^2}{n }+ \frac{\tau^2}{ \sum n_i} = \frac{\sigma^2}{n }+ \frac{\tau^2}{ m} $$

The approximate is exactly true when the $n_i$ are all the same. And we defined $m = \sum n_i$.

The variance decreases when we increase $n$ or when we increase $m$. With the changes being

$$\frac{\partial}{\partial n} \text{VAR}(\hat{\mu}) = - \frac{\sigma^2}{n^2} \\ \frac{\partial}{\partial m} \text{VAR}(\hat{\mu}) = - \frac{\tau^2}{m^2} \\$$

and the optimum will occur when the amount of observations per individual follows the ratio

$$\frac{m}{n} = \frac{\tau\sqrt{a}}{\sigma\sqrt{b}}$$

and there is also the limit $\frac{m}{n} > 1$ because we need at minimum one observation per individual.

$\endgroup$
1
  • 1
    $\begingroup$ The analysis is a bit approximate since I assume a continuous change in $m$ but in reality this will occur in discrete steps, and in addition not all individuals need to have exactly equal $n_i$ trials per individual. $\endgroup$ Feb 25, 2023 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.