The relative size of the SE for ratios compared to the SE for means

Question

Pardon a relative novice's question. I'm seeking a reference that describes, compares, and gives formulas for the standard error for ratios and the standard error for differences between means when small Ns are involved.

Here is the situation at hand:
- Multiple samples from two independent populations.
- The underlying variable is normally distributed in each population, but with somewhat different means and standard deviations.
- At least one sample will always be small, perhaps 3 to 15 cases, the other sample will generally be 50 or 100 cases, or even larger.
- Statistics of interest: ratio of means and difference between means.
- The variables are grades on multiple choice tests and such tests tend to be normally distributed. The p (i.e., probability of getting a question correct, AKA difficulty level) for the tests in question is around .7

Question: How will the standard errors of these two sample statistics compare?

Update based on comments

The specific question I have is whether the mean standardized difference between 2 groups will be a more or less stable measure (i.e., have a larger or smaller standard error) than the ratio of the means, when at least one of the groups is small.

Answer 1

Standard error of ratio of two approximately normal distributed variables

The standard error relates to the variance or standard deviation and because of this your question might be related to some other questions on stackexchange:

• What are the mean and variance of the ratio of two normal variables, with non-zero means?

  The variance of the ratio of two normally distributed variables does not exist. It is undefined. This is because the reciprocal of a normal distributed variable has no defined mean and variance.

• estimation of population ratio using delta method

  However, in practice variables are not always normal distributed (the normal distribution is just an approximation). For instance, in many cases the domain of the denominator has some lower bound that is positive (unlike the normal distribution). In that case the ratio will have finite moments.

  You can estimate the variance using the delta method (see for example estimation of population ratio using delta method). In that stackexchange link the ratio is approximated by the first terms in a Taylor series (which requires $\sigma << \mu$, ie. small errors) : $$Z = \frac{X}{Y} \approx \frac{\mu_X}{\mu_X} + (X-\mu_X) \cdot \frac{1}{\mu_Y} + (Y-\mu_Y) \cdot \frac{-\mu_X}{\mu_Y^2} $$ and the variance (if it exists) is: $$\sigma_{X/Y}^2 \approx \sigma_{X}^2 \cdot \frac{1}{\mu_Y^2} + \sigma_{Y}^2 \cdot \frac{\mu_X^2}{\mu_Y^4}$$

(see also https://en.wikipedia.org/wiki/Taylor_expansions_for_the_moments_of_functions_of_random_variables)

---

So those are some few thoughts about the expression of the standard error for a ratio. But your more direct question: "How will the standard errors of these two sample statistics compare?" is a bit unclear to me.

We can say very straightforward that $$\sigma_{X-Y}^2 = \sigma_{X}^2+\sigma_{Y}^2$$ and $$\sigma_{X/Y}^2 \approx \sigma_{X}^2 \cdot 1/\mu_Y^2 + \sigma_{Y}^2 \cdot {\mu_X^2/\mu_Y^4}$$

But is that what you are looking for? In what sense do you wish to compare the standard errors? Do you wish to evaluate which statistic is the best in order to test the hypothesis Y=X?

---

Stability

Below is an example for the variables $$\begin{array}{rcl} Z_1 &=& \frac{\bar{X}_n}{\bar{Y}_n} \\ Z_2 &=& {\bar{X}_n}-{\bar{Y}_n}\end{array}$$ with $$\begin{array}{cccc}\bar{X}_n &=& \frac{1}{n} (X_1, X_2, ... , X_n) & \qquad X_i \sim N(\mu_X=1,\sigma_X = 2) \\ \bar{Y}_n &=& \frac{1}{n} (Y_1, Y_2, ... , Y_n)& \qquad Y_i \sim N(\mu_Y=1,\sigma_Y = 2) \end{array}$$

$X_n,Y_n$ can be considered as a sort of random walk and as $n \to \infty$ you will get $X_n,Y_n \to \mu_X, \mu_Y$. You can see the distribution of $X_n,Y_n$ as a multivariate Gaussian that becomes more and more concentrated around the point $\mu_X, \mu_Y$ and this is like sort of zooming in to the point $\mu_X, \mu_Y$.

The distribution for the values of $Z_1$ and $Z_2$ can be imagined by superimposing to this random walk the isolines for values of $Z_1$ and $Z_2$. The lines for $Z_2$ are parallel and no matter what value of $n$ the statistic $Z_2$ is normal distributed. The lines for $Z_1$ are not parallel and the distribution for $Z_1$ is not normal distributed, but when you zoom in (when $n$ increases) the lines are more parallel, and the distribution of $Z_1$ becomes more and more normal distributed. In the case of $\mu_X=\mu_Y=1$ you will even get that $Z_1$ and $Z_2$ become as close as you wish for sufficiently large $n$.

• Note while the distribution for $\bar{X}_n/\bar{Y}_n$ is asymptotically normal distributed for $n \to \infty$ (see also https://stats.stackexchange.com/a/399952/164061 ) it will still have some sort of Cauchy/Lorentz distribution component and an expression of stability in terms the standard deviation (or error) is not defined.

  But the good news is that in practice we often do not have truly a normal distribution. So when your grades are always positive (no negative and zero values as the normal distribution would technically include as well) then the ratio will have a defined and finite deviation/variance.

• Note that the image above has $Z_1$ and $Z_2$ become equal for large $n$. But this will only hold for $\mu_X=\mu_Y=1$. You will not in general get the same sample variance for $Z_1$ and $Z_2$, this is even true when $\mu_X = \mu_Y$.

  In the image you can use the distance between the iso-lines as a measure for the standard deviation. When the lines are close to each other then changes in $X_n,Y_n$ will make large differences in the statistic. When the lines are far from each other then changes in $X_n,Y_n$ will make small differences in the statistic.

  You can see that neither $X/Y$ or $X-Y$ is the best in every region. $X/Y$ will 'perform' better when $X,Y$ are large and $X-Y$ will 'perform' better when $X,Y$ are small. If you would draw some boundary lines for a hypothesis test like $a < X/Y < b$ and $c < X-Y < d$, then the regions will be different geometries. The statistical test $a < X/Y < b$ will relate to a region in the shape of a cone. The statistical test $c < X-Y < d$ will relate to a region in the shape of a diagonal bar. These regions will not reject/accept all cases the same. One region will be more strict close to the origin, the other will be more strict far away from the origin.