This question already has an answer here:

I hope that this request will make sense. I am not extremely proficient in these kinds of stats, so please also excuse my limited vocabulary.

I have two datasets, we'll call A and B, and for each dataset I am calculating a statistic that is the ratio of two means, i.e. $\%_A=\bar{x}_A/\bar{y}_A$ and $\%_B=\bar{x}_B/\bar{y}_B$. What I want to figure out is whether the ratio for dataset A is statistically distinguishable from the ratio for dataset B (that is, looking at the ratios visually, they seem to be different, but can I actually say with any confidence that they are different based on the number of observations I have for each dataset?).

My understanding is that if I calculated the error for each mean, I could simply look to see if the error bars overlap - if they don't overlap then the difference might be real, but if they do overlap then I can't say if the difference is real or not. I would normally determine error by simply calculating standard error iof the mean (STDDEV/SQRT(num_obs) ), but I do not know how, or if, standard error would propogate when taking the ratio of two means this way.

Perhaps I am taking completely the wrong approach. I would appreciate any guidance you can offer.


marked as duplicate by kjetil b halvorsen, Michael R. Chernick, Siong Thye Goh, Peter Flom - Reinstate Monica Jun 12 at 9:32

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ If you have the data, you can perform a bootstrap test. $\endgroup$ – Noah Jun 11 at 20:16
  • 1
    $\begingroup$ The relationship between datasets is important. Are the independent (for example, A for male, B for female) or dependent (such as A for year 2018 and B for year 2019 from the same people)? $\endgroup$ – user158565 Jun 11 at 20:26
  • $\begingroup$ @Noah, I do have the data the means are based on. Would you mind elaborating on performing a boostrap test, or do you have any resources I could turn to? I am familiar with the concept of bootstrapping, but not how to apply it here. $\endgroup$ – AlJones1816 Jun 11 at 20:46
  • $\begingroup$ @user158565, I believe the datasets are independent, (the same observations being made in two different US States). $\endgroup$ – AlJones1816 Jun 11 at 20:50
  • $\begingroup$ To bootstrap: draw sample with replacement from your data and compute the relevant statistic (e.g., the difference between the ratios). Do this ~1000 times. The 2.5th and 97.5th percentiles of the distribution of computed statistics are a confidence interval for the original estimate. If the interval excludes the null value of the statistic, you can reject the null hypothesis that the ratios are equal in the population. $\endgroup$ – Noah Jun 11 at 23:21

You could set up your data in the following way: have a variable, $Z$, be your outcome variable, and have two columns, $X$ and $A$. Column $X$ is equal to 1 if the observation is in group $X$ and 0 otherwise, and Column $A$ is equal to 1 if the observation is in group $A$ and 0 otherwise.

You can run a generalized linear model with a log link, as follows: $$log(\mu_{Z|X,A}) = \beta_0 + \beta_1 X + \beta_2 A + \beta_3 XA$$ which is equivalent to $$\mu_{Z|X,A} = e^{\beta_0 + \beta_1 X + \beta_2 A + \beta_3 XA}$$

$ \mu_{Z|X=1, A=1}$ is the same thing as $\bar{x}_A$, $\mu_{Z|X=0, A=1}$ is the same thing as $\bar{y}_A$, and so on. So, we can rewrite the means in terms of the coefficients form this model: \begin{align}\bar{x}_A &= e^{\beta_0 + \beta_1 + \beta_2 + \beta_3}\\ \bar{x}_B &= e^{\beta_0 + \beta_1}\\ \bar{y}_A &= e^{\beta_0 + \beta_2}\\ \bar{y}_B &= e^{\beta_0} \end{align} So, we can reexpress the ratios of these means as ratios of the coefficients: \begin{align} \frac{\bar{x}_A}{\bar{y}_A}&=\frac{e^{\beta_0 + \beta_1 + \beta_2 + \beta_3}}{e^{\beta_0 + \beta_2}}=e^{\beta_1 + \beta_3}\\ \frac{\bar{x}_B}{\bar{y}_B}&=\frac{e^{\beta_0 + \beta_1}}{e^{\beta_0}}=e^{\beta_1} \end{align} So, testing $\frac{\bar{x}_A}{\bar{y}_A}=\frac{\bar{x}_B}{\bar{y}_B}$ is equivalent to testing if $e^{\beta_1 + \beta_3}=e^{\beta_1}$, which is equivalent to testing whether $e^{\beta_3}=1$ or $\beta_3=0$. This is a test that is given in the model output and doesn't require any additional calculation.

If all four means come from independent observations, this will work as written. If there is dependence among some units (e.g., all each unit in the sample has both an $x$ and a $y$ value), add an extra column for unit ID and add that as a cluster and use a cluster-robust standard error.


Not the answer you're looking for? Browse other questions tagged or ask your own question.