Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have measured the time taken to solve a problem by algorithm $X$ and by algorithm $Y$. It takes a quite long time, so I have only 10 data for each algorithm: $$ X : ( x_1, x_2, \dots , x_{10}) \\ Y : ( y_1, y_2, \dots , y_{10}) $$ In my paper I reported the ratio $$ r = \frac{\sum_{k=1}^{10} x_k }{\sum_{k=1}^{10} y_k } $$ and provided a Wilcoxon signed rank test results. However the reviewer of my paper asks for variance across the 10 ratios.

The distribution of a ratio is intuitively highly asymmetrical around 1. (you have only the interval $(0; 1)$ to capture the fact that algorithm $X$ is faster, but the entire $(1 ; \infty)$ to capture the fact that $Y$ is faster). So even a well estimated standard deviation can be of little use.

This question is closely related to my previous one. The accepted answer proposes to symmetrize the data using the logarithm function and to construct confidence intervals for the ratio. However I have very many ratios (since I tested on many problems) and I am afraid that writing confidence intervals for each ratio would decrease the readability of the paper.

  1. Should I decline to provide the variance of ratios?
  2. Is there some other way to report the variance other than standard error?
  3. If I decline, is there some literature which mentions that it is a bad idea to report the variance of ratios?
share|improve this question

1 Answer 1

up vote 5 down vote accepted

Why the variance? The Wilcoxon signed-rank test does not use the ratios of the observations. Therefore, this quantity would not help to understand the results.

Note also that the variances can vary a lot even if the distribution of $X$ and $Y$ is the same:

x = rgamma(10,1,1)
y = rgamma(10,1,1)
mean(x)/mean(y)
var(x/y)

and

vars = vector()
for(i in 1:1000){
vars[i] = var(rgamma(10,1,1)/rgamma(10,1,1))
}

plot(vars)
hist(vars)
summary(vars)

If the reviewer did not provide a specific reason for including the variances of the ratios, and you have many experiments, you could simply appeal to avoiding this for the sake of brevity. Be polite.

In your case, I would rather ask whether the interest is in the difference of the means of $X$ and $Y$ or $Pr(X<Y)$, considering that $X$ and $Y$ are paired.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.