Testing the significance of differences between ratios with small samples

Question

This is for a friend of mine. As an econometrician used to rely on large samples for inference, I find myself unsure whether the answer I have in mind is the best.

Suppose we have four continuous random variables, X, Y, W and Z. We have a small (5 to 10 observations) iid sample from each.

We want to test whether $\frac{E(X)}{E(Y)} = \frac{E(W)}{E(Z)}$.

My first reaction is to consider taking the logs, so as to think in terms of differences rather than ratios; but I fear this would lower the power of tests that rely on the normality of the variables (X, Y, W, Z are probably approximately normal; but then their logarithm is quite far from normal).

What are your thoughts on this? Any advice would be greatly appreciated!

Answer 1

You say that the variables are probably normally distributed, so you could test this by simulation (parametric bootstrap). Choose a test statistic like the ratio of the ratios of the sample means. Now generate data under the null hypothesis that the 2 ratios are equal, you will need to make some additional assumptions, i.e. what the variances are, but you can vary these assumptions to see what effect they have.

Now repeate the simulation a bunch of times calculating the test statistic each time. Your P-value is the proportion of simulated test statistics that are more extreeme than the observed test statistic.

Answer 2

Because the distributions of the random variables are not given, we must assign them. Now one is free to do this assignment however they want, so long as it conforms to the information you specify. Now we have three "pieces" of information so far:

1. all random variables are positve
2. all expectations of the random variables exist and are finite
3. the results of an experiment (some "data", denoted by $D$)

One general principle for assigning probability distributions is maximum entropy. Being an economist, you may not have heard of this. Arnold Zellner's work on Bayesian Method of Moments (BMOM) gives some economic applications of this principle, and a bit of an explanation. Edwin Jaynes is the "king" of this method. Searching these two authors should give good explanations of these methods.

Now in order to apply the principle, we require a reference measure $m_{X}(x)$. The most frequently used one is the uniform distribution $m_{X}(x)\propto 1$, and it usually works well. Now what we then do is constrain the moment of our new distribution $p_{X}(x)$ to have mean $\mu_{X}$, while having maximum entropy. The solution is the exponential distribution:

$$p_{X}(x|\mu_{X}I)=\frac{1}{\mu_{X}}\exp\left(-\frac{x}{\mu_{X}}\right)$$

We will have similar distribution for $W,Y,Z$. The notation $I$ is to indicate that this is the information (assumptions) that we are using. But we do not know the actual value $\mu_{X}$, only that it is relevant to the problem. So we need to multiply by a prior and integrate it out of the resulting equations. If you have information about the population means, this is where you specify it, otherwise the non-informative prior is given by the jeffreys prior

$$p(\mu_{X}|I)=\frac{1}{log\left(\frac{U}{L}\right)\mu_{X}}\;\;\;\;\;\;\; L<\mu_{X}<U$$

The bounds $L$ and $U$ are a safety device for now. If we can take the limits $L\to 0,U\to\infty$ we will, but at the end of the calculation, not at the start.

If this is treated as a parameter estimation problem, then we will seek as our final result, the probability:

$$Pr\left(\frac{\mu_{X}}{\mu_{Y}}>\frac{\mu_{W}}{\mu_{Z}}|DI\right)$$

With the direction of the inequality going the way that is supported by the data. You can interpret this as the amount of support against the hypothesis. But in order to get this we first need the joint posterior:

$$p(\mu_{X}\mu_{Y}\mu_{W}\mu_{Z}|DI)\propto p(\mu_{X}\mu_{Y}\mu_{W}\mu_{Z}|I)P(D|\mu_{X}\mu_{Y}\mu_{W}\mu_{Z}I)$$

All quantities have been assigned, so we just plug them in:

$$p(\mu_{X}\mu_{Y}\mu_{W}\mu_{Z}|I)=\frac{1}{\left[log\left(\frac{U}{L}\right)\right]^{4}\mu_{X}\mu_{W}\mu_{Y}\mu_{Z}}$$ $$P(D|\mu_{X}\mu_{Y}\mu_{W}\mu_{Z}I)=\frac{1}{\mu_{X}^{n_{x}}\mu_{W}^{n_{w}}\mu_{Y}^{n_{y}}\mu_{Z}^{n_{z}}}\exp\left(-\frac{n_{x}\overline{x}}{\mu_{X}}-\frac{n_{w}\overline{w}}{\mu_{W}}-\frac{n_{y}\overline{y}}{\mu_{Y}}-\frac{n_{z}\overline{z}}{\mu_{Z}}\right)$$

Where $n_{i}$ is the sample in the ith group, and the bar indicates an average over that group. Now you may not recognise them, but this posterior is proportional to the product of 4 independent inverse gamma distributions, with $(\mu_{X}|DI)\sim IGa(\mu_{X}|n_{x},n_{x}\overline{x})$ and similarly for $\mu_{W},\mu_{Y},\mu_{Z}$. Because inverse gamma distribution is proper on the positive real axis, we can take the limits $L\to 0,U\to\infty$ without harm. Now you could try to evaluate the integral required for $Pr\left(\frac{\mu_{X}}{\mu_{Y}}>\frac{\mu_{W}}{\mu_{Z}}|DI\right)$, but I would instead use monte carlo to evaluate this probability, because generating inverse gamma random variables is very cheap, and getting the range of integration correct seems to be difficult (I wasn't sure how to do it). Note that because you are monte-carlo sampling directly from the posterior, should be efficient (unlike monte-carlo from a non-informative prior).

The easiest way to program this is to use a gamma inverse cdf function, $ginv()$ and the random number generator $rand()$, then take your monte-carlo sample as $\frac{1}{ginv(rand)}$ for each variable. Let me know if you need more explanation.