1
$\begingroup$

I need to use the ratio of two variables as the dependent variable in a regression. Both variables are normally distributed but with positive values. I can either center them or use as it is.

If I center them: the ratio of two centered normals is a Cauchy distribution. Is it therefore meaningful to do a "Cauchy" regression (e.g. using JAGS or Stan?)

If I leave as they are: What would be the distribution of their ratio?

$\endgroup$
7
  • 2
    $\begingroup$ Why do you "need to use the ratio of two variables as the dependent variable"? If the variables are strictly positive the difference between the logarithms of their values would seem to be a better measure. And why do you care about the distribution of the outcome variable itself, rather than how the values of the outcome variable depend on the predictor variables? $\endgroup$ – EdM Mar 17 '16 at 14:21
  • $\begingroup$ Thanks EdM. The difference of the logarithms is essentially the logarithm of the ratio. That is why I am asking about ratios. I care about the distribution because it will guide me to how to build my model. For example your suggestion for the difference of the logarithms implies that the ratio would be a log-normal distribution. Is that correct? How can I justify it? $\endgroup$ – George Michaelides Mar 17 '16 at 15:55
  • 1
    $\begingroup$ If you look at the assumptions underlying linear regression you will see nothing about the distribution of the dependent variable per se. Normal distributions of error terms make certain types of inference possible. For a linear regression model you may want to investigate transformations of variables that provide linear relations to outcome and normal distributions of errors, but knowing the distribution of the raw outcomes themselves has little value for building a model. $\endgroup$ – EdM Mar 17 '16 at 17:14
  • 1
    $\begingroup$ Also, note that if you have necessarily positive variables they cannot be distributed normally, as values sampled from a true normal distribution can cover the entire real axis. The rationale for working in log scales rather than with ratios for your outcome variable is that the error terms in a regression model might then be much better behaved, perhaps amenable to some type of generalized linear model. $\endgroup$ – EdM Mar 17 '16 at 17:24
  • $\begingroup$ Thanks. This is exactly what I want to understand. What is the distribution of the residuals if the dependent variable is calculated as the difference of the logarithms of two normal distributions. Can I expect it to be normal? If yes - why? If not, what type of generalized linear model would be more appropriate? (and why?) $\endgroup$ – George Michaelides Mar 17 '16 at 17:34
2
$\begingroup$

You do not need to know the distribution of the dependent variable to design a useful regression model. One introduction to the assumptions of linear regression is available here. You need to determine whether there is a linear relation between the dependent variable and the predictor variables, or transform the variables in some way that the relation is linear. If you want further to do inference, this is simplest if the error terms are independent of the predicted values and have normal distributions in their (potentially transformed) scales.

A logarithmic transformation of the dependent (ratio) variable in your case would make sense if, as is often the case, the errors are proportional to the values. For example, if a variable used for calculating that ratio has an error of $\pm$ 3%, then the log transformation would provide errors in the log-transformed scale (measurement errors, at least) independent of the values. That would also make the distribution of (measurement) errors in the log of the ratio of the two variables independent of the value of the ratio, absent some interaction between them. Whether that will also provide normally distributed residual errors in your particular model cannot be predicted, as the residual distribution will also depend on how well the linear model captures the underlying phenomena. This page is an entry into the issues involved in choosing and using log transformations.

Note that normality of errors is perhaps the least important of all the assumptions for a linear model, unless you need to calculate p-values and the like directly from the regression and for some reason you can't use techniques like bootstrapping to estimate confidence intervals. Some type of generalized linear model, which allows for variance to depend on values, might be appropriate for your problem. The choice of generalized model would then be based on the particular way that you think that error terms will depend on the values and thus is specific to the problem at hand.

$\endgroup$
1
$\begingroup$

What is the distribution of the ratio of two normals?

A related question is A/B testing ratio of sums The following is from a part of an answer to that question.

(You state that both variables are positive. And thus you do not have real normal distributions, which can take the value zero. You will have something that resembles the Cauchy distribution, but it does not have the undefined mean.)

For two (asymptomatically) Gaussian distributed variables (that are potentially correlated) you can use the Delta method or use an exact expression to express the ratio distribution.

The use of the Delta method for the estimation of ratio's is described here. The result of this application of the Delta method actually coincides with an approximation of Hinkley's result, an exact expression for the ratio of two correlated normal distributed variables (Hinkley D.V., 1969, On the Ratio of Two Correlated Normal Random Variables, Biometrica vol. 56 no. 3).

For $Z = \frac{X}{Y}$ with $$ \begin{bmatrix}X\\Y\end{bmatrix} \sim N\left(\begin{bmatrix} \mu_x \\ \mu_y \end{bmatrix} , \begin{bmatrix} \sigma_x^2 & \rho \sigma_x \sigma_y \\ \rho \sigma_x \sigma_y & \sigma_y^2 \end{bmatrix} \right) $$ The exact result is: $$ f(z) = \frac{b(z)d(z)}{a(z)^3} \frac{1}{\sqrt{2\pi} \sigma_X\sigma_Y} \left[ \Phi \left( \frac{b(z)}{\sqrt{1-\rho^2}a(z)} \right) - \Phi \left( - \frac{b(z)}{\sqrt{1-\rho^2}a(z)} \right) \right] + \frac{\sqrt{1-\rho^2}}{\pi \sigma_X \sigma_Y a(z)^2} \exp \left( -\frac{c}{2(1-\rho^2)}\right) $$ with $$ \begin{array}{} a(z) &=& \left( \frac{z^2}{\sigma_X^2} - \frac{2 \rho z}{\sigma_X \sigma_Y} + \frac{1}{\sigma_Y^2} \right) ^{\frac{1}{2}} \\ b(z) &=& \frac{\mu_X z}{ \sigma_X^2} - \frac{\rho (\mu_X+ \mu_Y z)}{ \sigma_X \sigma_Y} + \frac{\mu_Y}{\sigma_Y^2} \\ c &=& \frac{\mu_X^2}{\sigma_Y^2} - \frac{2 \rho \mu_X \mu_Y + }{\sigma_X \sigma_Y} + \frac{\mu_Y^2}{\sigma_Y^2}\\ d(z) &=& \text{exp} \left( \frac {b(z)^2 - c a(z) ^2}{2(1-\rho^2)a(z)^2}\right) \end{array}$$ And an approximation based on an assymptotic behaviour is: (for $\mu_Y/\sigma_Y \to \infty$): $$ F(z) \to \Phi\left( \frac{z - \mu_X/\mu_Y}{\sigma_X \sigma_Y a(z)/\mu_Y} \right) $$ You end up with the Delta method result when you insert the approximation $a(z) = a(\mu_X/\mu_Y)$ $$a(z) \sigma_X \sigma_Y /\mu_Y \approx a(\mu_X/\mu_Y) \sigma_X \sigma_Y /\mu_Y = \left( \frac{\mu_X^2\sigma_Y^2}{\mu_Y^4} - \frac{2 \mu_X \rho \sigma_X \sigma_Y}{\mu_Y^3} + \frac{\sigma_X^2}{\mu_Y^2} \right) ^{\frac{1}{2}}$$

(Sidenote: The above quote has been updated with some references to earlier descriptions and simpler descriptions. An earlier description of the exact expression was given by George Marsaglia 1965 in the JASA Vol. 60, No. 309. A simple modern description is given in 2006 in Jstatsoft Volume 16 Issue 4)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.