2
$\begingroup$

I have a dataset, let's say of counts of apples and oranges. I would like to have a metric that equally reflects the ratio of apples to oranges but if I simply use the ratio:

apple count + 1 : orange counts +1

I get very large numbers when I have more apples than oranges but very small numbers when I have more oranges than apples. I would like a ratio that is more normally distributed so that when apples = oranges my ratio = 1 but my max value is 2 and min value is 0.

Any suggestions on how to achieve this?

$\endgroup$
1
  • $\begingroup$ Note: you can't be Normally distributed and bounded by 0 and 2. $\endgroup$ Feb 23, 2016 at 15:49

3 Answers 3

5
$\begingroup$

Let's forget apples and oranges and generalise painlessly to counts $n_1 + n_2 =: n$ and work with the proportions $p_1 = n_1/n, p_2 = n_2/n$.

As your own answer implies, the problem with $\ln (n_1/n_2) = \ln n_1 - \ln n_2 = \ln p_1 - \ln p_2$ is that either count $n_1, n_2$ could be zero. Hence over many years there have been proposals of various fudges of the form $\ln [(n_1 + c)/(n_2 + k)] = \ln (n_1 + c) - \ln (n_2 + k)$ for positive $c, k$ where $c$ and $k$ need not be different.

However, a fudge-free alternative is to use some folded power, say $p_1^\lambda - p_2^\lambda$, so that for example $\lambda = 1/2$ gives folded square roots and $\lambda = 1/3$ gives folded cube roots. Positive features of such folded powers include

  1. symmetry of definition, so that folded powers for $\lambda > 0$ range from $-1$ to $1$ (and so could be translated to the interval from $0$ to $2$ simply by adding $1$).

  2. being defined simply for $n_1 = 0$ or $n_2 = 0$ or both.

There is no guarantee that the results will be normally distributed, nor is there for any transform of $n_1$ and $n_2$. However, the results are likely to be closer to a normal distribution, and indeed to symmetry, than the results of $n_1/n_2$ would be.

More on folded powers at What is the most appropriate way to transform proportions when they are an independent variable? which in turn gives further references.

$\endgroup$
3
  • $\begingroup$ From a purely methodological perspective, I like the suggestion of using folded powers. However, how is the resulting metric to be interpreted? So, one of the beauties of using a natural log transform is that in regression models, depending on the specification, the resulting coefficients can have a direct interpretation as elasticities. Are folded powers a statistical version of many medical "cures" where the cure is worse than the disease? $\endgroup$
    – user78229
    Feb 23, 2016 at 20:54
  • $\begingroup$ The competitor to folded powers is the logit, not the log. Indeed, the logit is a limiting case of folded powers Without doubt, when logits apply, they are usually to be preferred. When modelling, logits not being defined for 0 or 1 doesn't bite because the logit is a link function applying to the predicted mean, not a transformation applied to the original data. All that said, this question requests a bounded transformation more nearly normal than ratios, and implicitly one not thrown off by zeros, and folded powers are a solution; the logit doesn't meet this query. $\endgroup$
    – Nick Cox
    Feb 23, 2016 at 21:04
  • $\begingroup$ It's naturally possible to ask the OP why boundedness is important, but we don't yet have any detail on what lies downstream of this and underlies the requests. $\endgroup$
    – Nick Cox
    Feb 23, 2016 at 21:06
2
$\begingroup$

Consider the transformation $$ \theta = \text{log}\,\frac{\pi_\text{oranges}}{\pi_\text{apples}} $$ where $\pi_{\text{oranges}}$ is the probability of seeing an orange, or if you prefer, the true proportion of oranges in a population of oranges and apples.

This is just the population version of the log of your ratio without the extra 1s. We'll get to those in a minute.

$\theta$ is a logit and thus zero when there are the same proportion of apples as oranges, and it increases (decreases) with the proportional increase (decrease) in oranges. Naturally it's not defined when there are no oranges and/or apples.

Assume a prior over $\pi_\text{oranges}$ $$ p(\pi_\text{oranges}) = \text{Beta}(a,b) $$ Setting $a=b=1$ would imply that all values of $\pi_\text{oranges}$, from 0 to 1 are equally likely.

With this prior, the posterior distribution $p(\pi_\text{oranges} \mid \text{apples}, \text{oranges})$ is exactly $$ \text{Beta}(\text{oranges}+a,\text{apples}+b) $$ and when there are more than a handful each of apples and oranges, the corresponding posterior for $\theta$ is, to good approximation (Lindley, 1965), Normal with mean $$ \text{log}\,\frac{\text{oranges}+a}{\text{apples}+b} $$ which really is your ratio, and variance $$ \frac{1}{\text{oranges}+a} + \frac{1}{\text{apples}+b} $$ So $\theta$ is a transformation to Normality that also makes sense.

If you don't like the whole prior-posterior business, you can think about the log of your original measure simply as a regularized estimator of $\theta$. It's shrunk towards 0, but decreasingly so as more fruit appears.

$\endgroup$
0
$\begingroup$

I am trying: $$ \text{log}\,\Big[\frac{apple count + 1}{orange count +1}\Big] $$ but is this transformation justifiable?

$\endgroup$
2
  • $\begingroup$ Yes, sure. This method is used for example for Bayes' factors representation. $\endgroup$ Feb 23, 2016 at 13:48
  • $\begingroup$ I think this really adds more information to your question (we always like to know what people have tried) rather than standing as an answer in its own right. I suggest you edit in to your question. There is an "edit" button at the bottom of it. $\endgroup$
    – Silverfish
    Feb 23, 2016 at 14:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.