23
$\begingroup$

I have some data in [0,1] which I would like to analyze with a beta regression. Of course something needs to be done to accommodate the 0,1 values. I dislike modifying data to fit a model. also I don't believe that zero and 1 inflation is a good idea because I believe in this case one should consider the 0's to be very small positive values (but I don't want to say exactly what value is appropriate. A reasonable choice I believe would be to pick small values like .001 and .999 and to fit the model using the cumulative dist for the beta. So for observations y_i the log likelihood LL_iwould be

 if  y_i < .001   LL+=log(cumd_beta(.001))
 else if y_i>.999  LL+=log(1.0-cum_beta(.999))
 else LL+=log(beta_density(y_i))

What I like about this model is that if the beta regression model is valid this model is also valid, but it removes a bit of the sensitivity to the extreme values. However this seems to be such a natural approach that I wonder why I don't find any obvious references in the literature. So my question is instead of modifying the data, why not modify the model. Modifying the data biases the results (based on the assumption that the original model is valid), whereas modifying the model by binnning the extreme values does not bias the results.

Maybe there is a problem I am overlooking?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ It isn't really possible to give a good answer to this question without knowing more about the particular problem. The key question is whether the exact zeros and ones are generated by a different process to that which generates the data in (0,1). A classic example is rainfall, where there are exact zeros reflecting days where it doesn't rain. In your application are zeros and ones "special" in some way? $\endgroup$
    – Dikran Marsupial
    Oct 10, 2012 at 11:31
  • $\begingroup$ Related/duplicate: stats.stackexchange.com/questions/48028. $\endgroup$
    – amoeba
    Feb 17, 2017 at 23:06

5 Answers 5

Reset to default
29
$\begingroup$

According to Smithson & Verkuilen (2006)$^1$, an appropriate transformation is

$$ x' = \frac{x(N-1) + s}{N} $$

"where N is the sample size and s is a constant between 0 and 1. From a Bayesian standpoint, s acts as if we are taking a prior into account. A reasonable choice for s would be .5."

This will squeeze data that lies in $[0,1]$ to be in $(0,1)$. The above quote, and a mathematical reason of the transformation is available in the [paper's supplementary notes].

Reference:
  1. Smithson, M. & Verkuilen, J. A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychol. Methods 11, 54–71 (2006). DOI: 10.1037/1082-989X.11.1.54
Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ +1 .. But could you fix the first link or at least cite the paper so we can find it independently? $\endgroup$
    – whuber ♦
    Jun 28, 2012 at 21:19
  • 2
    $\begingroup$ But that does not answer my question. I am well aware that one can transform the data. My questions is why not transform the model instead? $\endgroup$
    – dave fournier
    Jun 29, 2012 at 13:00
  • 2
    $\begingroup$ Dave, then please edit your question to reflect this: currently, it reads as if you are looking for a way to transform the data. In the process it would help for you to indicate what you think the difference is between a data transformation and a change of model, because if there is one, it is subtle. $\endgroup$
    – whuber ♦
    Jun 29, 2012 at 13:23
  • $\begingroup$ @davefournier, If you read the paper Cam sites it in part addresses your question. They also give alternate model recommendations (see page 69), and part of the recommendations hinge on the nature of the data. Your adjusted likelihood looks like the "mixed discrete-continous process" (mentioned towards the end of page 69). It may also be the case the Tobit model will be satisfactory given your data, although it would be best to see other references for the appropriateness of the Tobit model, like Scott Long's book on categorical regression. $\endgroup$
    – Andy W
    Jun 29, 2012 at 15:43
  • 1
    $\begingroup$ But they don't take this approach. They propose a different model, a mixed discrete continuous process. That is very different from binning the extreme values. As I said before if the beta model is valid then the binning model is valid. If the discrete continuous model is valid then the beta model is invalid. I suspect that they were mostly driven in their analysis by the kinds of mixed models they could fit with their software. The binned beta mixed model is a bit more difficult to fit. $\endgroup$
    – dave fournier
    Jun 29, 2012 at 15:59
6
$\begingroup$

I think the actual "correct" answer to this question is zero-one inflated beta regression. This is designed to handle data that vary continuously on the interval [0,1], and allows many real 0's and 1's to be in the data. This approach fits three separate models in a bayesian context, similar to what @B_Miner proposed.

Model 1: Is a value a discrete 0/1, or is the value in (0,1)? Fit with a bernoulli distribution.

Model 2: Fit discrete subset with a bernoulli distribution.

Model 3: Fit (0,1) subset with beta regression.

For prediction, the first model results can be used to weight the predictions of models 2 and 3. This can be implemented within the zoib R package, or home-brewed in BUGS/JAGS/STAN/etc.

Improve this answer
$\endgroup$
3
$\begingroup$

Dave,

A common approach to this problem is to fit 2 logistic regression models to predict whether a case is 0 or 1. Then, a beta regression is used for those in the range (0,1).

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ could you provide an example? or a paper discussing this in more detail? $\endgroup$
    – user1607
    May 25, 2019 at 12:22
2
$\begingroup$

The beta distribution follows from the sufficient statistics $(\log(x), \log(1-x))$. Do those statistics make sense for your data? If you have so many zeros and ones, then it seems doubtful that they do, and you might consider not using a beta distribution at all.

If you were to choose the sufficient statistic $x$ instead (over your bounded support), then I believe you end up with a truncated exponential distribution, and with $(x,x^2)$ a truncated normal distribution.

I believe that both are easily estimated in a Bayesian way as they are both exponential families. This is a modification of the model as you were hoping.

Improve this answer
$\endgroup$
-1
$\begingroup$

My experience with regression modeling, in general, involves also the exercise of judgment and in that regard, it is something of an art, especially when it comes to constructing good forecasting models. My observation is that actually parsimonious models appear to be superior, overfitting a model is not the apparent optimal best approach. My experience includes time series modeling and reading of the works of Box-Jenkins both of which expound judgment based parsimonious modeling.

If one reads the original work on beta regression, “BETA REGRESSION FOR MODELLING RATES AND PROPORTIONS”, the authors suggest several link functions or the use of no link function. The idea that the odds ratio link function is the most appropriate, or the only choice, given its singularity at O and 1, with many such (or near such) points in the data, is just bad judgment in my view.

I just noticed comments by Neil, "Do those statistics make sense for your data? If you have so many zeros and ones, then it seems doubtful that they do.." which appears to echo my point.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.