24
$\begingroup$

Beta regression (i.e. GLM with beta distribution and usually the logit link function) is often recommended to deal with response aka dependent variable taking values between 0 and 1, such as fractions, ratios, or probabilities: Regression for an outcome (ratio or fraction) between 0 and 1.

However, it is always claimed that beta regression cannot be used as soon as the response variable equals 0 or 1 at least once. If it does, one needs to either use zero/one-inflated beta model, or make some transformation of the response, etc.: Beta regression of proportion data including 1 and 0.

My question is: which property of beta distribution prevents beta regression from dealing with exact 0s and 1s, and why?

I am guessing it is that $0$ and $1$ are not in the support of beta distribution. But for all shape parameters $\alpha>1$ and $\beta>1$, both zero and one are in the support of beta distribution, it's only for smaller shape parameters that the distribution goes to infinity at one or both sides. And perhaps the sample data are such that $\alpha$ and $\beta$ providing best fit would both turn out to be above $1$.

Does it mean that in some cases one could in fact use beta regression even with zeros/ones?

Of course even when 0 and 1 are in the support of beta distribution, probability of observing exactly 0 or 1 is zero. But so is the probability to observe any other given countable set of values, so this cannot be an issue, can it? (Cf. this comment by @Glen_b).

$\hskip{8em}$beta distribution

In the context of beta regression, beta distribution is parameterized differently, but with $\phi=\alpha+\beta>2$ it should still be well-defined on $[0,1]$ for all $\mu$.

enter image description here

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Interesting question! I don't have any answer besides the points already made by Kevin Wright. I guess that exact zeros and ones in probabilities are pathological cases (like in logistic regression) so are not that interesting since they should not happen. $\endgroup$
    – Tim ♦
    Feb 17, 2017 at 20:10
  • 1
    $\begingroup$ @Tim Well, I don't know if they should or shouldn't happen, but they do happen quite often, otherwise people would not ask questions on how to deal with 0s and 1s in beta regression, would not write papers about 0-and-1 inflated beta models, etc. Anyway, I am still hoping for a more detailed answer than Kevin's. One should at least explain how these terms in the log-likelihood arise. $\endgroup$
    – amoeba
    Feb 17, 2017 at 21:50
  • 1
    $\begingroup$ Update: it's probably because if 0 and 1 are in the support then PDF at these points is equal to zero, meaning that the likelihood of observing these values is zero. I would still like to see an answer explaining this carefully. $\endgroup$
    – amoeba
    Feb 17, 2017 at 23:39
  • $\begingroup$ So, what distribution should one use then when the response variable assumes values in, say, $[0, \infty)$? $\endgroup$
    – Confounded
    Oct 31, 2018 at 0:19

2 Answers 2

Reset to default
21
$\begingroup$

Because the loglikelihood contains both $\log(x)$ and $\log(1-x)$, which are unbounded when $x=0$ or $x=1$. See equation (4) of Smithson & Verkuilen, "A Better Lemon Squeezer? Maximum-Likelihood Regression With Beta-Distributed Dependent Variables" (direct link to PDF).

Improve this answer
$\endgroup$
5
  • 4
    $\begingroup$ Thanks. Here is the direct PDF link to the paper. I can see that Eq. (4) will break down as soon as $y_i=0$ or $y_i=1$, but I still don't understand why this happens in the general scheme of things. $\endgroup$
    – amoeba
    Feb 17, 2017 at 14:30
  • 5
    $\begingroup$ (+1) Amoeba, just look at the pdf: for every Beta distribution, the densities at $0$ and $1$ are either $0$ or $+\infty$. In either case, the log likelihood will be undefined. Equivalently, as soon as there is a single $0$ or $1$ response, all values of the likelihood can be only zero, infinity, or indeterminate and there will be a nontrivial set of Beta parameters for which the minimum value of the likelihood is realized. Thus practical calculation is precluded and the model is not identifiable (in a severe sense). $\endgroup$
    – whuber ♦
    Feb 18, 2017 at 14:42
  • 1
    $\begingroup$ Together with @whuber's comment (that I did not notice until now), this does answer the question. The main point is that for the parameter values I was asking about, $0$ and $1$ have zero likelihood. $\endgroup$
    – amoeba
    Feb 19, 2017 at 22:40
  • 1
    $\begingroup$ @whuber The reason I got confused, is that there is probability zero to observe $0$ but there is also probability zero to observe, say, $0.5$ (let's take beta with $\alpha=\beta=2$ for concreteness). Nevertheless, $0.5$ is consistent with the model, but $0$ is not, and it's because the likelihood of observing $0.5$ is not zero but the likelihood of observing $0$ is... $\endgroup$
    – amoeba
    Feb 19, 2017 at 22:56
  • 5
    $\begingroup$ @amoeba The likelihood depends on the probability density, not the probability itself. Sometimes, one can avoid this issue either by considering each observation to include the probability of a tiny but finite (not infinitesimal) interval (determined, e.g., by the precision of the measurement) or by convolving the Beta distributions with a very narrow Gaussian (which eliminates the zero and infinite densities). $\endgroup$
    – whuber ♦
    Feb 19, 2017 at 23:23
2
$\begingroup$

besides the fact that the reason comes in practice from the presence of $log(x)$ and $log(1-x)$, I will try to complement the answer to the question by trying to frame the underlying reason why this happens.

as a matter of fact, the beta distribution is "often used to describe the distribution of a probability value" (wikipedia). It is the distribution of the possible tendencies $p$ of a binomial distribution, knowing the observation of $N$ independent binary draws of a random variable.

As a result, in my understanding of beta regression, 0s and 1s would intuitively correspond to (infinite) sure results.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.