2
$\begingroup$

The malleability of the beta distribution pleases me; it can be symmetric, asymmetric, platykurtic and so on, as the following picture shows us:

enter image description here

I thought it would be interesting to use it to model the errors in regression problems, but its support is only defined to be $(0,1)$ so I would like to know if there is a "beta" distribution with support $(-\infty,\infty)$ because a error lying on $(0,1)$ is such a strong assumption.

$\endgroup$
7
  • 7
    $\begingroup$ The question seems a little vague / the form of the connection of a beta to distributions on the real line is unclear, as is the proposed use of this in regression (how are you planning to implement a model with this sort of distribution - presumably a conditional distribution for the response - exactly?). There are many distribution families on the real line that have a diverse range of possible skewness and kurtosis. Which 'beta'-like properties do you (a) require, (b) would like but are not required and (c) don't care about? Would a robust regression do instead? $\endgroup$
    – Glen_b
    Nov 29, 2021 at 2:50
  • $\begingroup$ $e_i\sim real beta$, Id like a generic distribution which can be symetric, non symetric (to both sides), exponential. I know there are an enough number of distributions for it but Im looking for a one which can deal with most of situations like gradient boosting almost always give a good cross validate score. (I used gb only as example it is not even a distribution) $\endgroup$ Nov 29, 2021 at 5:29
  • 1
    $\begingroup$ Can you clarify what you mean by "exponential" for something on the real line? The properties you mention seem to have nothing clearly to do with the beta - except that the beta, too, can be skewed or symmetric, but so can any number of other families of distributions, it's not specifically a property of the beta. Might the distribution of the logit of a beta variate satisfy your requirements? $\endgroup$
    – Glen_b
    Nov 29, 2021 at 5:32
  • $\begingroup$ $real beta$ almost equal to some $exp$, with $P(real beta<0)$ near to 0, $\endgroup$ Nov 29, 2021 at 6:12
  • 1
    $\begingroup$ I still don't follow what you mean by that. Do you mean that the density of the tail would be close to exponential (e.g. such that as $x\to\infty$ for some $k$, $f(x).e^{kx}$ approaches a constant, and similarly on the other side, mutatis mutandis?) Or something else? $\endgroup$
    – Glen_b
    Nov 29, 2021 at 7:03

3 Answers 3

12
$\begingroup$

One way to make a "real-valued beta distribution" would be to transform the interval $(0,1)$ onto the real line. One way of doing that is the logistic function $$ \text{logit}(x)= \log(\frac{x}{1-x}) $$ known from, for example, logistic regression. So if $X \sim \mathcal{Beta}(\alpha,\beta)$, let $$ Y=\text{logit}(X)= \log(\frac{X}{1-X}). $$ Then $Y$ will be a random variable with range on the full real line, with a distribution we can call logistic-Beta.

By standard transformation methods we find the density function of $Y$ as $$ f_Y(y) =\frac{e^{\alpha y}}{B(\alpha, \beta) (1+e^y)^{\alpha+\beta}} $$ where $B$ is the beta function. This will give a flexible family of (unimodal) distributions on the real line. In this comprehensive reference that I just found, this distribution is called the Beta-Logistic Distribution. That is a useful search term!

An example is below:

 Two logistic-Beta densities

The same transformation could be used with mixtures of betas, for instance, to get even multi-modal distributions. In reality, this example is a special case of the log-ratio transformation used with Compositional data, see also How to perform isometric log-ratio transformation.

Transforming a beta random variable to the positive line (in a specific way) only gives the beta prime distribution, a recent post with an example is Distribution of the exponential of an exponentially distributed random variable?.

Code use for the figure is below:

dlogisticBeta <- function(x, alpha, beta, log=FALSE) {
    stopifnot( (alpha>0)&&(beta>0) )
    logans <- alpha*x - lbeta(alpha,  beta) -
        (alpha + beta)*log1p( exp(x))
    if(log) return(logans) else return(exp(logans))
}

oldpar <- par(mfrow=c(1, 2))
plot( function(x) dlogisticBeta(x, 0.5, 0.5), from=-10,  to=10, col="red", 
     main= expression(paste(alpha == 0.5,",   ",   beta==0.5)) )
plot( function(x) dlogisticBeta(x, 0.5, 3), from=-10,  to=10, col="red",  
     main= expression(paste(alpha == 0.5,",   ",   beta==3)) )
par(oldpar)
$\endgroup$
3
  • 3
    $\begingroup$ The logit of a beta is closely related to the Fisher-z distribution $\endgroup$
    – Glen_b
    Nov 29, 2021 at 7:00
  • $\begingroup$ great approach, I dont know if that transformation is flexible as beta is but think it's pretty enough. $\endgroup$ Nov 29, 2021 at 7:28
  • $\begingroup$ The utility of this distribution for modelling must be much reduced by the lack of location & scale parameters; if observations are on an interval scale - consonant with a distribution having support over the real line - estimates won't be equivariant to changes of the unit of measurement. You can add them in, but then "flexibility" is coming at a standard price - four free parameters. $\endgroup$ Dec 6, 2021 at 9:26
3
$\begingroup$

It seems to me the convex shape the beta distribution can take is a result of the bounded sample space. If we move to a sample space that is bounded on one side then we are dealing with distributions like gamma, Weibull, etc. If we move to an unbounded sample space then we of course have the normal distribution.

All of these distributions, including the beta and scaled beta, can be used in regression via generalized linear models. Rather than developed around an "error term" these models are typically developed by expressing a parameter in the likelihood or probability distribution in terms of a linear predictor. You could re-parameterize a model in terms of its mean or median and express this in terms of a linear predictor.

$\endgroup$
5
  • $\begingroup$ If you get beta support unbounded that will diverge. $\endgroup$ Nov 29, 2021 at 1:17
  • 1
    $\begingroup$ I agree. That is why the normal distribution is often used for an unbounded parameter space. $\endgroup$ Nov 29, 2021 at 2:01
  • $\begingroup$ but normal distribution is very simple and always symmetric regardless what its parameters are I'd like to know some 'adaptative' distribution as the beta is but real valuated. $\endgroup$ Nov 29, 2021 at 3:10
  • 3
    $\begingroup$ (1) Only a subset of Beta distributions has convex PDFs: namely, those with at least one parameter less than $1.$ (2) Distributions with unbounded support can have convex PDFs. The Pareto family is a well-known example. In light of these points, it is difficult to appreciate what your answer is trying to say. $\endgroup$
    – whuber
    Nov 29, 2021 at 14:45
  • $\begingroup$ The support of the Pareto distribution is bounded from below similar to a gamma distribution and can certainly be used for modeling. This distribution is not precluded from my answer. I am suggesting there are lots of distributions the OP can use depending on the support needed, not necessitating transforming a beta distribution. The skewness and convex shapes of these distributions come from the support being bounded. Even if one did form a skewed theoretical continuous distribution having a support that is not bounded from above nor from below it may not have much practical value. $\endgroup$ Nov 30, 2021 at 2:05
1
$\begingroup$

Beta regression is used to model continuous proportions; & that such observations are bounded by 0 & 1 is as reasonable an assumption as it is a strong one (if you'd call it an assumption rather than a definition). As noted in @GeoffreyJohnson's answer, it's the conditional distribution of each observation that's taken to be a beta distribution, not the error. In a typical beta regression you reparametrize the beta distribution with mean $\mu = \frac{\alpha}{\alpha + \beta}$ & precision $\nu = \alpha + \beta$; the joint density $f_\boldsymbol{Y}$ of $n$ independent observations $\boldsymbol{y}$ is given by $$f_\boldsymbol{Y}(\boldsymbol{y})=\prod_i^n\frac{\Gamma(\nu)}{\Gamma(\mu_i\nu)\Gamma\left[(1-\mu_i)\nu\right]} \cdot y_i^{\mu_i\nu-1}(1-y_i)^{(1-\mu_i)\nu-1}$$

The link function $g$ connects the predictor values $\boldsymbol{x}_i$ & the coefficients $\boldsymbol\beta_i$ to the mean for each observation $\mu_i$:

$$g(\mu_i) = \eta(\boldsymbol{x}_i,\boldsymbol\beta_i)=\boldsymbol{x}_i^\mathrm{T}\boldsymbol{\beta}$$

The precision is common across all observations.

Each row of plots below shows therefore how the distribution may be changed by changing the predictor values:

Beta densities from different combinations of mean & precisions

Now there's evident flexibility ('malleability') there, but there are also evident constraints, among them these:

  • $f_Y(y; \mu,\nu)=f_Y(1-y; 1-\mu,\nu)$. This provides a desirable equivariance—if Dr A models the proportion of the day an animal spends asleep & Dr B the proportion it spends awake, we'd hope they'd make substantively identical inferences/predictions.

  • $\operatorname{Var} Y_i =\frac{\mu_i(1-\mu_i)}{1+\nu}$. The variance is at a maximum when $\mu=\frac{1}{2}$, increasing as $\mu\rightarrow 0$ or $\mu\rightarrow 1$. Qualitatively this is what you'd often expect from a continuous proportion, squashing up against the bounds.

  • $\frac{\operatorname{E} (Y_i - \operatorname{E} Y_i)^3}{\left(\sqrt{\operatorname{Var} Y}\right)^3} =\frac{2(1-2\mu_i)\sqrt{1+\nu}}{(2+\nu)\sqrt{\mu(1-\mu)}}$. The skewness is positive when $\mu<\frac{1}{2}$, increasing as $\mu\rightarrow$; it's negative when $\mu>\frac{1}{2}$, decreasing as $\mu\rightarrow 1$; & of course it's zero when $\mu=\frac{1}{2}$. Again, this is qualitatively what you'd often expect.

The point of going into all this is first that in a sense there's no free lunch—you pay (in free parameters) for a certain amount of flexibility, & allocating it to some distributional properties means constraining others—and second that it's not at all clear what you'd want to carry over to modelling a real-valued response, beyond allowing skew & kurtosis to vary in some fashion (as @Glen_b comments).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.