6
$\begingroup$

If I have a set of continuous predictors $X$ and a binary outcome $Y$ and I wanted to build a predictive model of $P(Y|X)$, I would start with a logistic regression model.

However, in my particular case, my $Y$ isn't binary, it's continuous between 0 and 1. Is there a similar Generalized Linear Model that can be applied in this case? My optimistic/naive attempt in R reveals that

set.seed(123)
df <- data.frame(y=runif(8), x1=rnorm(8), x2=rnorm(8))
mod <- glm(y ~ ., data=df, family=binomial('logit'))

# Warning message:
# In eval(expr, envir, enclos) : non-integer #successes in a binomial glm!

rbind(yhat=predict(mod, newdata=df), y=df$y)
#              1         2          3         4         5          6         7           8
# yhat 0.7461449 0.4869853 -0.1092115 1.9854276 0.8328304 -1.3708688 1.0150934 -0.03496334
# y    0.2875775 0.7883051  0.4089769 0.8830174 0.9404673  0.0455565 0.5281055  0.89241904

Note that some of the predictions are outside of $(0,1)$. Any suggestions?

$\endgroup$
  • 3
    $\begingroup$ As to why some of the predictions are outside (0,1): this is because the result of predict.glm() is by default on the link scale, not the response scale. Try rbind(yhat=predict(mod, newdata=df, type="response"), y=df$y) and ?predict.glm, and enlightenment will follow. $\endgroup$ – Stephan Kolassa Nov 16 '12 at 21:58
9
$\begingroup$

If you have "continuous" (seemingly, as they could still be discrete) values in between 0 and 1 there are at least two cases:

  1. They came from a number of independent binary trials and the "continuous" value is the number of successes divided by trials. Then a binomial GLM might be appropriate. In this case you need to fit it in R as glm(cbind(numberSuccesses,numberFailures)~x,family=binomial)
  2. If that is not the case, then you might have something for which a Beta Model might be more appropriate. The link I provided shows how to do that in R.

Note that in R glm(y~x,family=binomial) with a "continuous" $y$ will throw a warning and in general the result will not be the same as in the case with number of successes and trials:

set.seed(1)
successes<-sample(1:10,100,replace=TRUE)
x<-1:100
n<-12
failures<-n-successes

summary(glm(cbind(successes,failures)~x,family=binomial))
Call:
glm(formula = cbind(successes, failures) ~ x, family = binomial)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.8197  -0.9434   0.0454   0.9358   2.4921  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept) -0.24622    0.11349   -2.17     0.03 *
x            0.00080    0.00195    0.41     0.68  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 134.99  on 99  degrees of freedom
Residual deviance: 134.82  on 98  degrees of freedom
AIC: 422.2

Number of Fisher Scoring iterations: 3

but

props<-successes/n
summary(glm(props~x,family=binomial))

Call:
glm(formula = props ~ x, family = binomial)

Deviance Residuals: 
   Min      1Q  Median      3Q     Max  
-0.852  -0.282  -0.105   0.394   0.760  

Coefficients:
             Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.134339   0.403836   -0.33     0.74
x            0.000281   0.006941    0.04     0.97

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 20.888  on 99  degrees of freedom
Residual deviance: 20.887  on 98  degrees of freedom
AIC: 141.3

Number of Fisher Scoring iterations: 3

Warning message:
In eval(expr, envir, enclos) : non-integer #successes in a binomial glm!
$\endgroup$
  • $\begingroup$ Thanks, @Momo. In my case a binomial doesn't really fit, these "probabilities" are actually somewhat subjective and involve a human judgment in their creation. I'm taking a look at the Beta Regression technique, it's going to take a while for me to figure out how to use it appropriately though. $\endgroup$ – Ken Williams Nov 19 '12 at 16:31
  • $\begingroup$ Glad if it helped. Beta regression probably does it for you then, and its R implementation is very similar to the GLM model family. Good luck. $\endgroup$ – Momo Nov 19 '12 at 18:44
  • $\begingroup$ How does GLM fit in the 2nd case (in which the warning is given)? Where can I find any reference? $\endgroup$ – colinfang Aug 12 '13 at 16:22
  • $\begingroup$ @colinfang I'm afraid you need to look at the source code to check that, as this is probably non-standard (Personally it should throw an error, not a warning). $\endgroup$ – Momo Sep 3 '13 at 20:11
  • $\begingroup$ Your two examples now yield the same coefficient estimates (i.e., both approaches give intercept=-0.134339 and x=0.000281), but estimates' precision varies. $\endgroup$ – jbaums Feb 27 '15 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.