# Prediction of a binary variable

I am establishing a model for prediction of a binary variable (Yes/No) depending on three continuous variables ($A$,$B$,$C$). I applied logistic regression analysis for a learning dataset vith the Tanagra software, and the results were good with high prediction accuracy.

My question is: is it possible to get the probabilities of the prediction via logistic regression? Something like (0.7 Yes)? If not, what test do I have to use to get such a result?

• This is really a software-specific question and belongs on StackOverflow. I'm not familiar with your software, but the answer you need should be available in the documentation. If you're amenable to a solution in a different programming language, you can do this in R with the glm package: ww2.coastal.edu/kingw/statistics/R-tutorials/logistic.html Commented Aug 8, 2013 at 14:47
• @DavidMarx, I mentioned that this is the software package that I am using, but my question is about "the possibility of getting the probability of a prediction via logistic regression". I downloaded Tanagra yesterday, and happy to use any other software that can do this. :) Thanks for the R link Commented Aug 8, 2013 at 14:53

A binary logistic regression is generally used for fitting a model to a binary output, but formally the results of logistic regression are not themselves binary, they are continuous probability values (pushed to zero or 1 by a logit transformaion, but continuous between 0 and 1 nonetheless). It sounds like the software you are using is rounding the output for you, which you don't want. Here's a simple example demonstrating how you could accomplish this in R, since it sounds like you are amenable to trying new software:

# generate sample data
set.seed(123)
x = rnorm(100)
y= as.numeric(x>0)

# let's shuffle a handful so we don't fit a perfect model
ix = sample(1:100, 10)
y[ix]= 1-y[ix]

# Let's take a look at our observations
df = data.frame(x,y)
plot(df)


# Build the model
m = glm(y~x, family=binomial(logit), data=df)

# Look at results
summary(m)

# generate predictions. Here, since I'm not passing in new data
# it will use the training data set to generate predictions
y.pred = predict(m, type="response")
plot(x, y.pred, col=(round(y.pred)+1))


• There are some helpful components to this answer but I guess I wonder why you would simulate your data from a probit regression model, flip 10% of the labels, and then fit a logistic regression model ... It would be just as easy to simulate legitimately from a logistic regression model. E.g. x <- rnorm(100); y <- b0 + b1*x + rlogis(100) > 0 would do the trick. Commented Aug 9, 2013 at 16:34
• @Macro I don't really see what the problem was with my example. I was just throwing some data together quickly as an excuse to build a model. I had a lot of direct control over the appearance of my data the way I did it, as opposed to your method where I'd need to play with the logistic coefficients, which is much less intuitive. Besides, I was just generating data for demo purposes: does it really matter what my method was? The demonstration of the model is what was important, not the generation of my sample data. Commented Aug 9, 2013 at 16:45
• @David, I don't recall downvoting or saying this was a bad answer. I'm only pointing out that it's odd that, by design, the wrong model was fit to the data when it would be just as easy to simulate data from the actual model you're fitting. In response to "does it really matter ...?", I guess I'd just say that the function $$\hat{f}(x) = \hat{P}(Y = 1|X=x)$$ you've plotted is a biased estimate of the true data generating conditional probability, because the wrong model was fit to the data. But, that may be beyond the scope of the question. Cheers. Commented Aug 9, 2013 at 16:55

Yes, you can get the predicted probability that an observation is yes from a logistic regression model. If you have the estimated coefficients from your model fit, you can use those to get the predicted probabilities thusly:
$$\widehat{p(y_i=1)} = \frac{\exp(\hat\beta_0 + \hat\beta_AA_i + \hat\beta_BB_i + \hat\beta_CC_i)}{1+\exp(\hat\beta_0 + \hat\beta_AA_i + \hat\beta_BB_i + \hat\beta_CC_i)}$$