I'm trying to understand logistic regression, and I keep getting hung up on the following point. Let $Y$ be the dependent variable taking its values in ${0,1}$ with a single independent variable $X$ in the model, and let $p(x) = P(Y=1|X=x)$. According to my understanding, what we're really doing is fitting a linear model to the log odds. That is, we seek the maximum likelihood estimation of $\beta_0$ and $\beta_1$ where $$\log(p(x)/(1-p(x))) = \beta_0 + \beta_1 x.$$ I'm confused, however, at how the left hand side is actually computed when this fit is made. In particular, I don't understand how $p(x)$ is computed in the case that there is only a single observation at a particular X value.

By way of example, in the iris data set included in R, suppose I wanted to fit a logistic regression to determine whether the species is setosa (1), or not (0) with the independent variable Sepal.Length. How does one compute $p(4.3)$ for the (singular) observation where Sepal.Length=4.3?

I came up against this when I tried to actually plot the linear function on the right side against the (observed) log odds computed from the data, and realized I had no idea how to compute the left side. Thanks for any help!


1 Answer 1


I think after you get you intercept and $\beta$ from the logistic regression you can directly calculate the probability for Sepal.Lenth=4.3

Here is the R code to do the logistic regression

iris$Species_new<-ifelse(iris$Species=='setosa',1,0) #recode the variable

myLogistic <- glm(Species_new ~ Sepal.Length, data = iris, family = "binomial")


  #Call:  glm(formula = Species_new ~ Sepal.Length, family = "binomial",    data = iris)

  #(Intercept)  Sepal.Length  
  #27.829        -5.176  

 #Degrees of Freedom: 149 Total (i.e. Null);  148 Residual
 #Null Deviance:      191 
 #Residual Deviance: 71.84        AIC: 75.84

Now your logistic regression model is:

$log\frac{p}{1-p}=27.829-5.176x \tag{1}$

$x$ is the Sepal.Length

Now Sepal.Length=$4.3$ plug this value into (1)


It is easy to solve this equation:


Solve this we get $p=0.9962123$

Let us check what the computer get automatically,


You need to check the 14th data point for $p_i$ which is Sepal.Length=4.3

The $p$ is


It is almost the same as calculated by hand, the difference is only caused by different float point I think.

  • $\begingroup$ Thanks for your response. But, isn't the quantity you've found above $\hat{p}$? My understanding may be incorrect here, so let me try to explain another way. In linear regression, the predicted $y$ values are $\hat{y}$ and the observed dependent variable values are $y_i$. Since logistic regression is in essence linear regression where the dependent variable is the log-odds (I know the coefficients are computed differently), how would one find the analog to the $y_i$ values? Or, how would one compute the residuals if we view logistic regression as linear regression with logit response? $\endgroup$
    – Josh
    Oct 3, 2015 at 18:30
  • $\begingroup$ The principal with logistic regression is the same as linear regression. You can check David W. Hosmer's Applied Logistic Regression second Editon p147 goodness of fit. The Hosmer-Lemshow Test part. The observed value is the observed frequency with some characteristics and predicted value are predicted frequency with the same characteristics. $\endgroup$
    – Deep North
    Oct 4, 2015 at 4:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.