# How to manually calculate the intercept and coefficient in logistic regression

I'm currently studying about Logistic Regression. But I'm getting stuck at calculating intercept ($\beta_0$) and coefficient ($\beta_1$). I've been looking for it through the internet, but only get tutorials using Microsoft Excel or built-in functions in R. I heard it can be solved by Maximum Likelihood, but I don't understand how to use it, because I don't have a statistical background. Can anyone can give me a brief explanation and simulation for calculating the coefficients manually?

• Do you understand optimisation in a general sense? Such as finding the minimum or maximum of a function? Commented Nov 27, 2016 at 12:18
• I really wish more people asked questions like this. Commented Feb 2, 2017 at 1:56

## 2 Answers

Unfortunately, unlike linear regression, there's no simple formula for the maximum likelihood estimate of logistic regression. You'll have to perform some kind of optimization algorithm, like gradient descent or iteratively reweighted least squares.

• That is generally true. An exception exists when the only explanatory variable is categorical, or when all explanatory variables are categorical and all (higher order) interactions are included in the model. In that case the coefficients are transformations of the means. Commented Oct 29, 2016 at 16:58
• Do you have a source for this? Commented Nov 9, 2018 at 22:04

I'd like to propose my method and hope it helps.

To calculate the coefficients manually you must have some data, or say constraints. In logistic regression, actually it is how logistic function is defined via the maximum entropy and lagrange multipliers, this constraint must be met with other two: $$E_p f_j = E_{\hat p}f_j$$. That is, the model's expectation should match the observed expectation, which has been illustrated in this paper. That's why logit-function as a link function in logistic regression is also termed mean function.

Take for example, the crosstab bellow shows how many males/females are in the honor class.

           |         female
hon |      male     female |     Total
-----------+----------------------+----------
0 |        74         77 |       151
1 |        17         32 |        49
-----------+----------------------+----------
Total |        91        109 |       200


As mentioned above $$\sum_i y_i x_{ij} = \sum_i p_i x_{ij}$$ holds. The left hand side(LHS) is the expectation of the observations(y's in the sample) and the right hand side(RHS) is the model's expectation.

Assuming the function is $$log(\frac{p}{1-p})=\beta_0 + \beta_1x_i$$ or equivalently $$p=\frac{1}{1+e^{-(\beta_0+\beta_1 * x_i)}}$$($$x_i$$ represents the feature of the observation being a female, it is 1 if the observation is a female and 0 otherwise), obviously we know that the following two equations hold respectively when $$X=1$$ and when $$X=0$$ with the data shown above:

$$\frac{32}{109} = \frac{1}{1+e^{-(\beta_0+\beta_1 * 1)}}$$

$$\frac{17}{91} = \frac{1}{1+e^{-(\beta_0 + \beta_1 * 0)}}$$

So the intercept($$\beta_0$$) is -1.47 and the coefficient($$\beta_1$$) is 0.593. You can manually get it.

Along the same lines, you can manually calculate coefficients of other logistic regression models(it applies also to softmax regression but it is out the scope of this question) if enough data are given.

I hope I am right, if not please let me know. Thanks.

• I'm not the downvote, so I can't say for sure. But I think you could improve your answer by 1) relating your calculations to the maximum likelihood problem that logistic regression solves, 2) Explaining why exactly this example can be worked by hand but others cannot, 3) fitting the regression using an iterative algorithm and showing that the answer is the same. Commented Nov 22, 2016 at 22:41