# What would be the exact function for a ridge logistic regression with multiple variables?

I am looking for the correct equation for a ridge logistic regression for multiple variables. I thought it simply was:

$$y=\frac1{1+e^{-(\beta_0+\beta_1X_1+\beta_2X_2+\cdots+\beta_nX_n)}}$$

$$\sum_i^n (y_i-\widehat{y_i})^2 + \lambda \sum_j^p \beta_j^2$$

Eventually

$$y=\frac1{1+e^{-(\beta_0+\beta_1X_1+\beta_2X_2+\cdots+\beta_nX_n)}}+\sum_i^n (y_i-\widehat{y_i})^2 + \lambda \sum_j^p \beta_j^2$$

Is this true for a ridge regression?

• Where does the sum of $(y_i-\hat y_i)^2$ come from? That has nothing to do with logistic regression or any regularization of it. Your initial formulation does not describe logistic regression, either: for it to be correct, you must equate the expectation of $y$ with the right hand side. Your final formulation makes no sense at all, because it completely changes the model. Please see stats.stackexchange.com/questions/29325. Perhaps stats.stackexchange.com/questions/228763 answers your question?
– whuber
Commented Jan 31, 2022 at 18:53
• Hi @whuber. Would this be the correct equation for a multiple logistic regression then: ln[Y/(1−Y)]=a+b1X1+b2X2+b3X3...? Commented Feb 1, 2022 at 8:07
• That's stated explicitly in an answer in the first link I provided.
– whuber
Commented Feb 1, 2022 at 14:23
• I am sorry, I only looked at the second link for my question. I thank you a lot! Commented Feb 1, 2022 at 15:22

The method of estimation does not affect the regression equation. Your logistic regression model is always the equation you posted in the first equation (save some technicalities).

If you estimate the parameters using the usual maximum likelihood estimation, you will get different estimated values of the parameters than if you use regularization, but those estimates are just guesses at the true values of the $$\beta_i$$.

As a heads up, the traditional loss function for logistic regression is log loss, not square loss.

$$L(y,\hat y)=-\sum\bigg[ y_i\log(\hat y_i)+ (1-y_i)\log(1-\hat y_i)\bigg]$$

In the equation, $$y=(y_1,\cdots,y_n)$$ is the vector of observed classes (so all $$0$$s and $$1$$s), and $$\hat y =(\hat y_1,\cdots,\hat y_n)$$ is the vector of predicted probability values.

• Thanks @Dave. Nice and clear. I used the ridge regression with an optimization of the maximum likelihood estimation. Does that influence the second equation (penalty parameter)? If so, what would the equation for maximum likelihood be? Commented Jan 31, 2022 at 12:33
• @Thomas The $L(y,\hat y)$ equation in my answer is the loss function corresponding to maximum likelihood estimation. I do not follow what you mean when you ask about the "second equation:. Could you please type out exactly what you think the correct regression equation is?
– Dave
Commented Jan 31, 2022 at 18:49
• NB: The first equation is not a correct representation of logistic regression, and therein may lie the source of the question.
– whuber
Commented Jan 31, 2022 at 19:00
• @Dave, sorry. I meant the loss function I added in my question, but you already answered my question though. Thanks. Commented Feb 1, 2022 at 8:09
• You're the best, thanks! Commented Mar 8, 2022 at 12:21