1
$\begingroup$

I am new to machine learning and I am simultaneously studying linear and logistic regression. Logistic regression is when there is one dependent variable and there may be more than one independent variable

Multivariable linear regression attempts to find a linear relationship between multiple variables where every independent variable x must have a dependent variable y.

Can anyone tell me the difference between the two?

$\endgroup$
1
  • $\begingroup$ Logistic regression involves a binary response. Linear regression does not. $\endgroup$ Commented Oct 19, 2019 at 0:49

2 Answers 2

3
$\begingroup$

In linear regression, you predict the target variable $y$ using a model that is based on a linear function of the predictors $X_1,X_2,\dots,X_k$

$$ y = \beta_0 + \beta_1 X_2 + \beta_2 X_2 + \dots + \beta_k X_k + \varepsilon $$

In logistic regression, the target variable is binary, and you are trying to predict probability of "success" $p(y=1)$ using a linear predictor $\eta$ passed through the link function $\sigma$ (logistic function is the most common choice), i.e.

$$ \eta = \beta_0 + \beta_1 X_2 + \beta_2 X_2 + \dots + \beta_k X_k \\ p(y=1) = \sigma(\eta) $$

So both can be functions of a single, or multiple variables. The differences are that the target variable differs (values on real line vs binary), and logistic regression, as other generalized linear models, uses a link function.

$\endgroup$
2
$\begingroup$

In multiple linear regression (MLR), you still have one dependent variable, and the relation is as follows: $$y_i=\beta_0+\sum_{i=1}^n\beta_ix_i+\epsilon$$ In logistic regression (LR), what you predict is a probability, namely the class posterior, and is bounded unlike MLR output. They're similar in the sense that they both use linear combination of features. The probability of class 1 is estimated as follows in LR: $$p_i=\frac{1}{1+e^{-(\beta_0+\sum\beta_i x_i)}}$$

So, LR is actually taking $\text{logit}$ of MLR output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.