1
$\begingroup$

Say I have a problem of predicting the house price.

Originally it is a regression problem. However I consider two classes (expensive houses and not) by set the threshold of 100.000$.

Say I develop a logistic regression (model 1) to predict a house is expensive or not. The model outputs a probability.

I also develop a linear regression (model 2) to predict the price of a house. The model outputs real numbers.

Could I expect the output probability reflects the price of the house as well? If not what should be the relationship between the output of model 1 and model 2?

$\endgroup$

1 Answer 1

1
$\begingroup$

There is a correspondence between regular linear regression and a generalized linear regression model with a probit link function. So this answer doesn't apply to the logistic regression model with a logit link.

Assume $\tilde{Y}$ is your home price, $x$ is your nonrandom predictor, $\beta_0$ and $\beta_1$ are your coefficients, and $\epsilon \sim \text{Normal}(0,\sigma^2)$ is one of your spherical error terms. If $$ \tilde{Y} = \beta_0 + \beta_1 x + \epsilon $$ then $\tilde{Y} \sim \text{Normal}(\beta_0 + \beta_1 x, \sigma^2)$. We have normality, so we can come up with probabilities.

Let $Y \in \{0,1\}$ be your binary predictor that tells you when a home price is greater than \$100,000. Or in other words, $Y=1$ whenever $\tilde{Y} > 100,000$. Then \begin{align*} P(Y=1 |x) &= P(\tilde{Y} > 1e5) \\ &= P( \epsilon > 1e5 - \beta_0 - \beta_1 x ) \\ &= P( \epsilon < -1e5 + \beta_0 + \beta_1 x ) \\ &= \Phi\left(\frac{-1e5 + \beta_0 + \beta_1 x}{\sigma}\right). \end{align*}

Coming from the other end of things, a probit model estimates $$ P(Y=1|x) = \Phi(\alpha_0 + \alpha_1 x). $$ Both models, if used to predict the binary outcome, predict with a function formed by composing the normal cdf with a linear transformation of $x$. The coefficients are not quite the same, but you can see the relationship: $\alpha_0 = (-1e5 + \beta_0)/\sigma$ and $\alpha_1 = \beta_1 / \sigma$. This means the models should give you the same predictions roughly for whether a house is expensive, but not the same parameter estimates.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.