# The relationship between probability in classification and real values in regression problems?

Say I have a problem of predicting the house price.

Originally it is a regression problem. However I consider two classes (expensive houses and not) by set the threshold of 100.000$. Say I develop a logistic regression (model 1) to predict a house is expensive or not. The model outputs a probability. I also develop a linear regression (model 2) to predict the price of a house. The model outputs real numbers. Could I expect the output probability reflects the price of the house as well? If not what should be the relationship between the output of model 1 and model 2? ## 1 Answer There is a correspondence between regular linear regression and a generalized linear regression model with a probit link function. So this answer doesn't apply to the logistic regression model with a logit link. Assume $$\tilde{Y}$$ is your home price, $$x$$ is your nonrandom predictor, $$\beta_0$$ and $$\beta_1$$ are your coefficients, and $$\epsilon \sim \text{Normal}(0,\sigma^2)$$ is one of your spherical error terms. If $$\tilde{Y} = \beta_0 + \beta_1 x + \epsilon$$ then $$\tilde{Y} \sim \text{Normal}(\beta_0 + \beta_1 x, \sigma^2)$$. We have normality, so we can come up with probabilities. Let $$Y \in \{0,1\}$$ be your binary predictor that tells you when a home price is greater than \$100,000. Or in other words, $$Y=1$$ whenever $$\tilde{Y} > 100,000$$. Then \begin{align*} P(Y=1 |x) &= P(\tilde{Y} > 1e5) \\ &= P( \epsilon > 1e5 - \beta_0 - \beta_1 x ) \\ &= P( \epsilon < -1e5 + \beta_0 + \beta_1 x ) \\ &= \Phi\left(\frac{-1e5 + \beta_0 + \beta_1 x}{\sigma}\right). \end{align*}

Coming from the other end of things, a probit model estimates $$P(Y=1|x) = \Phi(\alpha_0 + \alpha_1 x).$$ Both models, if used to predict the binary outcome, predict with a function formed by composing the normal cdf with a linear transformation of $$x$$. The coefficients are not quite the same, but you can see the relationship: $$\alpha_0 = (-1e5 + \beta_0)/\sigma$$ and $$\alpha_1 = \beta_1 / \sigma$$. This means the models should give you the same predictions roughly for whether a house is expensive, but not the same parameter estimates.