13
$\begingroup$

I'm using linear regression to predict a price which is obviously positive. I have only one feature which is gross_area. I standardized it (z-score) I got this kind of value:

array([[ 1.        , -0.48311432],
       [ 1.        ,  0.68052306],
       [ 1.        ,  2.1426852 ],
       [ 1.        , -1.17398593],
       [ 1.        , -0.16265712]])

Where the 1 is the constant for the intercept term. I predict the parameters(predictors) and I got this:

array([[ 31780004.85045217],
       [ 27347542.4693376 ]])

Where the first cell is the intercept term and the second cell correspond to the parameter found for my feature gross_area.

My problem is the following, when I take for example the fourth line and I compute the matrix multiplication XB to get my prediction, I got this:

In [797]: np.dot(training[4], theta)
Out[797]: array([-325625.35640697])

Which is totally wrong since I cannot have negative value for my dependent variable. It seems like because of my normalization where I got negative value for my feature, I ended up with a negative predicted value for some tuple. How can is it possible and how can I fix this ? Thank you.

This is what I have predict graphically: enter image description here

with y=price , x =gross area

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Can you elaborate on how price and area are related? The solution is either to use log price (though prediction gets tricky) or you need to use some sort of GLM, but the details will depend on what you're trying to model. $\endgroup$
    – dimitriy
    Apr 8, 2015 at 18:27
  • $\begingroup$ The Pearson's correlation coefficient is about 0.84392 between gross_area and price $\endgroup$
    – Marc Lamberti
    Apr 8, 2015 at 18:37
  • $\begingroup$ I don't mean statistically. Why would price change with area? What is the mechanism or model? $\endgroup$
    – dimitriy
    Apr 8, 2015 at 18:48
  • $\begingroup$ Because as the gross area increases, the prices increases as well.. But I don't think that I understand what do you mean exactly :( $\endgroup$
    – Marc Lamberti
    Apr 8, 2015 at 18:53
  • 1
    $\begingroup$ See the discussion in this answer, this one and this one, and perhaps also this. $\endgroup$
    – Glen_b
    Apr 10, 2015 at 9:42

1 Answer 1

Reset to default
14
$\begingroup$

Linear regression does not respect the bounds of 0. It's linear, always and everywhere. It may not be appropriate for values that need to be close to 0 but are strictly positive.

One way to manage this, particularly in the case of price, is to use the natural log of price.

Improve this answer
$\endgroup$
9
  • $\begingroup$ You mean applying log(price) before computing the prediction ? $\endgroup$
    – Marc Lamberti
    Apr 8, 2015 at 18:34
  • 8
    $\begingroup$ There's no mystery here. A straight line $y = a + bx$ will predict negative $y$ for some $x$ unless $a > 0$ and $b = 0$. That can bite within the range of your data, and is biting you. Using logarithmic scale seems natural not just because you reasonably prefer positive predictions, but also because curvature is evident too from your graph, and indeed uneven scatter. Whether you are better off with log transformation or a GLM with logarithmic link is a more subtle question. $\endgroup$
    – Nick Cox
    Apr 8, 2015 at 19:00
  • 2
    $\begingroup$ There are also many posts explaining why that statement (precisely that within "") is wrong. At most, there are some nice properties if the error distribution is normal. Suppose $y = a + bx + \epsilon$ and the error $\epsilon$ is uniformly distributed on $[-c, c]$. Is regression then invalid? Not so. Functional form is primary; marginal or conditional distribution is secondary. $\endgroup$
    – Nick Cox
    Apr 8, 2015 at 19:37
  • 4
    $\begingroup$ marcL -- There are three main problems with the model you fitted: (1) the relationship isn't linear; (2) the model you chose doesn't respect a known bound; (3) the spread isn't constant. The fact that the transformation would also make the conditional distribution less skew would be a bonus, rather than a requirement. (A normal assumption in regression comes in when you're trying to do inference - testing, confidence intervals - but other things can be done if you don't expect to have normality.) ... (ctd) $\endgroup$
    – Glen_b
    Apr 8, 2015 at 23:08
  • 4
    $\begingroup$ (ctd) ... However, if you take logs and want a mean prediction back on the original scale, you can't just exponentiate the mean on the log scale. There are several options there (though if I primarily wanted mean predictions I'd be inclined to fit a gamma GLM with log link; a very similar model but the calculation of original scale means is more direct). If I primarily wanted a prediction interval, the linear model in the logs is easier. $\endgroup$
    – Glen_b
    Apr 8, 2015 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.