0
$\begingroup$

I have a simple linear regression model that is trying to predict the cost of houses. In my data the cost of houses is normally distributed. My model specifically has trouble accurately predicting the price of very expensive houses (but does fine with very cheap houses).

Is there a way to essentially encourage the model to predict higher values? Or is there a different type of algorithm/model that sounds worth trying in this situation?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Are you sure the data looks like it is normally distributed? It could have a heavy right tail and a short left tail. $\endgroup$
    – Michael R. Chernick
    Commented Jul 6, 2018 at 6:13
  • 1
    $\begingroup$ You could try taking logs of prices before modeling. Though I agree with @MichaelChernick that house prices are usually more skewed than normally distributed. $\endgroup$
    – Stephan Kolassa
    Commented Jul 6, 2018 at 6:18
  • 1
    $\begingroup$ I had a similar problem many years ago with the cost of workers compensation insurance claims. There are catastrophic claims that cost millions of dollars while typical claims are below 100 thousand. Sometimes you can find features at the beginning of the claim that will indicate that it will end up very expensive. Otherwise models involving transformations might help. $\endgroup$
    – Michael R. Chernick
    Commented Jul 6, 2018 at 6:20
  • $\begingroup$ @StephanKolassa You are saying exactly what I was about to suggest. $\endgroup$
    – Michael R. Chernick
    Commented Jul 6, 2018 at 6:23
  • $\begingroup$ Transforming the prices sounds good. You're right, on second glance there does appear to be a slight right skew. I'll try log transformation and see if this helps. $\endgroup$
    – David Skarbrevik
    Commented Jul 6, 2018 at 6:36

1 Answer 1

Reset to default
0
$\begingroup$

As Stephan Kolassa and Michael Chernick suggested in the comments of my question. The problem is that the outcome variable is not truly, perfectly normal and this skew can be helped with a log transformation of the variable.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.