4
$\begingroup$

I'm building my first linear regression model with multiple features (predicting house prices in a specific city). After reading up on ways to improve my model, I see people talking about plotting the relationship between the target variable and the features. I then realized that one of my features, the construction year of the house, is kind of "jumpy" which probably messes up the coefficient.

My question: How does one handle features as this one? Drop them? Transform them somehow? Turn them into categorial variables?

Chart below. Y axis is mean house price (in Swedish kronor) per year.

Price after construction year

Edit: Added plot of residuals below. Residuals plot

Edi2: Added residual histogram below.Histogram of residuals

$\endgroup$
7
  • $\begingroup$ What is depicted on the Y axis here? $\endgroup$ Jun 8, 2017 at 19:11
  • $\begingroup$ Oh, sorry. That is the price the house was sold for. $\endgroup$
    – user153009
    Jun 8, 2017 at 19:11
  • 1
    $\begingroup$ 0.1e+7-1.0e+7. Mean prices per year. I've updated the question. $\endgroup$
    – user153009
    Jun 8, 2017 at 19:39
  • 1
    $\begingroup$ There is nothing "non-linear" with that feature, and there is no indication of doing anything in particular. But you should ask yourself why the mean price is highest arouind construction 1870-1880, maybe because construction then was in a now very centric and popualr zone? I guess there is a string interaction construction year with zoning, you should look at that sort of things. $\endgroup$ Jun 8, 2017 at 20:03
  • 2
    $\begingroup$ It is almost never reasonable to convert a numeric into a factor, so no. What you see in a plot like this is a marginal relationship, but what you are modelling is a conditional relationship taking into account all other variables. You could think about using splines, maybe, but first, and more important, understand your variables and the relationships between them! nand think about intyeractions ... $\endgroup$ Jun 8, 2017 at 20:18

2 Answers 2

1
$\begingroup$

Your residual plot appears normal enough to use linear regression with Ordinary Least Squares (OLS) loss.

The linear in linear regression refers to the OLS loss function:

$$ \hat{Y_i} = \beta_{0} + \beta_{1} X_{i} + \epsilon_i $$

Which is linear in each term. It does not refer to the linearity of the independent variables which are being regressed against the dependent output.

If you are looking for a linear regression-like model that fits a non-linear equation, check out Support Vector Machines (SVM) with a polynomial kernel.

$\endgroup$
7
  • $\begingroup$ Yeah, I realized I phrased that poorly. However, my question remains. How does one deal with features that don't seem to follow any discernible pattern across all values. How would you handle this particular feature? $\endgroup$
    – user153009
    Jun 8, 2017 at 20:43
  • 1
    $\begingroup$ there's nothing special to do. what is the distribution of your residuals? $\endgroup$
    – redress
    Jun 8, 2017 at 21:47
  • $\begingroup$ Oh ok. I added a plot of the residuals in my post. Does it look problematic? $\endgroup$
    – user153009
    Jun 8, 2017 at 22:04
  • $\begingroup$ please share a histogram $\endgroup$
    – redress
    Jun 8, 2017 at 22:24
  • 3
    $\begingroup$ Given the lack of fit in residuals vs fitted, I'm not sure there's much value in the histogram of residuals. I'd also worry about the dependence of errors and heteroskedasticy $\endgroup$
    – Glen_b
    Jun 9, 2017 at 4:19
1
$\begingroup$

You can relax linearity assumptions by adding nonlinear terms into your model. Usually polynomials or splines.

Transformations of such variables is also possible. For example things in nature are often related on a log scale, so a log transform would be appropriate. However such transformations doesn't help to any nonlinearity, only the ones following the speciffic distribution.

Discretization of continuous variables is not recommended due to many problems it creates.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.