0
$\begingroup$

I'm trying to build a model based on some training set.

The training set contains 1460 observations, with 79 variables each (features).

I'm using linear regression to build a model and after that building a step-regression model (forward selection algorithm), and still getting large value for RMSE.

I'm very new to data analysis, so If someone can point me to what it is I'm doing wrong that would be great.

Data:

79 Features of houses in some city (Lot Frontage, Pool area, Total area, etc.), And SalePrice label of that house.

43 Of them are categorical variables identifying various types of dwellings, garages, etc.

Cleanup:

In order to clean my data - I'm performing the following steps:

Lots of the factor variables have logic order to their levels, e.g.:

BsmtQual: Evaluates the height of the basement

   Ex   Excellent (100+ inches) 
   Gd   Good (90-99 inches)
   TA   Typical (80-89 inches)
   Fa   Fair (70-79 inches)
   Po   Poor (<70 inches
   NA   No Basement

So 1st, I'm converting them to numerical values, e.g.:

BsmtQual: Evaluates the height of the basement

   Ex records will be replaced with: 5  
   Gd records will be replaced with: 4
   TA records will be replaced with: 3
   Fa records will be replaced with: 2
   Po records will be replaced with: 1
   NA records will be replaced with: 0

This step is in order to reduce amount of factor variables Afterwards, I'm using mice() library to impute missing values that are left in the data.

Model building

  1. Running PCA on the training data
  2. Selecting K PCs based on cumulative scree plot
  3. Transforming test data into PCA
  4. Building linear regression model:

    lm.model <- lm(SalePrice ~ ., data = train.data)
    
  5. Using predict() with the lm.model on my test set.

I'm getting very high RMSE for lm.model - 2.256228e+04, Although Rsquared param seems high - 0.91928

I also tried running step regression based on the lm.model, but still I get high RMSE.

Additionally, PCA doesn't seem to make a big difference also.

How can I improve? What am I doing wrong?

Thanks

$\endgroup$
  • $\begingroup$ Is that a "very high" RMSE though? That's about a $20k error in sale price, which doesn't seem that bad compared to the variation in sale prices. What kind of RMSE were you expecting to get? $\endgroup$ – The Laconic Feb 25 '17 at 19:00
  • $\begingroup$ This is part of a Kaggle project (with no prizes) we are required to submit in academic course. For example, my model gets to the 3800 position out of 4600 submissions, which seems very poor. $\endgroup$ – Adiel Feb 25 '17 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.