I'm trying to build a model based on some training set.
The training set contains 1460 observations, with 79 variables each (features).
I'm using linear regression to build a model and after that building a step-regression model (forward selection algorithm), and still getting large value for RMSE.
I'm very new to data analysis, so If someone can point me to what it is I'm doing wrong that would be great.
79 Features of houses in some city (Lot Frontage, Pool area, Total area, etc.), And SalePrice label of that house.
43 Of them are categorical variables identifying various types of dwellings, garages, etc.
In order to clean my data - I'm performing the following steps:
Lots of the factor variables have logic order to their levels, e.g.:
BsmtQual: Evaluates the height of the basement Ex Excellent (100+ inches) Gd Good (90-99 inches) TA Typical (80-89 inches) Fa Fair (70-79 inches) Po Poor (<70 inches NA No Basement
So 1st, I'm converting them to numerical values, e.g.:
BsmtQual: Evaluates the height of the basement Ex records will be replaced with: 5 Gd records will be replaced with: 4 TA records will be replaced with: 3 Fa records will be replaced with: 2 Po records will be replaced with: 1 NA records will be replaced with: 0
This step is in order to reduce amount of factor variables
Afterwards, I'm using
mice() library to impute missing values that are left in the data.
- Running PCA on the training data
- Selecting K PCs based on cumulative scree plot
- Transforming test data into PCA
Building linear regression model:
lm.model <- lm(SalePrice ~ ., data = train.data)
lm.modelon my test set.
I'm getting very high RMSE for
lm.model - 2.256228e+04, Although Rsquared param seems high - 0.91928
I also tried running step regression based on the
lm.model, but still I get high RMSE.
Additionally, PCA doesn't seem to make a big difference also.
How can I improve? What am I doing wrong?