I am having trouble training a model for nested data about house prices. Lets say my data looks like following:

  logPrice bedCount bathCount                city
 0.6517920        4       2-3        Redwood City
 0.4402192        1       1-2 South San Francisco
 0.5922396        2       1-2           San Mateo
 0.4606918        3       1-2 South San Francisco
 0.7592523    5plus       3-4           San Mateo
 0.4710397        1       1-2        Redwood City

bedCount, bathCount and city are factors.

As a baseline, I trained a simple linear model ignoreing nested structure of the data (houses are nested within cities).

lm.model = lm(formula = logPrice ~ 1 + bedCount + bathCount + city)

which corresponds to following assumption:

logPrice$_i = \beta_0 + \beta_1\cdot$ bedCount$_i + \beta_2\cdot$ bathCount$_i + \beta_{3,j[i]}\cdot I$(city$_{j[i]}) + \epsilon_i$


$\epsilon_i \sim N(0, \sigma^2_{logPrice})$ and $I$(city$_{j[i]}$) is the indicator function for city of the $i^{th}$ house (which is 1).

Now, I trained a 2-level hierarchical model:

lmer.model = lmer(formula = logPrice ~ 1 + bedCount + bathCount + (1 | city))

which corresponds to the following assumption:

logPrice$_i = \beta_0 + \beta_1\cdot$ bedCount$_i + \beta_2\cdot$ bathCount$_i + \beta_{3,j[i]}\cdot I$(city$_{j[i]}) + \epsilon_i$

where $\epsilon_i \sim N(0, \sigma^2_{logPrice})$ and $\beta_{3,j} \sim N(0, \sigma^2_{\beta_3})$

Now, on the training data, lm.model gives me lesser average RMSE than lmer.model which shouldn't happen because linear regression is a special case of multilevel linear regression (I didn't care to check average RMSE on test data because that on training data itself should be lower for 2nd model than that for 1st model). In fact, my data has multiple levels (houses nested within subdivisions, which are nested within zipcodes, which in turn are nested within cities) and the performance gets worse as I add more and more levels to the model (i.e. model with random effect (1 | subdivision) does worse than that with random effect (1 | zipcode) + (1 | zipcode:subdivision), which in turn does worse than a model with random effect (1 | city) + (1 | city:zipcode) + (1 | city:zipcode:subdivision)).

What am I missing?

  • $\begingroup$ How do you calculate RMSE? $\endgroup$ – Roland Mar 21 '14 at 12:17
  • $\begingroup$ Average RMSE is mean((fitted(model) - data$logPrice)^2). Well, I realize it's average MSE rather than RMSE. $\endgroup$ – abhinavkulkarni Mar 21 '14 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.