Should I use a multilevel model if I have lots of observations?

Question

I have a dataset with data for 284,000 trips. The trips are grouped into nine cities. The number of trips per city varies between 3,446 and 89,000. I am predicting trip time with seven independent variables. I would like to study how the effect of these IV's varies across cities. I was wondering whether a multilevel model where slopes and intercepts vary by city would be useful, compared to simply fitting a single-level regression for each city.

From what I understand, a multilevel model is useful in accounting for uncertainty in data due to small numbers of observations. In my case, all my groups have 3,500+ observations. Would this make the multilevel model useless?

---

Note: some of my IVs' values vary greatly by city. For instance, I am using distance, whose mean varies between 4 km and 15 km. Would the multilevel model somehow allow me to use the impact of distance on time when distance is large in order to make better predictions for such distances for cities that don't have data for these distances?

Answer 1

Welcome to the site, Ben. You are right that multilevel modeling (MLM) is quite useful when you have limited information about some groups and more information about others. In cases of small groups, it "shrinks" their estimates toward the overall average. This shrinkage happens whenever you allow an effect (intercept or slope) to be random.

Because you only have 9 cities, some would question using MLM. Typically we like to have a large level 2 (city) sample size for these models. That said, you can use MLM and if you do, make sure to use REML estimation (the default in R's lme4), which provides better estimates when the number of clusters is small. REML random effect estimates are more uncertain than full ML estimates. Along similar lines, you should use the Kenward-Roger standard error correction for the fixed effects standard errors to avoid inflated Type-I error rates. A good reference on this is McNeish (2017).

You should also look to see whether your random effect estimates are approximately normally-distributed, which is an assumption of MLM and often hard to meet with a small number of groups.

You could use plain old OLS or GLM to analyze your data, treating the cities as dummy variables. Then to investigate whether the associations between predictors and your outcome vary across cities, you would include interaction terms between predictors and the 8 city dummy variables. This will give you a lot of coefficients. As you mention, you could run the model separately by city. I think I would probably try out the MLM and check on the normality of your random slopes and intercepts. MLM will allow you to be much more parsimonious in your modeling.

With regards to your IVs varying greatly by city, I would suggest that you center within cluster (CWC) all your predictors (for each predictor, calculate the mean by city and then subtract that mean value from the raw value). For example in R's dplyr:

df <- df %>% group_by(city) %>% mutate(mn_x = mean(x)) %>% ungroup() %>% mutate(cwc_x= x - mn_x)

Then your predictors are interpreted relative to the city's average. This is also helpful in estimating random slopes and then looking at interactions between trip and city variables (sometimes called cross-level interactions).