Clustered standard errors - Why are SE smaller or bigger than OLS depending on cluster level?

Question

I am analyzing some data using an OLS model. Data represent managers working for US cities. Within each city, we surveyed more than one manager (max 5). Multiple cities per state were surveyed.

I'd like to use clustered standard errors to account for possible clusters at the city or at the state level (state policies might be relevant in our study). Some researchers (e.g., Cameron & Miller, 2015) suggest clustering at the highest level (the state level, in my case). As a check, I tried to estimate clustered SE at both levels and:

1. When I use clustered SE at the city level, standard errors become slightly larger, but overall they are very similar to OLS results.
2. When I use clustered SE at the state level, standard errors become much smaller, leading to quite different results than the OLS model.

What are the possible reasons for such differences? What estimates should I consider? The fact that the standard errors change more when I look at the state-level, would this be a suggestion that state-level clusterization is more important?

My sample contains 2250 observations, grouped in 487 cities (average cluster size = 4, but some cluster size = 1) and 49 states (average cluster size = 30).

Answer 1

What you observe can be explained by the correlations in the measurements within the clusters. Namely, when you select an analysis, such as OLS that does not account for these correlations, you expect that standard errors of within clusters effects to be overestimated, and standard errors of between clusters effects to be underestimated.

Answer 2

The variance inflation equation (6) on page six (adjusted for unequal cluster size below) in the Cameron and Miller paper you linked contains the intuition. If you have positive correlation in either the regressor of interest or the errors within cities (the two $\rho$s), but a negative correlation within states, that could explain the pattern of what you are seeing. This could be amplified by the unequal cluster size multiplying the $\rho$s at the two levels of clustering. You can estimate these to confirm this.

You don't provide any details of your setting, so it is hard to give an example of how this could happen in your case. One example is if you have a pattern of migration from rural to urban areas in your data driven by local booms. Then all the observations from cities could have positively correlated positive residuals capturing the booms there and the rural areas will have positively correlated negative residuals because of the busts, but within the states, the rural observations' residuals would be negatively correlated with the urban ones if the migrants move in-state. There is another example here with more explanation.

Also, you should use bigger and more aggregate clusters when possible, up to and including the point at which there is concern about having too few clusters. In other words, you definitely don't want to always cluster at the highest level (say the four census regions in the US). Unfortunately, there's no clear definition of "too few", but fewer than 50 is when people start getting worried.