I will try diagnosing your problem without seeing your data. Please edit the question and show us a subset of your data frame to receive the best possible response in the future.
It appears you acquired a panel of individuals observed over several years. Individuals were likely sampled across many regions in the United States, and so your data has a nested structure.
Whenever I control for states, the model drops this variable for collinearity reasons.
According to your R code, you indexed individuals $i$ and years $t$ (see, e.g., index = c("id", "year")
. Individuals $i$ appear to be nested within state $s$. I am assuming id
denotes individuals in your sample. It shouldn't index states as you attempted to adjust for state membership, denoted by the inclusion of factor(states)
in the model formula. Before addressing the collinearity, I advise you to familiarize yourself with the plm documentation before building your model. Inside of the plm()
function you specified model = "within"
. The model is using the "within transformation" (fixed effect transformation) to estimate your model. As a consequence of setting id
(i.e., individuals) as your index, your "within transformation" is applied to your individuals. Thus, you're estimating individual fixed effects.
Without the variable states, the model works with no problem. Could someone explain to me why this could be happening?
The model is working fine, you just have linearly dependent columns. The individual fixed effects absorb your state fixed effects. To be clear, incorporating factor(states)
as a covariate is equivalent to estimating state fixed effects. Including dummies for all $i$ individuals or $s$ states in your sample is equivalent to estimation in deviations from means.
Your code translates to a one-way fixed effects model. However, redundancies exist. Here is what you're trying to estimate:
$$
y_{ist} = \alpha_i + \gamma_s + X_{ist}'\beta + \epsilon_{ist}
$$
where you observe individual $i$ in state $s$ over years $t$. $a_i$ and $\gamma_s$ represent individual and state fixed effects, respectively. $X_{ist}$ denotes a vector of presumably time-varying covariates at the individual and/or state level. The problem with this model is the $i$-level effects will absorb the $s$-level effects, hence why the state effects don't appear in your output. But why is this? Here, $\alpha_i$ captures effects that are specific to individuals but constant over time. Is a state to which individual $i$ belongs changing, or is their membership constant? Suppose I sample one individual $i$ from your panel, discarding all other observations. I begin to note individual $i$'s characteristics. I observe individual $i$ resides in the State of New York. Individual $i$ always bears this characteristic. Put differently, individual $i$ retains their "New Yorkness" in all time periods. Thus, state membership is a time constant attribute, which is already captured by the individual fixed effects (i.e., $\alpha_i$). Including state fixed effects is redundant. As a consequence, $\gamma_s$ is dropped.
If you're not using dummy variables to estimate the fixed effects, it is a little harder to see why the higher level fixed effects are being dropped. I will demonstrate it using the Produc
dataset. I don't have your particular data to play with so this should work for demonstration purposes. For simplicity, I will investigate the number of highways as a function of labor input (i.e., hwy ~ emp
). Here, states are embedded within geographic regions, so we have a similar structure. Pretend the states represent your individuals and the regions represents your states. The nested structure of the data generalizes to a setting involving individuals nested within states. I assume individuals retain their state membership throughout the panel. In other words, individuals do not jump between states over time. The example that follows is much more explicit, as a state should retain its geographic location within the United States over time. For example, Maine is a fixed jurisdiction in the Northeast United States. Thus, I should know the regional territory as a consequence of knowing the state.
I will estimate state and region fixed effects using the plm()
and lm()
functions. In lm()
I will estimate all fixed effects using dummy variables. As indicated earlier, if I know the state (individual), then I should know the region (state). A state's geographic location is fixed; it is a time invariant attribute of the state. The state fixed effects already adjust for this. Thus, the region fixed effects are collinear with the state fixed effects.
In the example below, plm()
silently drops the region fixed effects, while lm()
does so explicitly. If you don't believe me, try running summary(mod_lm)
in your console and you should see NA
values in your output for all regions. Again, the region fixed effects (similar to your state fixed effects) are redundant. Attempting estimation does not even affect your estimate of emp
.
Data
library(plm)
data("Produc")
df <- Produc %>%
select(region, state, year, everything()) %>% # states nested within regions
arrange(region) # arrange by region
plm()
# linear panel model estimated via the within transformation
# the region effects are silently excluded
mod_plm <- plm(hwy ~ emp + as.factor(region), index = c("state", "year"), model = "within", data = df)
summary(mod_plm)[["coefficients"]]
Estimate Std. Error t-value Pr(>|t|)
emp 1.119484 0.07663479 14.60804 8.040036e-43
lm()
# linear model with a full set of state and region dummies
# state effects absorb the region effects
mod_lm <- lm(hwy ~ emp + as.factor(state) + as.factor(region), data = df)
summary(mod_lm)[["coefficients"]]
Estimate Std. Error t value Pr(>|t|)
(Intercept) 6899.016720 191.50194668 36.025831 4.152722e-167
emp 1.119484 0.07663479 14.608036 8.040036e-43
as.factor(state)ARIZONA -2417.051442 235.64248909 -10.257282 3.200004e-23
as.factor(state)ARKANSAS -3384.433525 238.08979432 -14.214946 7.086798e-41
as.factor(state)CALIFORNIA 28553.785112 635.46382733 44.933769 4.957521e-217
as.factor(state)COLORADO -2711.162396 234.30726002 -11.570971 1.183251e-28
as.factor(state)CONNECTICUT -1164.600522 234.20039961 -4.972667 8.152971e-07
as.factor(state)DELAWARE -5082.891321 246.43435499 -20.625742 1.550344e-75
as.factor(state)FLORIDA 5150.426272 280.89163197 18.335990 1.499589e-62
as.factor(state)GEORGIA 631.348246 241.70122834 2.612102 9.174782e-03
as.factor(state)IDAHO -4622.038557 245.43058670 -18.832366 2.545365e-65
as.factor(state)ILLINOIS 14436.899662 347.06063365 41.597630 8.387607e-199
as.factor(state)INDIANA 953.329367 242.03304769 3.938840 8.933793e-05
as.factor(state)IOWA 1767.748379 234.73435102 7.530847 1.418867e-13
as.factor(state)KANSAS -1116.659783 236.03909138 -4.730826 2.661861e-06
as.factor(state)KENTUCKY 2740.126855 234.29929900 11.694985 3.440603e-29
as.factor(state)LOUISIANA 3196.562563 234.30833390 13.642547 4.244857e-38
as.factor(state)MAINE -4863.757198 243.21033319 -19.998152 6.291509e-72
as.factor(state)MARYLAND 800.383760 235.67611424 3.396117 7.186799e-04
as.factor(state)MASSACHUSETTS -696.608731 253.50849167 -2.747871 6.139215e-03
as.factor(state)MICHIGAN 8729.752681 282.91047371 30.856944 1.456615e-136
as.factor(state)MINNESOTA 3045.978538 235.70787879 12.922684 1.052963e-34
as.factor(state)MISSISSIPPI -1746.396813 237.23704057 -7.361400 4.697256e-13
as.factor(state)MISSOURI 1850.755976 238.95647681 7.745159 3.022418e-14
as.factor(state)MONTANA -3302.813778 246.34110351 -13.407481 5.606840e-37
as.factor(state)NEBRASKA -2718.026910 239.65595032 -11.341370 1.137109e-27
as.factor(state)NEVADA -5023.866050 244.53306859 -20.544731 4.550161e-75
as.factor(state)NEW_HAMPSHIRE -5083.590882 244.04403362 -20.830630 1.012769e-76
as.factor(state)NEW_JERSEY 2938.891317 268.10915546 10.961548 4.478248e-26
as.factor(state)NEW_MEXICO -3788.571543 242.69123535 -15.610665 6.517626e-48
as.factor(state)NEW_YORK 21104.078129 512.13433531 41.208091 1.230557e-196
as.factor(state)NORTH_CAROLINA 158.974226 245.74852320 0.646898 5.178915e-01
as.factor(state)NORTH_DAKOTA -4189.658798 247.18457990 -16.949515 5.684312e-55
as.factor(state)OHIO 12821.045562 324.35291103 39.528073 3.325567e-187
as.factor(state)OKLAHOMA -2014.416928 234.83528304 -8.577999 5.310668e-17
as.factor(state)OREGON -1631.933372 235.48902407 -6.929976 8.927248e-12
as.factor(state)PENNSYLVANIA 12712.678848 345.92861804 36.749428 2.635576e-171
as.factor(state)RHODE_ISLAND -5289.929593 243.43124060 -21.730693 5.810162e-82
as.factor(state)SOUTH_CAROLINA -3618.604138 234.40165161 -15.437622 5.075261e-47
as.factor(state)SOUTH_DAKOTA -3944.328466 247.15751750 -15.958764 1.013413e-49
as.factor(state)TENNESSE 1438.081915 235.99257374 6.093759 1.745019e-09
as.factor(state)TEXAS 17202.744343 383.78891908 44.823452 1.943332e-216
as.factor(state)UTAH -3781.314167 241.13047926 -15.681610 2.800101e-48
as.factor(state)VERMONT -5183.378799 248.05579092 -20.896020 4.233164e-77
as.factor(state)VIRGINIA 4271.244391 240.94274963 17.727217 3.420277e-59
as.factor(state)WASHINGTON 1894.400135 234.40162830 8.081856 2.477649e-15
as.factor(state)WEST_VIRGINIA -1009.650163 239.58746680 -4.214119 2.806007e-05
as.factor(state)WISCONSIN 1574.866429 237.76087075 6.623741 6.589015e-11
as.factor(state)WYOMING -4137.734117 248.41143902 -16.656778 2.096972e-53
id
denoting in your model formula? Is it a unique identifier for each person nested within your states? $\endgroup$