# plm in fixed effects model doesn't work with id and time

I am writing currently my thesis and I am stuck with a problem. I am trying to figure out how firm level, country level and industry level variables influence corporate social responsibility. I want to add Industry (SEC) and time fixed effects. But I can run the code only with one of them both. Actually the code should look like this, where the first part are country level specific variables and TA and LV are firm specific:

within <- plm(ESG ~ VOI_AC + Political.Stability + Government.effectiveness+ Regulatory.Quality +Rule.of.law + control.of.Corruption +Press.Freedom + pdi + idv + mas + uai + GI + HD + TA + LV, data=Neu1, index=c( "SEC", "year"),  model="within")


I tried some trouble shooting like:

Neu1\$year <- group_indices(Neu1, year, SEC.NAME)


and some combinations out of this. I know the problem is, that I have duplicate observations per Industry and year. But this is, because different firms will be in the same industry in one year. I can't get rid of the error:

  duplicate couples (id-time)
Zusätzlich: Warnmeldungen:
1: In pdata.frame(data, index) :
duplicate couples (id-time) in resulting pdata.frame
to find out which, use e.g. table(index(your_pdataframe), useNA = "ifany")
2: In is.pbalanced.default(index[[1]], index[[2]]) :
duplicate couples (id-time)


The first part of my data looks like this:

The last part of my data looks like this:

I am thankful for any help. I am a newcomer to R and I really read a lot to this problem, but I didn't find a workaround.

• Is it possible to post a sample of your data? If you could provide data for a few firms, some of which are in the same industry and some of which are in more than one industry that would be helpful. – Erik Ruzek Jan 25 at 3:30
• Sorry if I didn't point it out in a clear way, the firms always belong to one industry. I will add an example. – Julia Pfaller Jan 25 at 11:22
• Is there any reason why you don't just use a multilevel (mixed effects) model ? I assume that firms are nested in (belong to) one industry, while firms are also nested in country ? Is there any other level of nesting/grouping? It sounds very much like a multilevel model would be appropriate here. – Robert Long Jan 25 at 12:11
• Yes makes sense to me that this is more appropriate. I wasn‘t sure how to set up the model. I was on the wrong path and I had the same clue today. But is in this case the lmer function more suitable or is plm also working? – Julia Pfaller Jan 25 at 13:57
• I don't know anything about plm but lmer is fairly standard for multilevel models and is very flexible in terms of specifying nested (or crossed) factors. I will write an answer shortly. – Robert Long Jan 25 at 16:07

I would suggest fitting a multilevel model, with company/firm nested within industry and firm also nested within country. This is just a special case of a mixed effects model and could be specified with this kind of formula (using the notation adopted by the lme4 library and others):

ESG ~ fixed_effects + (1 | industry) + (1 | industry:firm) + (1| country) + (1 | country:firm)


which is equivalent to:

ESG ~ fixed_effects + (1 | industry/firm) + (1| country/firm)


So here we are fitting random effects (random intercepts) for the grouping factors, to handle the non-independence of observations within in grouping factor. In the mixed model framework, anything that is not specified in the random part of the formula is a fixed effect. Note that it is also possible for fixed effects to be allowed to vary within levels of the random intercept, by specifying them as random slopes. For example, if we wanted to allow for a fixed effect X to also vary within levels of country we could do so like this:

ESG ~ X + other_fixed_effects + (1 | industry) + (1 | industry:firm) + ( X | country) + (1 | country:firm)


...and this is why it is sometimes better to write the expanded version of the formula

Note that with mixed models methodology it is not necessary to specify at what level a particular variable varies - provided that the nesting is specified correctly, the model will automatically handle variables that vary at the country level, or the industry level or the firm level. If you happen to specify a variable to be a random slope, ie to vary within by levels of a grouping variable, but it does not vary at that level, then the random effects will not be identified and you should get a singular fit or an error.

This is all in the documentation of package plm, e.g., the package's vignette. I believe you got confused due to the various fixed effects you would like to estimate.

You would need to specify correctly what the observational units (individual dimension) and what the time dimension of your data is and put those into the index argument. If you look at firms, the industry (variable SEC) is not the individual dimension but firm is the individual dimension (I will assume firm indentifiers are in variable firm; looking at your data, it seems like it is ASSET4.Company.Name).

Now, I suggest you set up the data for panel structure first and then estimate the model (I consider this a somewhat cleaner approach). Set up data:

pdata <- pdata.frame(your_data, index = c("firm", "year"))


Let's fist estimate a two-way fixed effects model with individual (firm) and time fixed effects:

fe1 <- plm(ESG ~ VOI_AC + Political.Stability + Government.effectiveness + Regulatory.Quality +Rule.of.law + control.of.Corruption + Press.Freedom + pdi + idv + mas + uai + GI + HD + TA + LV, data = pdata,  model = "within", effect = "twoways")


To add more fixed effects, include them in the formula with factor(your_variable) like this for industry:

fe2 <- plm(ESG ~ factor(SEC) + VOI_AC + Political.Stability + Government.effectiveness+ Regulatory.Quality + Rule.of.law + control.of.Corruption + Press.Freedom + pdi + idv + mas + uai + GI + HD + TA + LV, data = pdata,  model="within", effect = "twoways")


The approach by Robert with the R packages lme4 (or nlme) is a maximum likelihood approach. plm's vignette contains a comparision and there are some good questions and answers about both approaches here on Cross Validated. What I laid out here is probably more what econometricians use and what you originally asked for.

• I've just been looking at the docs for plm. Very interesting, and thanks for pointing me towards it (+1 btw). I wonder what the econometricians "point of view" is, as they say there (without saying what their POV is)? Frankly I think it's a bit unhelpful to the broader picture for econometricians to distance themselves from statisticians. Can you shed any light on this ? Perhaps a topic to discuss in chat ? – Robert Long Jan 25 at 18:40
• I would not frame it as econometricians distances themeselves from statisticians. It is rather about different swings within statistics (GLS and ML frameworks, where, e.g., accounting and finance empiricists typically use the first while social studies typically use the last, best to my knowledge). Here is a discussion for further insights: stats.stackexchange.com/questions/238214/… – Helix123 Jan 25 at 19:03
• But saying "from the econometrician’s viewpoint" is literally making themselves seem different - ie distancing themselves. Anyway, it's not that important (but it still bothers me) - thanks for the link, I had not seen that before :) – Robert Long Jan 25 at 19:13
• My reading of "from the econometrician’s viewpoint" is: It is a heads up what you are going to get and that is a good because there are different strands/frameworks in the literature. And there is a little chapter to both views - something found too seldom. – Helix123 Jan 25 at 19:38
• Just to chime in, and this would be a good topic post, I think the big difference is that some (many) econometricians are very uncomfortable with the assumptions about the group random error term in mixed models. They worry a lot about endogeneity (correlation of predictors with the error term) and by using fixed effects, the grouping variable goes into the fixed part of the model so no assumptions to deal with other than OLS assumptions. They also don't seem to care as much about the generalization properties of a random effects estimator. – Erik Ruzek Jan 25 at 21:22