I know there are already lots of questions around this topic (especially this one and this one) but I haven't really seen anything that directly helps me (It will be obvious I'm not a great statistician, but I'll do my best to explain).
I am running an ordinal regression in R (
clmm). My response variable is a rating between 0 and 4. I have two types of explanatory variables: individual and scenario variables [let's say
Six different scenario variables (all dummies with at most 4 different values) represent potential collaboration scenarios that get rated by the respondent (between 0 and 4) creating the response variable. (Research design is a conjoint analysis; there are a total of 192 different scenarios possible)
On top of that I have a variety of individual characteristics about the respondent (age, gender, work experience, networking skills, ...) all derived from a survey.
Every respondent rates between 3 and 16 different scenarios (average 8.1); every scenario is rated by at least 8 respondents. Every respondent and every scenario have a unique identifier (called
SVid). So they are non nested within each other.
Thus the basic regression looks like this:
clm.base <- clm(rating ~ SVs + IVs, data = dt)
The hypothesis I am trying to test is that there are specific individual characteristics, that will influence the rating of the scenarios, independent of the actual content of the scenarios. Basically, some people are more or less favourable to all types of collaboration scenarios.
Now a reviewer of my paper asks me to include individual fixed effects (which in management [my field] basically means dummies for each individual). My assumption originally was that this would result in all individual variables being dropped. This is exactly what happens when I use another model (package
felm.complete <- felm(rating ~ SVs + IVs | SVid + IVid | 0 | IVid, data = dt)
In this regression basically all my variables are perfectly collinear as expected.
However, when I approximate this in the ordinal package, there is no perfect collinearity. I presume this is related that
clmm adds so-called 'random effects'. The regression takes a couple of minutes to run but eventually returns results
clmm.complete <- clmm(rating ~ SVs + IVs + (1|SVid) + (1|IVid), data = dt)
Now, the results here are pretty useless:
- All but one of my IVs are insignificant
I am trying to understand what exactly happens when adding the
(1|IVid) term in the
clmm model. If it basically adds something like an individual dummy than the fact almost everything is now insignificant is no surprise. The coefficients of the
IVid dummies would capture the effect I am looking for (some people rate all scenarios higher or lower, regardless of scenario content) most accurately.
Now I wonder whether this interpretation is correct or whether the results I got from running the simple
clm regression are just not reliable?
Concretely, I'd like to find out:
- What happens when adding a random effect to
- A laymen explanation of how the Laplace approximation works
- How to group errors around individuals when running
- Is it possible to extract the coefficients of these random effects
(1|id)for as far as there is such a thing?