1
$\begingroup$

I'm wondering what a suitable regression model would be to predict a bounded, continuous, non-normally distributed dependent variable from a binary explanatory variable with partially crossed data.

I'm trying to see how the scoring procedure (binary) in online challenges influence the predictability of outcomes. I measure predictability as the absolute difference between the rank percentile of a participant in a challenge and the overall rating percentile of a participant at that time. If a challenge is predictable that means that highly rated participants also rank highly (i.e. low difference between these two values).

So basically: $predictability = abs(percentile(rank)_{in\_challenge} - percentile(rating)_{in\_challenge})$

So the dependent variable is the difference between two percentile values and is hence bounded between 0 and 1 (or 0 and 100 depending on the scale) and has a peculiary distribution with a couple of 0s a lot of low values and few high values. This is what the distribution of the dependent variable looks like: Distribution of the DV

Now this is obviously not normally distributed as it is bounded and the mode and median are far to the left.

Each observation (n = 1425) in my dataset is the participation of an individual in a challenge. Hence, I have partially crossed data as not all participants contribute in all of the challenges. Further, the data is divided into strata of comparable challenges that only differ significantly in the treatment variable (i.e. scoring procedure). Therefore, as I understand, I need to fit a mixed effects model with random effects for the individual and the strata as well as an interaction between the random effects. In R I use the lme4 package to do so:

lmer(predictability ~ treatment + (1|coder_id) + (1|stratum) + (1|coder_id:stratum), data = my_data)

However, I don't think an ordinary linear model is appropriate here, because as the QQ-Plot shows, the residuals are not normally distributed: QQ-Plot

Hence my question: What would be a suitable model for my data?

Thank you very much for any help.

Improve this question
$\endgroup$
7
  • $\begingroup$ Do you really need to use the absolute value on the predictability variable? If you want to use the predictability in a regression model, note that the values of the independent variables that would give a positive value of the predictability without the absolute value might be different from the values of the independent variables that result in a negative predicability without absolute value. $\endgroup$
    – Ertxiem - reinstate Monica
    Commented Apr 28, 2019 at 2:01
  • $\begingroup$ Thanks for your comment! My theory is that the treatment leads to less predictability. I don't care though if a participant does better than expected or worse than expected, I just want to know if he did differently than expected. Hence the absolute values. $\endgroup$
    – Sebas.M
    Commented Apr 28, 2019 at 2:10
  • 1
    $\begingroup$ Beta distribution comes to mind, as it is defined on the interval [0,1]. There is a CV answer with info on estimating a mixed model with a Beta-distributed response in R: stats.stackexchange.com/a/347673/241093. $\endgroup$
    – AlexK
    Commented Apr 28, 2019 at 5:00
  • $\begingroup$ Hey @AlexK, thanks for your comment! Beta regression sounds plausible indeed. $\endgroup$
    – Sebas.M
    Commented Apr 28, 2019 at 17:49
  • $\begingroup$ @AlexK Thanks for the hint. I now used linear regression, a log transformation and beta regression and they all yield similar results. One more thing that came up though is that the variance of the outcome declines with increasing rating of a participant. This is what the precision model of the beta regression shows. So it seems that the treatment doesn't have a constant effect but a higher effect on participants with lower ratings. To account for that I introduced an interaction term between treatment and rating. Is that an appropriate method? $\endgroup$
    – Sebas.M
    Commented Apr 30, 2019 at 15:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.