2
$\begingroup$

I have an outcome variable that is bimodal, this is because in about half the sample is measured from 0 to 5, and half the time from 0 to 7.

Because of the different scales, I have decided to normalize this variable, but the normalization is particular; I discuss it below.

In my study, there are two people who are assessed by three people, so there are six assessments; I rank these six assessments (1st, 2nd, 3rd..., 6th), subtract 1 so that the rank is 0th, 1st, ...5th, and divide by 5. In this way, this transformed variable goes from 0 to 1. The sample is composed of a few hundred of such six assessments (i.e. this gives about 1k assessments in total).

As mentioned above about half of the times the three assessors asses the two assessed people on a scale from 0 to 5, while other times they assess them on a scale from 0 to 7. Within the triplet of assessors there is no scale variation, and what scale is going to be used within a triplet of assessors is random.

I run an OLS with this transformed outcome on multiple explanatory variables, and cluster the standard errors at group and assessor level.

If I run an OLS on the original variable, without normalization, the residuals are not normally distributed. Moreover, any other transformation of the outcome (i.e. natural logarithm; standardization at group level; unitization at group level, with zero minimum), with clustered standard errors at group and assessor level, do not solve the non-normality of the residuals.

  • Have you heard of this type of normalization? Could you provide a reference?
  • Should I instead focus on a different regression technique? Something for binomial outcome variables?
  • Should I use the ordered logit model instead?

The outcome variable is a discrete count variable, and the range could vary a lot (i.e. the scale used by the three assessors per group varies)

$\endgroup$
14
  • $\begingroup$ Why is it important to obtain normally - distributed residuals? No common statistical procedure requires this. $\endgroup$
    – whuber
    Jun 24, 2022 at 15:53
  • $\begingroup$ the OLS assumes it: residuals should be normally distributed $\endgroup$
    – Fuca26
    Jun 24, 2022 at 15:56
  • $\begingroup$ No, it does not. Certain tests suppose the sampling distributions of their statistics are approximately normal, but that's all. Indeed, no test makes assumptions about the actual residuals: the assumptions are always about the (theoretical, unobservable) error terms. OLS assumes only that the error terms are independent and have a common finite variance. $\endgroup$
    – whuber
    Jun 24, 2022 at 15:57
  • 1
    $\begingroup$ BTW, stats.stackexchange.com/questions/16381 has some good remarks about OLS assumptions. stats.stackexchange.com/questions/12053 adds to that. $\endgroup$
    – whuber
    Jun 24, 2022 at 16:03
  • 2
    $\begingroup$ How to do you combine a 0 to 5 scale with a 0 to 7 scale into a single outcome? This seems like a very relevant detail. And perhaps more important than the fine details how normal is normal enough for a test to be valid and with sufficient power. $\endgroup$
    – dipetkov
    Jun 27, 2022 at 12:25

1 Answer 1

0
$\begingroup$

It turned out this is simply the percentile rank. It is used especially in education to rank test scores of e.g. NEET, GRE, SAT, LSAT.

For the second and third questions, I did not find a straightforward answer.

Perhaps, I should use a separate regression on the separate subsamples, characterized by different scales of the outcome variable. However, in this case I loose statistical power. Moreover, the outcomes are on different scales, but they are measuring exactly the same thing (e.g. imagine comparing scores across two GRE tests that have a different amount of questions).

The ordered logit model would be used to provide additional insights, but it does not solve the issue of different scales for the same outcome, across the dataset.

$\endgroup$
2
  • 2
    $\begingroup$ Just because two scales are measuring the same thing it doesn't mean that it's easy to combine them. It's not trivial in general. In your favor is the fact that the same assessor used both scales. So ask them how to map one scale to the other? $\endgroup$
    – dipetkov
    Jun 28, 2022 at 10:37
  • 1
    $\begingroup$ You might find this post interesting: How to convert percentage values from 7 point scale to 5 point scale? though it's about converting the distribution of responses from one scale to the other, not mapping the scales themselves. $\endgroup$
    – dipetkov
    Jun 29, 2022 at 0:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.