2
$\begingroup$

I have an outcome variable that is bimodal, this is because in about half the sample is measured from 0 to 5, and half the time from 0 to 7.

Because of the different scales, I have decided to normalize this variable, but the normalization is particular; I discuss it below.

In my study, there are two people who are assessed by three people, so there are six assessments; I rank these six assessments (1st, 2nd, 3rd..., 6th), subtract 1 so that the rank is 0th, 1st, ...5th, and divide by 5. In this way, this transformed variable goes from 0 to 1. The sample is composed of a few hundred of such six assessments (i.e. this gives about 1k assessments in total).

As mentioned above about half of the times the three assessors asses the two assessed people on a scale from 0 to 5, while other times they assess them on a scale from 0 to 7. Within the triplet of assessors there is no scale variation, and what scale is going to be used within a triplet of assessors is random.

I run an OLS with this transformed outcome on multiple explanatory variables, and cluster the standard errors at group and assessor level.

If I run an OLS on the original variable, without normalization, the residuals are not normally distributed. Moreover, any other transformation of the outcome (i.e. natural logarithm; standardization at group level; unitization at group level, with zero minimum), with clustered standard errors at group and assessor level, do not solve the non-normality of the residuals.

  • Have you heard of this type of normalization? Could you provide a reference?
  • Should I instead focus on a different regression technique? Something for binomial outcome variables?
  • Should I use the ordered logit model instead?

The outcome variable is a discrete count variable, and the range could vary a lot (i.e. the scale used by the three assessors per group varies)

Improve this question
$\endgroup$
14
  • $\begingroup$ Why is it important to obtain normally - distributed residuals? No common statistical procedure requires this. $\endgroup$
    – whuber ♦
    Commented Jun 24, 2022 at 15:53
  • $\begingroup$ the OLS assumes it: residuals should be normally distributed $\endgroup$
    – Fuca26
    Commented Jun 24, 2022 at 15:56
  • 1
    $\begingroup$ BTW, stats.stackexchange.com/questions/16381 has some good remarks about OLS assumptions. stats.stackexchange.com/questions/12053 adds to that. $\endgroup$
    – whuber ♦
    Commented Jun 24, 2022 at 16:03
  • 1
    $\begingroup$ I would mildly disagree with @whuber in two senses (while agreeing on ~everything else): 1. The derivations of the t- and F-distribution for tests of coefficients & multiple terms respectively do rely on normality of errors, though in large samples this is def. a non-issue for the type I error rate. 2. More importantly, type I error rates are not the only consideration with tests. Asymptotic efficiency (→ power) in those tests (&estimates) relies on at least approximate normality; if you had some strongly non-normal distributional model you might do considerably better to take advantage of it. $\endgroup$
    – Glen_b
    Commented Jun 25, 2022 at 2:15
  • 2
    $\begingroup$ How to do you combine a 0 to 5 scale with a 0 to 7 scale into a single outcome? This seems like a very relevant detail. And perhaps more important than the fine details how normal is normal enough for a test to be valid and with sufficient power. $\endgroup$
    – dipetkov
    Commented Jun 27, 2022 at 12:25

1 Answer 1

Reset to default
0
$\begingroup$

It turned out this is simply the percentile rank. It is used especially in education to rank test scores of e.g. NEET, GRE, SAT, LSAT.

For the second and third questions, I did not find a straightforward answer.

Perhaps, I should use a separate regression on the separate subsamples, characterized by different scales of the outcome variable. However, in this case I loose statistical power. Moreover, the outcomes are on different scales, but they are measuring exactly the same thing (e.g. imagine comparing scores across two GRE tests that have a different amount of questions).

The ordered logit model would be used to provide additional insights, but it does not solve the issue of different scales for the same outcome, across the dataset.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Just because two scales are measuring the same thing it doesn't mean that it's easy to combine them. It's not trivial in general. In your favor is the fact that the same assessor used both scales. So ask them how to map one scale to the other? $\endgroup$
    – dipetkov
    Commented Jun 28, 2022 at 10:37
  • 1
    $\begingroup$ You might find this post interesting: How to convert percentage values from 7 point scale to 5 point scale? though it's about converting the distribution of responses from one scale to the other, not mapping the scales themselves. $\endgroup$
    – dipetkov
    Commented Jun 29, 2022 at 0:25
  • 1
    $\begingroup$ You can combine different scales in a single ordered logit model. Some coefficients can be shared between the two, I think this journals.sagepub.com/doi/full/10.1177/00491241231186655 is an example. $\endgroup$
    – Gijs
    Commented Jun 25 at 8:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.