6
$\begingroup$

There is currently a debate whether the EQ-5D score that has a ceiling problem and a bimodal distribution can be used in a linear regression model or not.

Background

The score is very simple and frequently used to assess patient's health related quality of life and consists out of five questions where each has 3 possible answers (there is a newer with 5 answers but it's less used).

The score is commonly used in national registries as a patient reported outcome measure (PROM) and is very convenient because the questions are easy to answer and the completeness is therefore good.

Continuous score

The score is created by using a "tariff" where unique combinations of the 5 variables translate into a continuous-like variable but with the above mentioned limitations. I'm not sure how they decide on the tariff score but the different combinations of the answers combine into a unique value, for instance if you have answered best health on all five categories you get a code of 11111 that gives the maximum of 1.000. If you've answered best on the first 2 question and worst on the last 3 you have a code of 11333 and get a score of -0.066. The score is country-adjusted and ranges between -0.594 to 1.000 in my Swedish tariff.

The Paretian calculation

In most orthopaedic studies we have a preoperative score and a postoperative score. By comparing the two models as the Paretian Classification of Health Change suggests we get four possible outcomes; no change, worse, improved, or mixed change. Mixed meaning that one category became improved while another one deteriorated. As I understand the Paretian outcome is best analyzed using a multinomial logistic model.

My questions

  • When having large datasets of > 10 000 patients does it matter that the score is not normally distributed and is the Paretian way of analyzing the score better?
  • Scores like this are very frequently used today - what are the limitations?

Update

After taking all these wise arguments and discussing them closer with our statistician I got some interesting input:

  1. In large sample the central limit theorem will kick in as long as the sample isn't heavily skewed
  2. If the score itself has a flaw (as the EQ-5D score) it might not be right to expect a normal distribution because the bimodality is not due to a subgroup but due to a score feature (I think this is a different way of putting what @whuber wrote: "... The residuals will closely reflect that error distribution"
  3. The normality of the sample helps in calculating the p-value/confidence interval and this could be circumvented by using bootstrapping
  4. Using ordinal regression and leaving out the mixed group we can validate the results from the linear regression - i.e. show that the predictors behave similarly when used "non-parametric"
Improve this question
$\endgroup$
4
  • $\begingroup$ Hi Max, how is the score from the test use?. Assuming the possible total score ranges from 5 to 15, are the incremental changes in score important? $\endgroup$
    – Michelle
    Feb 9, 2012 at 4:27
  • $\begingroup$ Hi Michelle, I've added what I know of how the score is calculated and tried to clarify the question some. I think the main discussion is if it's OK to have a non-normal variable in a regression as the outcome or if this breaks some fundamental assumption. I'm not that familiar with multinomial regression so I'm also a little curious of how this impacts the interpretation, it's much easier to have a score being ".2 better in females" than "20% more improved in females and 30% more mixed results in females" $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 5:33
  • $\begingroup$ I'm not familiar with this usage of the word "tariff"; I've tried googling it but all I can find are references to taxation :) I think you just mean the rules/weights by which the score is produced, but if I'm wrong, let me know. $\endgroup$
    – Jonathan
    Feb 9, 2012 at 19:45
  • $\begingroup$ Sorry if that's unclear - I guess it's more commonly used in Swedish where it's like a taxtable - depending on your answers you're taxed differently. What I mean is simply the way the answers are translated into a "continuous variable". I've changed the question to clarify. $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 22:10

3 Answers 3

Reset to default
1
$\begingroup$

First, categorizing continuous variables is generally a bad idea; Royston, Altman and Saurbrei wrote a good article on why dichotomizing is bad, and the same arguments apply to more categories. Altman wrote an article on categorizing variables, but only the abstract is freely available, and I have not read the whole article.

Second, the assumptions of linear regression are not that the dependent variable is normally distributed, but that the residuals from the model are. So, before you can see if your model violates the assumptions, you need to run it and look at the results.

Third, if the residuals are not normally distributed, you have several choices:

  1. Multinomial logistic regression with the four categories you list
  2. Ordinal logistic regression with "mixed" excluded.
  3. Looking at each category separately
  4. Some sort of robust regression

Before doing any of these, my impulse would be to look at the variables graphically, with density plots and possibly quantile normal plots.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thank you very much for your answer, I'll check how it behaves with a qqplot or is there a better method that you suggest? Part of my question is in regard to very large series, when looking at the central limit theorem almost the underlying distribution quickly disappears when n is large. In registry studies the n is > 30 000 and therefore even if the qqplot would suggest some non-normal distribution I wonder if this really matters? $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 16:59
  • $\begingroup$ I'm not sure if I understand what you're referring to in your comment on categorical variables; if you're referring to the design of the survey itself, I would just add that the ease of answering a categorical question is typically much higher than a continuous question, and can make for higher-quality responses (lower respondent fatigue) and higher response rates, which can be beneficial for overall sample quality and also for operational purposes. $\endgroup$
    – Jonathan
    Feb 9, 2012 at 19:47
  • 1
    $\begingroup$ Max, the CLT is not relevant. As the amount of data go up, you collect more and more draws from the error distribution. The residuals will closely reflect that error distribution. Thus, their empirical density (think: histogram) should be converging towards their theoretical ("population") density, whatever shape it may have. The CLT would apply to statistics like averages of the errors, but such statistics are just not germane to the question. $\endgroup$
    – whuber ♦
    Feb 9, 2012 at 22:12
  • 1
    $\begingroup$ @whuber: Thank you for your input. So if I know that this population has a bimodal distribution, should I then expect the residual histogram to have a bimodal distribution? $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 22:56
  • 1
    $\begingroup$ The hope, Max, is that the independent variables give you so much information about the dependent variable--including which mode it is near--that the residuals wind up being centered at zero, unimodal, and symmetric (all approximately, of course). (Bimodal residuals usually suggest some factor has not been accounted for in the regression.) $\endgroup$
    – whuber ♦
    Feb 10, 2012 at 0:18
8
$\begingroup$

One question that you ought to ask first is whether or not using a weighted score is correct for the kind of analysis that you want to do. There is a discussion of that in Parkin, D., Rice, N. and Devlin, N. (2010) Statistical analysis of EQ-5D profiles: Does the use of value sets bias inference? Medical Decision Making. 30(5), 556-565 doi: 10.1177/0272989X09357473 The Paretian classification is described and discussed in Devlin, N., Parkin, D. and Browne, J. (2010) Patient-reported outcomes in the NHS: New methods for analysing and reporting EQ-5D data. Health Economics. 19(8), 886-905.DOI: 10.1002/hec.1608

If using the score is OK, what really matters is the distribution of the regression residuals, not the score. If you are using the Paretian classification, you are correct that an ordered model can't be used.

'Tariff' is a slightly silly label, but for historical reasons has been used in this context. Just means a set of scores attached to categorical EQ-5D health states.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 This kind of well-researched, authoritative, and clear answer is what we aim for in all threads. $\endgroup$
    – whuber ♦
    Feb 11, 2012 at 15:35
  • 1
    $\begingroup$ +1 for an interesteing, as whuber wrote, well-researched answer. $\endgroup$
    – Max Gordon
    Feb 11, 2012 at 19:37
0
$\begingroup$

if you have some predictor variables (which I'm assuming you have as you mention regression in your question), I'm wondering if an ordinal logistic regression, using the Paretian measure as the dependent variable (which appears to be ordered categories of pre- versus post- differences), is the best way forward. I love the UCLA websites for the clarity of their explanations of various methods, here is their outline of ordinal logistic regression using SPSS and here is their example for Stata. As you can see from these sites, you will need to verify that the proportional odds assumption is met with your data.

Ordinal logistic regression is an accepted statistical method, and Professor Agresti has written about it in his books on categorical data analysis. I recommend buying any of his books if your work is taking you down categorical data analysis paths.

Improve this answer
$\endgroup$
4
  • $\begingroup$ My initial thought was an ordinal logistic regression but then I noticed the annoying "mixed" outcome that is hard to order in comparison to the "no change". I'm also very interested to know exactly why assumptions of linear regression are broken in this case as some say it is. $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 6:15
  • $\begingroup$ Can you do any investigations with the patients in the mixed category. Are there any clinical hallmarks of this group, e.g. on one particular measure of the test? $\endgroup$
    – Michelle
    Feb 9, 2012 at 8:20
  • $\begingroup$ Thank you for your suggestions. What I'm mostly interested in is trying to figure out exactly when I need to fall back to a categorical methods. $\endgroup$
    – Max Gordon
    Feb 9, 2012 at 17:03
  • $\begingroup$ The mixed category will present a challenge for any method where you're looking at ordered outcomes. So even in the case of using the score as a continuous measure, rather than a category, it sounds like the mixed scores will be an issue because they don't "fit" with an increasing score representing better functioning. $\endgroup$
    – Michelle
    Feb 9, 2012 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.