7
$\begingroup$

The problem I'm trying to solve here is very simple but the available data is very limited. That makes it a hard problem to solve.

The available data are as follows:

  1. I have 100 patients and I need to rank order them in terms of how healthy they are.
  2. I only have 5 measurements for each patient. Each of the five readings is coded as a numeric value, and the rule is that the bigger the reading the healthier is the patient.

Should I have some sort of doctor's "expert judgement based ranking" I could use that as the target variable and fit some sort of an ordinal logistic regression model trying to predict doctor's assessment. However, I don't have that. The only thing I have is (1) and (2).

How would you come up with a simple "scoring" algorithm which would combine those five measurements into a single score which would be good enough (not perfect) in rank ordering patients?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
2
$\begingroup$

A simple approach would be to calculate the sum score or the mean. Another approach would not assume that all variables are of equal importance and we could calculate a weighted mean.

Let's assume we have the following 10 patients and variables v1 to v5.

> set.seed(1)
> df <- data.frame(v1 = sample(1:5, 10, replace = TRUE),
+                  v2 = sample(1:5, 10, replace = TRUE),
+                  v3 = sample(1:5, 10, replace = TRUE),
+                  v4 = sample(1:5, 10, replace = TRUE),
+                  v5 = sample(1:5, 10, replace = TRUE))
> 
> df
   v1 v2 v3 v4 v5
1   2  2  5  3  5
2   2  1  2  3  4
3   3  4  4  3  4
4   5  2  1  1  3
5   2  4  2  5  3
6   5  3  2  4  4
7   5  4  1  4  1
8   4  5  2  1  3
9   4  2  5  4  4
10  1  4  2  3  4

1. Sum score and ranks

> df$sum <- rowSums(df)
> df$ranks <- abs(rank(df$sum) - (dim(df)[1] + 1))
> df
   v1 v2 v3 v4 v5 sum ranks
1   2  2  5  3  5  17   4.0
2   2  1  2  3  4  12   9.5
3   3  4  4  3  4  18   2.5
4   5  2  1  1  3  12   9.5
5   2  4  2  5  3  16   5.0
6   5  3  2  4  4  18   2.5
7   5  4  1  4  1  15   6.5
8   4  5  2  1  3  15   6.5
9   4  2  5  4  4  19   1.0
10  1  4  2  3  4  14   8.0

2. Mean score and ranks (note: ranks and ranks2 are equal)

> df$means <- apply(df[, 1:5], 1, mean)
> df$ranks2 <- abs(rank(df$mean) - (dim(df)[1] + 1))
> df
   v1 v2 v3 v4 v5 sum ranks means ranks2
1   2  2  5  3  5  17   4.0   3.4    4.0
2   2  1  2  3  4  12   9.5   2.4    9.5
3   3  4  4  3  4  18   2.5   3.6    2.5
4   5  2  1  1  3  12   9.5   2.4    9.5
5   2  4  2  5  3  16   5.0   3.2    5.0
6   5  3  2  4  4  18   2.5   3.6    2.5
7   5  4  1  4  1  15   6.5   3.0    6.5
8   4  5  2  1  3  15   6.5   3.0    6.5
9   4  2  5  4  4  19   1.0   3.8    1.0
10  1  4  2  3  4  14   8.0   2.8    8.0

3. Weighted mean score (i.e. I assume that V3 and V4 are more important than v1, v2 or v5)

> weights <- c(0.5, 0.5, 1, 1, 0.5)
> wmean <- function(x, w = weights){weighted.mean(x, w = w)}
> df$wmeans <- sapply(split(df[, 1:5], 1:10), wmean)
> df$ranks3 <- abs(rank(df$wmeans) - (dim(df)[1] + 1))
> df
   v1 v2 v3 v4 v5 sum ranks means ranks2   wmeans ranks3
1   2  2  5  3  5  17   4.0   3.4    4.0 3.571429    2.5
2   2  1  2  3  4  12   9.5   2.4    9.5 2.428571    9.0
3   3  4  4  3  4  18   2.5   3.6    2.5 3.571429    2.5
4   5  2  1  1  3  12   9.5   2.4    9.5 2.000000   10.0
5   2  4  2  5  3  16   5.0   3.2    5.0 3.285714    5.0
6   5  3  2  4  4  18   2.5   3.6    2.5 3.428571    4.0
7   5  4  1  4  1  15   6.5   3.0    6.5 2.857143    6.0
8   4  5  2  1  3  15   6.5   3.0    6.5 2.571429    8.0
9   4  2  5  4  4  19   1.0   3.8    1.0 4.000000    1.0
10  1  4  2  3  4  14   8.0   2.8    8.0 2.714286    7.0
Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks. My measurements are on different scale. For example v1 goes from 0 to 5 v2 goes from 1000 to 2000 etc.... How do you recomend dealing with that? $\endgroup$
    – user333
    Jun 3, 2011 at 7:43
  • $\begingroup$ (1) You really want to update your question. More experienced people might come up with better solutions since I am really not an expert for this kind of questions. (2) You could standardize your measures (z-score) and then proceed as explained above. (3) Are your measures correlated? Do they measure the same theoretical construct. If yes, you could run sth like a factor analysis. $\endgroup$
    – Bernd Weiss
    Jun 3, 2011 at 9:46
  • $\begingroup$ @bremd Is there any "quantitative" method of coming up with weights ... I'm primarily thinking of using correlation and or multicolinearity as a valuable source of information? $\endgroup$
    – user333
    Jun 29, 2011 at 7:44
  • $\begingroup$ @user333 I am sorry but I am not aware of such a method. I had theoretical weights in mind... $\endgroup$
    – Bernd Weiss
    Jun 29, 2011 at 10:35
5
$\begingroup$

Any function $f: \mathbb{R}^5 \to \mathbb{R}$ that is separately increasing in each of its arguments will work. For example, you can select positive parameters $\alpha_i$ and any real parameters $\lambda_i$ and rank the data $(x_1, x_2, x_3, x_4, x_5)$ according to the values of

$$\sum_{i=1}^{5} \alpha_i (x_i^{\lambda_i} - 1) / \lambda_i \text{.}$$

Evidently some criterion is needed to select among such a rich set of distinctly different scores. In particular, the simple obvious solutions (frequently employed, unfortunately) of just summing the scores or first "normalizing" them in some fashion and then summing them will suffer from this lack of grounding in reality. To put it another way: any answer that does not derive its support from additional information is a pure fabrication.

Because this problem is essentially the same as Creating an index of quality from multiple variables to enable rank ordering, I refer you to the discussion there for more information.

Improve this answer
$\endgroup$
0
$\begingroup$

I would just simply sum them up, weighting each factor if necessary.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The scale of the measurements is very different... Would need to normalize first! $\endgroup$
    – user333
    Jun 2, 2011 at 22:26
  • $\begingroup$ How do you recommend I derive weightings $\endgroup$
    – user333
    Jun 2, 2011 at 22:27
  • 1
    $\begingroup$ @user333 Wouldn't you agree that the information that the "scale of the measurements is very different" is an important information? You might want to update your question. $\endgroup$
    – Bernd Weiss
    Jun 2, 2011 at 22:40
-1
$\begingroup$

How about generating a synthetic binary target variable first and then running a logistic regression model?

The synthetic variable should be something like... "If the observation is in the top decile on all of the input variable distributions flag it as 1 else 0"

Having generated the binary target variable... Run logistic regression to come up with probabilistic metric 0 to 1 assesing how far/close in the tails of multiple distributions observation is?

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ @user That sounds like a sophisticated way to cover up the fact that you are making up the answer! $\endgroup$
    – whuber ♦
    Jun 8, 2011 at 19:00
  • $\begingroup$ How do you mean that? The data will speak for itself... I'm more concerned about the methodology... Is it sensible thing to do $\endgroup$
    – user333
    Jun 8, 2011 at 20:07
  • 1
    $\begingroup$ @user There are limits to what data can tell you. From the original description, they contain no information concerning which of the five factors (or combinations thereof) deserve the greatest weights when assessing overall health. Of course you have partial ranking information given by dominance: when all the readings of one patient exceed the corresponding readings for another, the first patient is healthier. But you have to get beyond this and that involves calculating trade-offs. Logistic regression won't do that for you. $\endgroup$
    – whuber ♦
    Jun 8, 2011 at 20:12
  • $\begingroup$ Thanks for good comment. What exactly do you mean by calculating trade-offs? Can you give me one example? $\endgroup$
    – user333
    Jun 8, 2011 at 21:01
  • 2
    $\begingroup$ @user Consider two specific readings per patient, one on a scale from 0 to 10 and another on a scale from 0 to 100. If a patient with readings of (10, 50) is as healthy as a patient with readings of (5, 70), then 10-5 = 5 units of the first reading have been traded off for 70-50 = 20 units of the second. The questions to answer are (1) are the trade-offs constant across levels; e.g., is a (0, 90) also as healthy as a (5, 70); and (2) does the amount of trade-off depend on the levels of the third, fourth, and fifth variables? $\endgroup$
    – whuber ♦
    Jun 8, 2011 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.