Create composite score from categorical and binary variables

Question

Let's say I have a dataset with a number of variables on clinical history and behaviours in the context of COVID transmission. Ultimately i'd like to create a binary variable that is an indicator of COVID risk (though I do NOT have access to any variables about whether they have/had COVID). So each person in the dataset would be classified as either High or Low risk, based upon about 6 categorical indicators.

For all 6 variables we know theoretically, which direction the risk would go. For example, let's say these are our 6 variables:

1. Diabetes (Yes/No) Yes would be higher risk
2. Obesity (Yes/No) Yes would be higher risk
3. Smoker (Yes/No) Yes would be higher risk
4. Works in public (Yes/No) Yes would be higher risk
5. Doesn't wear mask (Yes/No) Yes would be higher risk
6. Doesn't wash hands (Yes/No) Yes would be higher risk

Would it be a valid thing to do, to score Yes's to each question with a "1". Then add up the scores. Then say something like every one with >=4 is "High Risk"?

Or is there another technique you'd use to create the Risk variable, in absence of the outcome?

I'm familiar with cluster analysis, but not really sure if it would work well on only a few variables. And I might run the risk it not being clear which cluster is high vs low risk. I know PCA can't typically be used with categorical variables. Anything else I should consider?

Answer 1

The best thing to do here is really to get labeled data. It will do far more for you than any attempt to solve this problem. Especially in something as life-and-death crucial as COVID, deploying a poor model is unethical.

What you’ve suggested is to take an unweighted average and threshold it: $\mathrm{COVID?} = \left[ 4 \geq \sum_{i=1}^6 f_i\right]$, using the Iverson bracket notation. The issue here is that certain features may be more important than others, and it’s hard to choose where to place the threshold.

---

A principled way of learning from partially observed data is a generative model. Naive Bayes is a generative model of the joint distribution of classes and features $p(c, \vec{f})$. Normally we train it with supervision about the true class $c$, but we can also train it unsupervised on the unlabeled data: maximize $p_\theta (\vec{f}) = \sum_c p_\theta (c, \vec{f})$. This can be done either with EM or by directly optimizing the marginal likelihood by gradient ascent.

If you decide in advance that there are two classes, you can cluster the data points into two sets based on which class has a higher $p(c \mid \vec{f})$. (This decision rule is separate from the probabilistic model. When you actually decide which points to put into which cluster, you may want to tweak the threshold.) Alternatively, you could use the ratio $\frac{p (c_2 \mid \vec{f})}{p (c_1 \mid \vec{f})}$ to rank them.

---

You mentioned PCA. You’re right that PCA (as a probabilistic model) isn’t appropriate for categorical variables. What I’ve suggested is a special case of exponential family PCA, which generalizes PCA to distributions like the Bernoulli distribution. That makes it appropriate for your binary variables.

You aptly note that the true class is hard to find. One heuristic is to say that the cluster with more false variables overall is the not-high-risk cluster.

In short, your intuitions about how to approach this problem in the absence of data are good. Getting more data is probably better.