For context

I work with a new youth outreach program that's trying to reduce incidents of STDs in our city. So far we've been using a naive approach that target everyone equally, but this has been stretching us very thin with little results... However, it turns out we have access to a lot of data that could be quite valuable (I think). Unfortunately none of us really know how to use it - yet.

The data

We have several datasets that I'd like to analyze, but there are three particularly promising ones:

  • Sparse demographic data: Race, gender, age, and some academic data (truant days, GPA, etc). The parents can opt-in so we either have all or none of this data for a student (~80% opt-in). What data we have is completely accurate.
  • Self-reported lifestyle survey: Scale questions (1-5) on different lifestyle choices. Roughly 100 questions per student covering everything from sexual activity to drug use, collected over multiple sessions. We have this data for every student in our program but it is self-reported. This is collected by a parent organization so we also have data for students that haven't even entered the program yet (!).
  • STD results: Yes/No/Not-Reported indicating whether the student tested positive for some STD in the past 6 months. The answer is accurate if the parent opt to provide results, but a sizable portion declined to report (~35%).

All datasets have a common identifying label so we can track an anonymous student's results across the data. Roughly 2000 students have yes/no STD answers and appear in both the other datasets.

The question

Given a large background dataset, how can we predict which background attributes correlate with test results? I've done some research but this is a new field for me.

Right now my thought is to join all the data into one big file, and then build two matrixes representing "Answers for students that tested positive" and "Answers for students that tested negative".

Then I was planning to use PCA to determine which questions/demographics are most important for that population. Hopefully the output will give us the most important features (eg frequent drug use as a junior or senior with no job means high probability for STDs) so we can pre-emptively target those populations.

Does this sound like a reasonable approach? Are there better methods of analysis?

What sort of literature should I be looking for if I want to learn more about this kind of analysis?


1 Answer 1


It seems to me that your question of interest is trying to predict youth, based on other risk factors, that are considered to be high risk for STDs based on test information as well as demographic information.

The relevance of using lifestyle survey information depends upon how you intend to identify eligible children for this intervention. In other words, if you incorporate that information, you will have to administer the lifestyle survey to all eligible children before deciding who may receive the information. This can be problematic in large populations. On the other hand, it may be good to assess as a secondary hypothesis. We may be interested in incorporating this information if, say, a lifestyle question can predict with 95% accuracy the risk of encountering any STD.

Predicting STD risk is a matter of building a regression model. The event of acquiring an STD is the outcome. All regressors are considered possible risk factors. A logistic regression model would be suitable if STD incidence is measured in terms of yes/no for each individual. A log linear model would achieve similar results for a table of incidence. A complementary log-log model would be appropriate if you have measured the number of encounters a person has had.

With any of these models for risk, you can compare the relative risk of outcome, and identify who is at the highest risk so that they may receive the intervention.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.