1
$\begingroup$

I am working with a high-dimensional dataset (e.g., ~150 features) and possible confounders (e.g., ~10 features to control). The goal is to identify association with an outcome variable that is categorical. Some example problems where this is relevant are genomics and protoemics. You would have a dataset where each row is an individual and columns are genes. Confounding variables include age, weight, gender, blood pressure, etc. Outcome variable would be a clinical feature, like presence or absence of a disease.

As a first step to approach this problem, I identify individual associations, i.e., perform 150 regressions, and use False Discovery Rate to find significant associations.

In addition, I am interested in identifying groups of features that together are associated with the outcome variable. I do not know a straightforward method for this kind of analysis. Association rule mining comes to mind, but I wanted to know if there are standard procedures that give associations along with their statistical significance. The hypothesis here is that usually it is not an individual gene, but groups of several genes acting together that influence the outcome. There could be more than one group which are associated, and the goal would be to rank those groups by the magnitude of their associations. One possible problems is correlation among the features.

This could sound like an exploratory question, but knowing more about possible directions would also be of great help. The reason is that although I have a fairly broad knowledge of statistics and machine learning, I do not remember seeing these problems in standard textbooks or coursework. Pointers to research papers that detail such methods are welcome too. So are ways to utilize non-traditional methods like decision trees, or neural networks to identify such associations (while being less explainable).

Improve this question
$\endgroup$
1
  • $\begingroup$ We are going to need more information. Please tell us more about the data and where they come from ? What is your research question(s) ? Do you know all of the potential confounders, you mentioned that you know around 10 - are the remaining featues known to be not confounders ? Also, a confounder is specific to a particular causal path. So a confounder for $X_1 \rightarrow Y$ may not be a confounder for $X_2 \rightarrow Y$. It might be a mediator and therefore you would want to exclude it from the latter model but include it in the former. $\endgroup$
    – Robert Long
    Commented Jul 31, 2020 at 5:53

1 Answer 1

Reset to default
1
$\begingroup$

Challenging question. Groups of feature variables are likely to show some deeper dependencies. You have a large search space and the optimal classifier is really difficult to find. The discriminative performance of your classifier is the only criterion that is useful in determining whether a subset of features predict your categorical outcome well.

My suggestion is that you perform a parallel search for the most well-performing feature subsets in terms of classifier performance. On such approach is that by Siedlecki and Sklansky. They developed a promising genetic algorithm approach for walking through the feature space. You can use their approach to feature selection with for example neural networks - or random forest classifiers.

Reference

Siedlecki W., & Sklansky J. (1989). A note on genetic algorithms for largescale feature selection. Pattern Recognition Letters, 10 (5), 335–347.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.