9
$\begingroup$

In SVM (linear kernel) classification analyses of a data-set of gene expression (~400 variables/genes) for ~25 each of cases and controls, I find that the gene expression-based classifiers have very good performance characteristics. The cases and controls do not differ significantly for a number of categorical and continuous clinical/demographic variables (as per Fisher's exact or t tests), but they do differ significantly for age.

Is there a way to show that the classification analysis results are or are not influenced by age?

I am thinking of reducing the gene expression data to principal components, and doing a Spearman correlation analysis of the components against age.

Is this is a reasonable approach? Alternately, can I check for correlation between age and class-membership probability values obtained in the SVM analysis.

Thanks.

$\endgroup$
  • 1
    $\begingroup$ Is this a case-control study? Or cohort study? Why is there an age difference (sampling scheme? pathomechanism?)? Is age age at diagnosis? Or is this a chronic disease and age is current age at taking tissue sample for gene expression analysis? Is age known to be related to the disease? Is the age effect on gene expression more the effect of time since birth or since diagnosis? --- I would need the answers to these questions to see your question if "the classification analysis results are or are not influenced by age?" in proper perspective. $\endgroup$ – GaBorgulya Apr 6 '11 at 0:07
  • $\begingroup$ This is a retrospective study on blood microRNA expression and lung cancer. The cases have lung cancer. The controls do not and were chosen from patient population appearing at a lung cancer screening clinic usually because of a history of smoking. Matching for age, gender, etc., was not done when selecting cases and controls. Lung cancer typically is diagnosed after 45-50 y of age. It is not known if blood microRNA expression is affected by lung cancer, but some other diseases are known to affect expression. $\endgroup$ – user4045 Apr 6 '11 at 3:50
  • $\begingroup$ The effect of age on blood microRNA expression is unknown. The mean (and std. deviation) age of cases and controls of the study are 71 (7) and 60 (9) y, respectively. $\endgroup$ – user4045 Apr 6 '11 at 4:01
  • $\begingroup$ When you say "influenced by age", what exactly do you mean? Here are two possibilities. One possibility is that your microarrays contain no disease markers whatsoever. But, they do contain information about age, and since in your case the sick and control populations are of different age, you get the illusion of good classification performance. Another possibility is that the microarrays do contain disease markers, and, moreover, these markers is exactly what SVM focuses on. However, since in your data the ages are different, there is still correlation between age and category. $\endgroup$ – SheldonCooper Apr 6 '11 at 18:37
  • $\begingroup$ @SheldonCooper: Right, and I want to know if we can or cannot figure out which of the two possibilities it is. If not, can we roughly estimate the extra value the gene markers provide over age? The SVM classifier has good performance characteristics (accuracy in internal cross-validations >90%, and AUC >0.95). AUC in ROC analysis of age is 0.82. $\endgroup$ – user4045 Apr 6 '11 at 19:20
2
$\begingroup$

There are at least two possibilities for this data. One possibility is that your microarrays contain no disease markers whatsoever. But, they do contain information about age, and since in your case the sick and control populations are of different age, you get the illusion of good classification performance. Another possibility is that the microarrays do contain disease markers, and, moreover, these markers is exactly what SVM focuses on.

It seems like the principal components of the data may be correlated with age in both of these possibilities. In the first case it will be because age is what the data expresses. In the second case it will be because disease is what the data expresses, and this disease is itself correlated with age (for your dataset). I don't think there is an easy way to look at the correlation value and conclude which case it is.

I could think of several ways to assess the effect differently. One option is to split your training set into groups of equal age. In this case, for 'young' ages the normal class will have more training examples than the disease class, and vice versa for the older ages. But as long as there are enough examples, this should not be a problem. Another option is to do the same with the test sets, i.e. see whether the classifier tends to say 'sick' more often for older patients. Both of these options could be difficult since you don't have that many examples.

One more option is to train two classifiers. In the first, the only feature will be the age. It seems this has AUC of 0.82. In the second, there will be age and the microarray data. (It seems that currently you train a different classifier which only uses the microarray data, and it gives you AUC 0.95. Adding the age feature explicitly is likely to improve performance, so AUC will be even higher.) If the second classifier performs better than the first, this indicates that age is not the only thing of interest in this data. Based on your comment, the improvement in AUC is 0.13 or more, which seems fair.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the various suggestions. I think you are right that checking for correlation of age with principal components doesn't provide an answer. I did do that analysis and there are good correlations (Spearman r > 0.5) for each of the first three PCs (they together contribute to ~55% of variance). There is also a good correlation of age with the probability values from the SVM analysis. For the first two options you suggest, I have to check if there are enough samples and how to go about it (I use LOOCV and 1000-iteration Monte Carlo CV with 4:1 split for training and testing). $\endgroup$ – user4045 Apr 6 '11 at 22:38
  • $\begingroup$ Regarding ROC using both age and microarray data, I will try it. An increase in AUC from 0.95 (microarray data alone) will suggest that the expression data has disease-specific information that is independent of age. An absence of an increase, however, will mean nothing since the expression data is affected by age. Right? $\endgroup$ – user4045 Apr 6 '11 at 22:45
  • $\begingroup$ You already have an increase in AUC, from 0.82 for age only to 0.95 for microarray. This is what's important I think. If you get further increase, great. If you don't get further increase, you are right that it doesn't mean anything. The important part is that you have the increase from 0.82 to 0.95. $\endgroup$ – SheldonCooper Apr 6 '11 at 23:03
  • $\begingroup$ In a new analysis, with age added as a variable to the expression data-set, the AUC increases ~0.04. I guess one cannot conclude anything from this. $\endgroup$ – user4045 Apr 7 '11 at 1:51
  • $\begingroup$ Is the new AUC (for age + microarray) 0.99, or is it 0.86? $\endgroup$ – SheldonCooper Apr 7 '11 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.