0
$\begingroup$

To briefly explain what I've been working on,

  • I have two datasets: one as a reference(larger) and another for validation.
  • I performed the PCA analysis with my reference dataset to create the corresponding PC space (1).
  • Then, I tried to project my test data on this PC space created with the reference set (2).

The two datasets I have are of same type (gene expression data) but were generated from different sources(or cohorts/studies).

I read that it is recommended to scale and center the data when running the PCA analysis, so I did them on step (1). And the output provided the scaling coefficients and centering factors used.

My question is:

  • When I project my test data on the PC space created with the reference data, do I just normalize the data with the scaling coefficients and centering factors obtained from the PCA analysis using the reference set and multiply it by the rotation matrix ?

  • OR, is it okay if I initially normalize my test data (Z-score normalization within itself), then use these values for the projection (apply scaling coefficients and centering factors + rotation matrix derived from the reference dataset)?

Sounds like the second question is to normalize the test data twice (within itself and with the values from the reference).

I've tried both and found that the results are quite different. If my second question is of misconception, can you please explain me why?

I'm not proficient at data science techniques or stats, so my questions might sound silly but I would really appreciate it if you could provide a sincere answer.

$\endgroup$
4
  • 1
    $\begingroup$ Has calibration transfer studies been carried out to ensure that the actual technical output of the assays in each cohort is equivalent? This usually is much more subtle than mean and variance scaling. It requires correction for process specific biases so that the output is directly comparable. If you don't correct these biases first they will just pop up in a transformed way after scaling. $\endgroup$
    – ReneBt
    Sep 25, 2021 at 4:44
  • $\begingroup$ In addition to the critical issue raised by @ReneBt about technical equivalence between the reference and validation sets, how many cases are there in each of those sets? Separating data sets like that can hurt precision both in the modeling and in the validation. $\endgroup$
    – EdM
    Sep 25, 2021 at 14:41
  • $\begingroup$ As I said, the reference and validation dataset are from different sources but of same type. The reference set contains 104 cases and 150 controls, while the validation set contains 97 cases 27 controls. Is there any rule that I need to follow when preparing data for tasks like this? (Thanks for the feedback above @ReneBt and @EdM) $\endgroup$
    – Jay
    Sep 25, 2021 at 15:31
  • $\begingroup$ Data arising from chemical and instrumental analysis will need to be calibrated and standardised according to established calibration transfer methods for the assay type. Usually there will be standards run as part of the analysis that allows comparison of different sources, but these may not be available unless specifically requested. You would use these standards to compare sources. $\endgroup$
    – ReneBt
    Sep 26, 2021 at 6:12

1 Answer 1

1
$\begingroup$

There are problems with using PCA for this type of analysis. First, even if your gene-expression data come from the same general platform (e.g., a particular RNAseq method), issues like correcting for batch effects within and between cohorts need to be handled carefully, as noted by @ReneBt in a comment.

Second, if the gene-expression data come from different technologies it might be difficult to translate PCA models reliably from one technology to another. For example, microarray and RNAseq gene-expression data have different characteristics in terms of dynamic range and mapping raw or partially processed data to individual genes. Even within each of those broad technology classes there can be differences in implementation that pose difficulties in cross-cohort comparisons.

Those fundamental problems must be addressed before trying to use a model from one cohort on data from a different cohort as you propose. This isn't impossible, but it does take careful attention. This recent paper explains one approach, with some references to other approaches in the literature.

If you can address those issues, then what needs to be done to proceed with PCA is straightforward. The original PCA needs to be done on centered and scaled data if you want to put all genes onto an equal basis. (It's typically best to omit genes with little variance between treatments/groups first.) The centering and scaling, however, might differ in the weights placed on the same gene between your reference set and the validation set. I suspect that some combination of that difference in centering/scaling and the technical issues noted above have led to the differences you found in results.

As presumably it's the gene-expression values that fundamentally matter, it makes sense to express the coefficients from your reference-set model back into the scale of gene-expression values and apply those back-transformed coefficients on your validation set. That is, don't work with centered/scaled data for the validation set but work directly in the scale of gene-expression values.

That leads to another problem with this type of PCA for gene expression. Although PCA diminishes the dimensionality of the model in an important sense, the model you get could in principle still contain coefficients for all 20,000 or so genes. Is that what you really want? How useful will such a model be?

I suspect that for this type of comparison, particularly if your interest is in prediction and you want to extend your results to a prognostic or diagnostic test, you would be better off using a method like LASSO that selects a restricted subset of genes. Although LASSO makes somewhat arbitrary selections of single genes from among a group of genes with correlated expression, that often works well in practice. Gene-expression studies provide many of the examples of LASSO in textbooks. If you're not familiar with LASSO, it's outlined nicely in An Introduction to Statistical Learning and covered in great detail in Statistical Learning with Sparsity.

Finally, think carefully about whether this separation into reference and validation sets makes the most efficient use of the data. Although separate groups seems to have an intuitive appeal, it often is suboptimal. A combined model including all available data with rigorous internal validation and evaluation of cohort-specific effects typically provides more power unless there are tens of thousands of cases. Quoting from Frank Harrell on that page, in a paragraph that seems applicable to your study:

Many investigators have been told that they must do an “external” validation, and they split the data by time or geographical location. They are sometimes surprised that the model developed in one country or time does not validate in another. They should not be; this is an indirect way of saying there are time or country effects. Far better would be to learn about and estimate time and location effects by including them in a unified model. Then rigorous internal validation using the bootstrap, accounting for time and location all along the way. The end result is a model that is useful for prediction at times and locations that were at least somewhat represented in the original dataset, but without assuming that time and location effects are nil.

$\endgroup$
2
  • $\begingroup$ Thanks for your beautiful answer. Seems like there are tons of stuff to study. I feel like LASSO might be a great option for my task (yes I'm working on developing a predictive index using gene expression data). I just have one more question. "The centering and scaling, however, might differ in the weights placed on the same gene between your reference set and the validation set." Does that mean using the scaling and centering coefficients derived from the reference set would work differently(different in their effects) on the reference and validation dataset (even if values are the same) ? $\endgroup$
    – Jay
    Sep 26, 2021 at 13:01
  • $\begingroup$ @Jay centering and scaling depend on the particular values of mean and SD in your data sample from the underlying population. SDs among samples in particular are quite variable. Thus centering/scaling and back-transformation are necessarily sample-dependent to some extent. That's why I recommend moving back to original-scale gene-expression data as soon as possible. For your application, you might consider a blend between ridge regression and LASSO, the "elastic net." That will typically return more predictors with non-zero coefficients than LASSO; see Statistical Learning with Sparsity. $\endgroup$
    – EdM
    Sep 26, 2021 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.