# Using UMAP or other non-linear dimension reduction techniques on response variables prior to learning?

Background

Suppose you have a training set where the response measurements are some $$N$$-dimensional vectors of related measurements - in my specific case, they happen to be cell viability scores for cell lines treated at about 10 different concentrations.

I'm interested in learning which features in the training set are most predictive of cell line response to treatment. One approach that I've been testing is the use of random forest variable importance scores, measured across different subsets of the features.

Prior to generating the random forest models, however, I first need to transform the $$N$$-dimensional response vectors for each cell line into $$1$$-dimensional values.

I.e. I need to go from an $$m$$ x $$n$$ cell line by concentration matrix to an $$m$$ x $$1$$ vector of response measurements for each cell line.

Figure 1: Cancer cell viability measured at increasing concentrations of a drug for about 45 different cancer cell lines.

There are a number of different ways to achieve this, with the most common approach being to fit a log-logistic dose-response curve and use either AUC or some parameter related to the fit curve (the most commonly used one is "AC-50" or "IC-50" - the concentration at which half-maximal activation or inhibition is achieved).

I've also been exploring other more basic approaches, however, that don't depend on the quality of the curve fit, such as using the first principle component (PC1), with the goal being to preserve as much information as possible from the original higher-dimensional representation of the response data.

While this seems to hold some promise, there are some obvious limitations of using PCA, and visualization of $$2$$-d projected PCA, t-SNE, and UMAP representations of the data indeed seems to suggest that the non-linear techniques do a better job of capturing important relationships between the data points than PCA does.

Figure 2: t-SNE embedding of drug response AC-50 scores across all drugs. Each point represents a 2d projection a dose-response matrix such as that shown in figure 1, colored by the drug class. The AC-50 scores represent one way to project the full dose-response matrices to a single dimension. Note that here t-SNE is used purely to visualize the resulting response data, and is not actually used for any later steps itself.

Figure 3: t-SNE embedding of drug response first principle components across all drugs. Plot is constructed similarly to that shown in figure 2, except that instead of performing log-logistic curve fits for each cell-line/drug pair to determine AC-50 scores, the first principle component (PC1) is computed separately for each drug and used to compress each drug/cell-line matrix into a 1d vector of PC1 scores. Again, t-SNE is used purely for visualization purposes here and has nothing to do with the underlying goal of the question. Note that the PC1-projected data appears to do a better job preserving inter-drug relationships within the response data. The overall question of this post is, can we do even better by adopting a non-linear embedding such as UMAP?

Question

Do you think it is possible to use Uniform Manifold Approximation and Projection (UMAP) 1 to project the $$N$$-dimensional response variables to $$1$$-d measurements, prior to learning random forest, etc. models on the data?

This certainly would not work with something like t-SNE, which does not preserve global relationships between the data points. UMAP, however, is thought to do a better job at preserving such larger-level structure in the data, and as such, seems like it could be a good way to preserve the nature of the relationships between data-points after projection to a lower-dimensional space.

If so, any thoughts on how to parameterize the UMAP fit with such a goal in mind? At first glance, increasing the size of the neighborhood considered for each data point and the minimum distance allowed between data points should both help with preserving global structure in the data. Are there any other parameters that should be considered?

Any suggestions or feedback would be greatly appreciated!

References

1. McInnes,L., Healy,J. and Melville,J. (2018) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv [stat.ML].
• Fun question. Just to clarify something before answering: what is the input data here and what is the output? You say that "I (..) need to transform the N-dimensional response vectors for each cell line into 1-dimensional values." do you mean turning your input data into a scalar? ie. that the whole trajectory of the cell-line is represented by a single number that is used as the predictor? (Because I see no real reason to confine this to a single predictor!) – usεr11852 Dec 12 '18 at 21:14
• @usεr11852 Not predictor -- response. – amoeba Dec 13 '18 at 16:57
• @Keith Is your image your actual data? Or is it only for illustration purposes? – amoeba Dec 13 '18 at 16:58
• @amoeba: Thank you for the clarification. This totally changes the nature of the game. So, Keith, you want to predict the UMAP embedding? Assuming we do predict it, we will be having substantial difficulties in interpreter the model. Why not use a FPCA or a Wavelet basis that is orthogonal, immediately interpretable and we can use it in a series of separate multivariate regressions? – usεr11852 Dec 13 '18 at 19:24
• @usεr11852 That's correct - only the response data is being embedded. As far as the interpret-ability, it actually isn't so much of an issue in this case in that I'm not actually interested in the models themselves, but rather in the importance scores associated with features that are generated along the way. The purpose of using UMAP or something similar is just to try and preserve as much of the information in the original response data prior to training. Thanks for the suggestions regarding FPCA / Wavelets - that's not something I considered, but is probably worth looking into. – Keith Hughitt Dec 15 '18 at 21:12