I have a data set with 2000 continuous predictor variables and a binary outcome variable. I would like a few easy ways to visualize this data. A box plot or histogram of all the variables seems that it would be too much. Are there any good ways of simultaneously visualizing the data?
$\begingroup$
$\endgroup$
4
-
2$\begingroup$ What would be the objectives of the visualization? $\endgroup$– whuber ♦Commented Aug 27, 2016 at 22:10
-
$\begingroup$ @whuber: To look at the distributions of the variables and see if there are any outliers. $\endgroup$– machinelearningguyCommented Aug 27, 2016 at 23:58
-
$\begingroup$ If you put all the predictors onto the same "scale" (e.g. standardize or normalize), then you could do histograms of all variables using the same set of bins. Say you have 100 bins, then you could visualize the conditional PDFs of your predictors as two 100 by 2000 grayscale images (one image for each value of the outcome). These could be composited in different ways into a single RGB(A) image if you wish. $\endgroup$– GeoMatt22Commented Aug 28, 2016 at 3:41
-
$\begingroup$ @machinelearningguy Finding multivariate outliers by eye in a dataset this size is out of the question. (I have enough trouble visualizing 3-dimensional space; 2,000-dimensional space is best left to Lovecraft.) If you want multivariate outliers, try these methods. $\endgroup$– KodiologistCommented Aug 28, 2016 at 23:27
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
High dimensional data visualization depends on your overall objective. If for example you want to see if the data is actually separable, you could consider looking at a dimentionality tool like PCA, LDA or non-linear tools MDS, kernelPCA ,local linear Embedding, Isomap (these tools will depend on what structure of the data you would like to preserve).
But If your concern is to see the relationships between the variables and/or their distribution with outliers, the best option is to use a pair-plot (scatter plot matrix) with subsets of the data, I admit this is a bit laborious.
