In STATS classes, one is always taught to draw a picture to look for outliers, to look for the distribution type, to look for patterns in general. However, when you have a dataset with a lot of predictors, this becomes very hard. My questions are as follows:

  • How does one go about visualizing a dataset with a lot of predictors?
  • How does one go about figuring out which methods are appropriate to use for prediction (especially if visualization is not feasible)?
  • 2
    $\begingroup$ How much is "a lot"? $\endgroup$ – whuber Feb 27 at 20:05
  • 1
    $\begingroup$ Maybe a simple way to visualize is pairwise plots for each predictor with your response. Though you cannot capture interactions with other variables. $\endgroup$ – Tom Feb 27 at 22:36
  • $\begingroup$ @whuber I mean like hundreds or thousands (such as genetic data) $\endgroup$ – Merry Feb 28 at 19:56

Having many predictors is not per sé a bad thing (although in some cases one must pay attention to the bias-variance trade off, especially when using parametric models containing many variables to fit the data); this said:


In general it is beneficial to perform descriptive statistics before starting any analysis: in particular one can:

  • calculate pairwise correlations among numerical variables; this gives an indication of whether or not some pairs of variables have a linear dependence between each other, which in turn can help sub-selecting the ones you want to use as predictors for the eventual model (to make a long story short if two variables are correlated you can replace the one with the other)

  • boxplot the response variable against categorical variables in your predictors, if any; this helps understanding how the responses are distributed within your classes and can help identify outliers

In python there a very straightforward way to visualise the above, namely using the seaborn library (specifically pairplot if you want to do it all at once). However, as mentioned, the limitation of the pairplots is that you will have no information about possible interactions among sets of variables themselves.


There is really no general rule to guess what model may fit the data the best. If you have good reasons to believe that the decision boundaries are linear then one may try more or less complicated polynomial fits (by including possible interactions or higher order terms), eventually performing backward/forward variables selections (that is, adding/removing one variable at a time, usually the least significant one, to see whether the goodness of the fit changes - notice that in this case the coefficients of your variables will of course change).

If not, one must move towards non-linear models like support vector machines, nearest neighbours, neural networks or ensemble methods, like the whole category of random forests and the like.


Since you are looking for points that are considered "outliers" with respect to predictors, I recommend that you create a set of added-variable plots showing the relationship between each predictor and the response, having filtered out the effect of other predictors. The advantage of this kind of plot (over a simple scatterplot of the raw values of those variables) is that it takes account of the multivariate relationship by filtering out the effects of the other predictors in the model. You can create one of these plots for each predictor, and it will show the partial relationship between that pair of variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.