I'm trying to plot the predictions of a linear model with multiple numeric predictors as a collection of plots such that $\hat Y$ is on the y-axis, each $X_j$ in turn is on the x-axis, and the remaining $X_{i\ne j}$ are all set to plausible values given $X_j$. Here is some example R code (syntax might look strange, it's that way for conciseness, but it does parse correctly in R):



Note that in the above dataset, x2 and x3 are both correlated with X1, and all three are predictors for y.

The widespread approach of setting each $X_{i\ne j}=0$ or to the population mean or median is misleading if there is the possibility of multicollinearity among the $X$s because no distinction is made between commonly-occurring combinations of variables and ones that are completely out of the support range of the data. Example using above objects:


## Do you trust this? I don't.

## Ditto.

## Ditto, even worse, and yet the most common value at which to fix x_js.

I thought of fitting each of the $X_j$ to a given $X_i$ and using the predicted values for a given sequence of $X_i$....


...and that doesn't look as obviously nonsensical as the plots for globally fixed values of $X_2$ and $X_3$ but given that there are additional interactions involving $X_2$ and $X_3$ that are being ignored, these predictor values are not representative of the population either.

But. For a given sequence of values for $X_1$ it should be possible to construct a smoothed path through the variable space corresponding to the most likely values of $X_2$ and $X_3$. I can think of brute-force optimization approaches, but given how obvious and universal this problem is for any real-life regression model with multiple predictors, there must be an already-implemented and/or simpler approach to dealing with it.

So, my revised question is, does anybody know the name of such a method or the name of the subfield of statistics that deals with it, or the name of an R package that implements it?


  • 1
    $\begingroup$ If all you have is the correlation matrix, then you don't have enough information. As an extreme example, take $p=2$ and compare the situation $X_1=X_2$ to $X_1=X_2+10^6$, with any nondegenerate distribution for $X_2$. They share the same correlation matrix but one would predict extremely different values for $X_1$ in the two cases. $\endgroup$
    – whuber
    Jun 4, 2013 at 16:50
  • $\begingroup$ But, doesn't the presence of additional $X_i$ constrain the plausible values? Falling back on the physical problem, if you have a blob of points in N space, and you hold the value for one of the dimensions fixed (or limited to a slice of possible values), you now have a blob of points in (N-1) space and this blob has a centroid. Is there anything I need besides a correlation matrix to find the centroid? $\endgroup$
    – f1r3br4nd
    Jun 4, 2013 at 17:02
  • 1
    $\begingroup$ Yes: you need the location and size of that blob. The correlation matrix only tells you its shape. $\endgroup$
    – whuber
    Jun 4, 2013 at 18:01
  • 1
    $\begingroup$ I can't follow your question, because it's not evident what role the $X_i$ play in your "linear model" or what you're really trying to do. If you have the raw data, isn't this just a tangential exercise of little relevance? $\endgroup$
    – whuber
    Jun 4, 2013 at 19:18
  • 1
    $\begingroup$ Question now revised to make the problem clearer, with example code. $\endgroup$
    – f1r3br4nd
    Jun 5, 2013 at 11:25


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.