How can we visualize multiple regression with 3 or more continuous variables or with categorical variables

Question

I'm trying to understand multiple regression visually. So far I understand that when you have two continuous variables explaining y, this will take the shape of a plane (see the image).

What happens if you have 3 continuous variables? Does this take the shape of a cube or rectangular prism? What about 4 continuous variables? I don't think we can visualize 4 dimensions? As a size note, I am aware that I can use Level plots. But I still wanted to build from the plane example.

Also, how can we imagine adding a categorical variable? How would that effect the plane. Using the example from the image bellow, we could add the "make of the car" to the model. "Make" could be a vector contaning ("Toyota", "Subaru", "Chevrolet")

So we have MPG~Weight+Horsepower+Make.

Would "Make" be a new axis? Or would it instead be a group of points. For example Toyota could be red points, Subaru = blue points, and Chevrolet being black points?

Thank you!

Answer 1

3 continuous variables: make a row of such plots at 3-5 defined levels (or 'slices') of the third variable.

4 continuous variables: make a column of such plots at 3-5 defined levels of the fourth variable.

I would also consider using colour for the third dimension, though it makes it much harder to show the points in addition to the fitted surface. This can be partially ameliorated by showing an additional plot of observed vs. fitted values.

Here's an example from a recently published paper that shows the output of a theoretical model, but the same approach can be used for a statistical model. (https://aslopubs.onlinelibrary.wiley.com/doi/abs/10.1002/lno.12040):

In principle this plot matrix approach can be used with the 3D plots too, but it will likely become too crowded.

Categorical variable: same as above, it's just 2 panels (for a binary variable, more otherwise) instead of 3-5 levels as in the case of the continuous variable. Or if you use the 3D plots instead of colour, you may be able to squeeze 2-3 categories into the same panel using different colours.

What about 5 continuous variables? Either make several sets of these plot matrices at 3-5 defined levels of the fifth variable, or animate each panel in the single plot matrix i.e. use time as the 5th dimension. These options will almost never work well, but there will be a few datasets for which it's possible.

Answer 2

You usually use so-called matrix plots, a collection of 2-dimensional scatter plots. Categories are usually represented by different colors of the data points.

Answer 3

I have used matrix plots before, but I also used another way. I made a regression plot between the measured variable and the main input variable. Then I took the residuals, and plotted those as a function of the second most important variable. And so on.

I feel this is a nice graphical representation of how much “value” you get from each step in making the model more complicated. I personally find 3D plots of any kind difficult to read and interpret, so my choice is very much dependent on my personal aesthetic preferences.

I obviously do present the regression results for the “all at once”: THAT is the analysis, everything else is pretty pictures for illustrative purposes.

Main Trouble with this approach is that a lot of people pretty much ignore entire sections as soon as the word “residual” comes up. It’s up to me to educate these people ;)