1
$\begingroup$

I am doing a Multiple Linear Regression with 19 explanatory variables and around 500,000 data points. When I plot the Residuals vs Fitted plot, I see weird shapes where there appears to be multiple regions in it. One centered around 0, one vertical band below 0 and a funnel like band above 0.

Generally the funnel shape may indicate heteroskedasticity, but what would these multiple shapes in my plot indicate?

Weird Residuals vs Fitted plot

$\endgroup$
1
  • $\begingroup$ Can you say more about your data? What are the variables? How many data do you have? Can you post some plots of your raw data (eg, a scatterplot matrix, some boxplots, etc)? Are you trying to predict, or are you testing variables? $\endgroup$ Sep 3 '20 at 13:14
3
$\begingroup$

There are clearly latent clusters in your data. Do you have any other variables, especially categorical variables, that might account for the different bands? If so, there is an interaction between that (those) categorical variable(s) and something else.

In general, you should look at your data before you fit a model. You don't want to be surprised by this. What are your variables? What do they mean? Try looking at scatterplot matrices, etc.

$\endgroup$
2
  • $\begingroup$ Yes, I do have categorial variables (1, 0) in my data. Would this be a problem by itself? I will take a re-look at my data to see how it looks. Not relevant to this question but do you know of ways to detect latent variables. Say, using some clustering algorithms. $\endgroup$
    – Bharat
    Sep 2 '20 at 21:48
  • $\begingroup$ @Bharat, it isn't a problem to have categorical variables, but there might be some relevant ones that are missing from the model, or interactions with them that are missing. There are lots of clustering algorithms, you could try reading some of the threads categorized under our clustering tag. In R, there is the flexmix package, which is for situations like yours, but it's been a long time since I've dabbled with it. $\endgroup$ Sep 3 '20 at 13:17
1
$\begingroup$

I saw something like this in a data set once. The graph was of charitable contributions (Y) vs income (X). There was a distinct linear pattern with a slope of 0.10 that looked different from the rest of the data scatter. These were (I strongly believe) the tithers. The variable "tither", a 0/1 variable, is a type of latent variable mentioned by gung. It is latent because you don't know who the tithers are.

Bottom line, it looks like there are different mechanisms at work in your data, and you do not necessarily know which observations are responding to which mechanism (like tithers vs non-tithers). To tease out these latent structures, you could consider using switching regressions.One reference is here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.