6
$\begingroup$

enter image description here

I do not quite understand when a graph shows homoscedasticity. Can someone please explain this to me with the help of the plot I provided?

$\endgroup$
  • 4
    $\begingroup$ you need to tag the question with self-study tag since it's a home work, and show your work. this site is not to get ready solutions but to help you find them. read the rules on the tag here: stats.stackexchange.com/tags/self-study/info $\endgroup$ – Aksakal Mar 28 '18 at 15:01
  • $\begingroup$ @Aksakal What evidence do you see that this might deserve the self-study tag? (The plot is taken from our site itself, BTW.) $\endgroup$ – whuber Mar 28 '18 at 20:43
  • $\begingroup$ @whuber, in the comments to my answer OP posted a set of questions that looked like a home work assignment $\endgroup$ – Aksakal Mar 28 '18 at 20:45
  • $\begingroup$ @Aksakal Thank you--I see those deleted comments now and they prove your point. $\endgroup$ – whuber Mar 28 '18 at 20:46
  • $\begingroup$ @whuber could you post a link to the source of the plot? $\endgroup$ – Jim Mar 28 '18 at 22:02
7
$\begingroup$

I don't think a graph can necessarily "show" homoscedasticity, but it can indicate to deviations from it. Your plot shows a very obvious trend in residuals vs. predicted. Anytime you see a some sort of a structure in these plots it's a source of concern. Ideally you should see a shapeless cloud of dots without any kind indication of a trend up or down. Yours is clearly downward sloping. It's not good.

$\endgroup$
  • 5
    $\begingroup$ I don't think it's obvious that there is a downward slope. I think you may have been fooled by the fact that it seems to be a collection of downward sloping lines. But the total slope isn't necessarily negative just because every part has negative slope (see Simpson's paradox). And that is a matter of bias, not homoscedasticity, anyway. $\endgroup$ – Acccumulation Mar 28 '18 at 17:37
  • 1
    $\begingroup$ @Acccumulation, this is not a simpson paradox case. something's wrong with them for sure. i though about bias but then on the left side the residuals are clearly more dense on negative values, and could be that the mean is zero, while on the right side they're more symmetrical around zero. so they may work out to be zero in mean across the data set, however, the range bounds are clearly going down $\endgroup$ – Aksakal Mar 28 '18 at 18:31
  • 2
    $\begingroup$ You're right about the range bounds, but when I look at this, I see an upward slope. On the left, more points are towards the bottom. On the right, more points are towards the top. I would like to see a smoother to settle this. $\endgroup$ – eric_kernfeld Mar 28 '18 at 18:52
  • 3
    $\begingroup$ -1 until you explain what exactly the "source of concern" is in this plot. Note that there is not an upward or a downward slope/trend. The residuals and the predicted values are orthogonal by construction. $\endgroup$ – Jake Westfall Mar 28 '18 at 20:40
  • 1
    $\begingroup$ I don't think this answer is useful. The question was about homoscedasticity, not structure. Visible structure in the residuals is much more general, and you could have homoscedastic residuals even in the presence of apparent "structure." $\endgroup$ – Josh Mar 29 '18 at 22:09
5
$\begingroup$

enter image description hereI must confess that I've never seen a plot where the fitted values are standardized - usually, we standardize the residuals but not the fitted values.

The first thing you should do is draw an imaginary horizontal line through zero in this plot. This line will anchor the plot and really help you understand what is going on.

If the observations in the plot are randomly scattered about the horizontal zero line such that the level of the scatter is roughly the same about this line as you move from the left to the right along the line, that would indicate that the linearity and homoscedasticity assumptions are not violated by the data. If, on top of this, most of the standardized residuals fall within +/- 3, then that might support the normality assumption (although, for checking the normality assumption, you are better off to look directly at the distribution of residuals via a histogram or density plot and also at a normal probability plot of residuals).

In your plot, it looks like all 3 assumptions are violated by the data.

The linearity assumption is violated because the cloud of points shows a systematic downard trend, as pointed out by @Aksakal.

The homoscedasticity assumption is violated because the spread of the residuals is not (roughly) the same as you move along the horizontal line going through zero.

The normality assumption is violated because the residuals do not form a cloud of points randomly and roughly evenly scattered between -3 and 3.

$\endgroup$
  • 6
    $\begingroup$ The unusual features of these residuals may be causing some confusion. The conditional means of the residuals appear to be constantly zero. One standard meaning of heteroscedasticity refers to whether the conditional variances change appreciably with the fitted values. It is not so clear that they do vary in this plot. It is even less clear what semivariances ("spread of the positive and negative residuals") or normality might have to do with this question. Perhaps you understand "heteroscedastic" more generally to mean that the conditional distributions vary with the fitted values? $\endgroup$ – whuber Mar 28 '18 at 18:25
  • $\begingroup$ @whuber: I'm one of your long time admirers (from afar), so it's great to see you engaging with my (possibly flawed) answer, which I edited to remove the reference to positive/negative residuals. I can't see that the conditional means of the residuals are constantly zero from the picture I attached. That would be the case, say, if the distribution of residuals would be nearly symmetric about zero. However, with such skewed conditional distributions towards the left hand side of the plot, how can you tell whether the means of those distributions are zero? $\endgroup$ – Isabella Ghement Mar 28 '18 at 23:34
  • 3
    $\begingroup$ That's a great question, Isabella. Without some method of jittering or transparent overlay it's impossible to tell on this image, but with some generosity--along with the assumption these might be residuals from a least squares fit with a constant term--it's possible to imagine that at each x value, the masses of dots just below zero counterbalance those above zero. I would therefore give this assumption the benefit of the doubt and acknowledge that at least the graphic is not inconsistent with trendless residuals. It takes more imagination to see homoscedasticity here, but it's plausible. $\endgroup$ – whuber Mar 28 '18 at 23:48
2
$\begingroup$

The graph does show that there are issues with this model, but the question is whether or not it shows deviations from the hypothesis of homoscedasticity, and that is not quite clear from the graph. I think it does show some deviation: sigma seems smaller at low values of the predicted variable, but it is hard to tell.

The graph does show that the mean of the residuals is going down from right to left (although, as pointed out by whuber, there is some level of optical illusion in this, because of the larger amount of points in the bottom left). So, the distribution of the residuals (specifically, the mean of the distribution) is probably not the same at high and low values of the predictions (can fit a model of the form a*x+b to the residuals and show it in the same plot) . I would not call this a violation of linearity (residuals in non-linear regression models are also expected to have the same mean throughout the application domain), but it does seem to violate the assumption that all residuals are coming from the same distribution.

The simplest way to test whether the mean of the residuals has a trend is to look at at ANOVA table of the regression model (this is a bit different from, although related to, doing an ANOVA analysis of the residuals). In Mathematica, the property "ANOVATable" is available for regression models obtained with LinearModelFit and NonlinearModelFit. In fairness, using the standard ANOVA is not quite justified, because the distribution of the residuals does not look normal. There are non-parametric ANOVA methods that could be used (Kruskal-Wallis). Honestly, I think a test is hardly necessary, the trend seems clear to me from the graph. The plot I suggest above will give you a value for the slope. A test like ANOVA (or similar) would give you a p-value to judge whether that slope is significantly different from zero (one cannot expect it to be exactly zero).

Back to homoscedasticity, you would need to check that the standard deviation (sigma, for short), not the mean, is the same at different values of the predicted variable. One simple way to do this is to bin the data in a few bins and compute sigma in each bin. Of course, there is some freedom as to how to bin the data (so this simple approach relies on judgement). Instead of computing the sigmas for each bin, you can just do a test on the groups of residuals in each bin. The Bartlett test, for example, takes several samples as its input and decides whether the samples came from distributions with the same sigma (not whether they came from the same distribution). In your case, it looks like there are serious deviations from normality, so the Bartlett test may not be the best, Levene is more robust. In Mathematica, the function VarianceEquivalentTest takes in several samples (which would just be your residuals in different bins) and returns the results of several tests (Bartlett, Levene, Conover, etc.), on whether the samples have the same sigma. I think it will refrain from reporting on a test that does not seem applicable to the data.

From your graph, I would be inclined to say that at low values of the predicted variable your sigma looks smaller (same level of spread with more data points usually means lower sigma). But it is not as clear to me as is the trend in the mean, for example (again, even for the trend in the means, it has been suggested that the graph could be deceiving because of the variations in population density). A clear violation of homoscedasticity is a graph with very narrow spread in some parts and very wide in others, and yours does not display that very clearly. But is does not clearly show the opposite either.

Another way to test for homoscedasticity directly on the residuals (without binning the data) is to run the Breusch-Pagan test or the White test on the residuals (you will need the residuals AND the predicted values, because these tests check whether sigma seems to be a function of the predicted values).

The residuals do show a clear deviation from normality, it even seems clear that they are bimodal. For example, at large values of the predicted variable, the residuals are clearly concentrated around two values, one high and one low. You could run a normality test on the residuals (seems hardly necessary), like Anderson-Darling, Smirnov, etc., but since the residuals seem to be coming from different distributions, these tests are not very meaningful. A test of normality makes sense in a sample that was all taken from the same distribution. In this sense, a test of normality should usually come last, after you have established that all the residuals come from the same distribution. Some people just do a test of normality and nothing else, assuming that if the residuals came from different distributions, most likely a test of normality would fail. There is some truth to that, but it is very shaky statistics. It is like saying that if the body temperature is normal, then the patient is healthy.

In general, you want to see evidence that all your residuals are coming from the same distribution. The first thing to check is the mean and sigma. In your case, the mean does not not seem constant to me, but you should check, and for sigma it is just harder to say from your graph (so the short answer to your original question, which this certainly isn't, is that it is hard to tell from your graph). Of course, a distribution is more than just its mean and its sigma, but if those two look good (meaning, if they constant throughout the application domain), then there is reason to celebrate. If the residuals all come from the same distribution, then one would also want for that distribution to be centered around zero, to be unimodal, preferably symmetric, and, ideally, normal. But normality is like the last nice-to-have, not a requirement for a good model.

Lastly, remember that the scatter plot of the residuals for a good model can look a bit messy. To a large degree this depends on how uniformly populated the different regions of the model domain are. If one has significant amount of data in one region and very little in others, then there are issues related to leverage of different data points. This is in part why one looks at standardized residuals and studentized residuals. In my mind, a plot of residuals that looks too ideal (a perfectly horizontal, evenly populated rectangle), suggests a fudged model, rather than a good model.

$\endgroup$
  • 2
    $\begingroup$ I stopped reading at "The graph does show that the mean of the residuals is going down from right to left" because I think you are deceived in your impression (as explained in comments to the question and other answers). If the conditional mean isn't correctly estimated it seems unlikely subsequent analyses of the conditional variances will be correct, either. $\endgroup$ – whuber Mar 29 '18 at 14:15
  • $\begingroup$ whuber, thanks for your observation. Agree: possible optical illusion in estimating the means (large population at low values on the left). Could plot a fit (a*x+b) of the residuals as a function of the predicted variable. If the vector of residuals is perpendicular to the vector of the predicted variable, then 'a' will be zero. Still, to get a p-value a test is needed. I appreciate the observation, but I do not think it invalidates the rest of my answer. My answer focuses on the methods that can be used to tests the related hypotheses. I really did not do any analyses of any variances. $\endgroup$ – Otto Linsuain Mar 29 '18 at 14:53
  • 1
    $\begingroup$ I edited my answer to incorporate whuber's comments. As it happens, I recall having a pretty much horizontal rectangle made out of subgroups of downward sloping lines. I mean in a real model development application, not in an abstract drawing. Going through the exercise of producing fits of the residuals, running the tests, etc., helps avoid such pitfalls. I recommend this site: itl.nist.gov/div898/handbook/index.htm $\endgroup$ – Otto Linsuain Mar 29 '18 at 16:04
  • 2
    $\begingroup$ @whuber: You make a great point! In practice, the four major assumptions underlying a linear regression model should be checked in this order of importance: linearity (most important), independence, homoscedasticity and normality (least important). If the linearity assumption is violated, the model should be refined and then the assumptions should be checked again for the refined model. This plot is sure challenging to interpret! $\endgroup$ – Isabella Ghement Mar 29 '18 at 19:33
  • $\begingroup$ Isabella, could you clarify the linearity assumption? Just wondering if it is the same as (or a subset of) what I remember as the i.i.d. (identically, independently distributed) assumption of the errors. As far as I remember the i.i.d. assumption is common to linear and non-linear regression models (which is why I avoid calling it linearity assumption). But I could be missing something...A subtle point in these analyses is that the i.i.d. assumption is made about the true errors (observed - true), but it is tested by analyzing the model errors (observed - predicted). $\endgroup$ – Otto Linsuain Mar 30 '18 at 14:50
1
$\begingroup$

Note: I use capital letters to refer to an entire vector, and lower case letters to refer to a specific observation of a vector. Hopefully this isn't confusing.

I think the answer ought to be no. I offer an intuitive explanation, as well as some quick and dirty (emphasis on dirty) R code that supports my assertion.

The variance of the $Y$ values is increasing with $x$, because even though the range of $Y$ appears to be somewhat constant over the domain of $X$, the average distance of the points from their mean (as a function of $x$) is actually increasing. You can see this because the density of $Y$ in regions between its extents is actually decreasing with increasing $x$, and therefore the average squared deviation (variance) from the mean is increasing. In other words, pretty much all the $y$ values' separation from the mean (conditional on $x$) are "really large" for large $x$, while at smaller $x$, they're anywhere from zero to "really large."

Anyway, I wouldn't be surprised if folks find that explanation really confusing. If so, just run the following in R, and observe the resulting plot.

set.seed(999)
n.lines <- 10
n.points.per.line <- 100
min.intercept <- -3
max.intercept <- 3
x <- vector('numeric')
y <- x
intercept <- seq(from=min.intercept, to=max.intercept, length.out=n.lines)

for (i in 1:n.lines) {
  min <- -1.5
  max <- intercept[i]^2
  x.new <- runif(n=n.points.per.line, min=min, max=max)
  x <- c(x, x.new)
  y <- c(y, -0.1*x.new+intercept[i])
}

ylim <- c(min(y)*1.1, var(y)*3)
xlim <- c(min(x)*1.1, max(x)*1.1)
xlab <- 'x'
ylab <- 'y'
plot(x, y, type='p', ylim=ylim, xlim=xlim, ylab=ylab, xlab=xlab)

buckets <- ceiling(seq(from=min(x), to=max(x)))
var.y <- sapply(buckets, function(i) {
  y <- y[which(x <= i & x >= (i-1))]
  var(y)
})
par(new=T)
plot(x=buckets, y=var.y, col='blue', type='l', ylim=ylim, xlim=xlim, ylab=ylab, xlab=xlab)

enter image description here

This is far from exact, but the data look a lot like yours. I bucketed $Y$ into about ten different intervals (based on integer values of $X$) and computed their variance; the blue line shows this in the plot. As you can see, it is increasing with $x$, and therefore the data are not homoscedastic.

$\endgroup$
  • 1
    $\begingroup$ The data in your plot are similar in some ways to the plot from the OP, particularly the presence of slanted lines of residuals, but the heteroskedasticity is visually obvious in your plot whereas it's not at all obvious in the OP's plot. $\endgroup$ – Jake Westfall Mar 30 '18 at 2:57
  • $\begingroup$ I think the intuitive explanation I provided holds true for both datasets, although without the actual data form the original plot it's probably not possible to say with any real certainty. That said, my data is definitely less "noisy," and lacks the high density region in the lower left corner. $\endgroup$ – Josh Mar 30 '18 at 4:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.