1
$\begingroup$

So to preface I'd like to say that this is for homework and I am not very good at statistics. Please explain things to me like I am 5 years old.Also english is not my first language.

So the homework has a column of the amount of money a company spent on marketing each month over a 50 month period, and their sales each month over the same 50 month period. Money spent on marketing is the independent variable and the units of the product sold is the dependent variable. After doing the regression with the data analysis tab on excel, it shows that the determination coefficient is 0.99. According to my notes from class the determination coefficient shows you if the model is good or not, and that the closer it is to the number one the better.

But my notes also say that linear regression is not the correct model if the residuals are not distributed normally and also that the average of these residuals must be 0. The residuals in this case are not normally distributed. According to the internet the kurtosis needs to be 3 or lower for it to be normal distribution, and in this case it is around 7. Also the average of the residuals is not 0. It is -2.8422E-15, so close to 0, but not really actually 0.

Excel shows that the standarized residuals are all within a range of -3 and 3 except month 49, which is -4.39. HELP

$\endgroup$
5
  • $\begingroup$ Have you tried plotting out the residuals vs the independent variable? A visual examination is a good place to start to see if all the assumptions of for regression are true. -2.8422E-15 is a rounding error and considered 0. If this is real data, one month out of 50 being unusual probably shouldn't be cause for alarm. $\endgroup$
    – Dave2e
    Commented Jun 20 at 1:10
  • 1
    $\begingroup$ $-2.8422\times10^{-15}$ is so close to zero that the difference between that and $0$ almost certainly results only from rounding. $\endgroup$ Commented Jun 20 at 2:26
  • $\begingroup$ A coefficient of determination that high makes me suspect it's not real data. $\endgroup$ Commented Jun 20 at 2:30
  • 1
    $\begingroup$ You may have read something that says linear regression is the right thing if the errors (not exactly the same thing as the residuals) are normally distributed. But that's not the same as saying linear regression fails to be valid in ALL cases where the errors are not normally distributed. $\endgroup$ Commented Jun 20 at 2:33
  • $\begingroup$ @MichaelHardy It's also not the same as saying that linear regression is the correct model when the errors are Gaussian! $\endgroup$
    – Dave
    Commented Jul 2 at 14:54

1 Answer 1

0
$\begingroup$

Since this is a homework problem, I will beat around the bush a bit and will not be giving a final determination about how to interpret your $R^2$ score. Hopefully, however, you can use what I write here to inform your homework submission.

Also the average of the residuals is not 0. It is -2.8422E-15, so close to 0, but not really actually 0.

There is no assumption about the residuals having a mean of zero. Residuals having a mean of zero in least squares linear regression is a mathematical fact. When you use a linear model (with an intercept) and compute the coefficients by finding the coefficients that minimize the sum of the squared residuals ("least squares"), it is a mathematical fact that the residuals will have a mean of zero.

As you point out, your residuals do not have a mean of exactly zero but of $-0.0000000000000028422$, very close to, but not exactly equal to, zero. Thus, your estimates are not the exact least squares estimates (I assume your Excel implementation includes an intercept, which I find to be a safe assumption).

However, the mean is so close to zero that this is pretty much irrelevant. The true least squares values probably have many more decimal digits then the computer is tracking. In most cases, I would say those do not matter, so your residual mean of -2.8422E-15 is (probably) safely assumed to be "basically zero".

Really, the residual mean being zero is more of a test of if the given solution really is a least squares solution than it is about any statistical assumptions, since the residuals having a mean of zero is a property of the least squares solution, whether a least squares approach is wise or not.

But my notes also say that linear regression is not the correct model if the residuals are not distributed normally

This is a dubious claim. For instance, while this might not be part of your course, there is this Gauss-Markov theorem that proves a nice property of the least squares estimator, and there is no reference to normality in the setup of the Gauss-Markov theorem. When you have normal errors (not quite the same as residuals, but you can use the residuals to gauge or estimate the errors) with a constant variance, what you get is that the least squares estimator coincides with maximum likelihood estimation, which implies some desirable properties and some typical calculations for coefficient confidence intervals and hypothesis tests that are not quite as accurate when the error normality is violated. However, that has nothing to do with the coefficient of determination.

Finally, it is even a mistake to conflate linear regression with anything involving estimation technique or error distribution. A linear model can be totally valid for a different estimation technique than square loss when you assume a different distribution for the errors, such as minimizing absolute loss when the errors are assumed to be Laplace-distributed (which turns out to be equivalent to maximum likelihood estimation). However, you still have the same $\mathbb E\left[y\right] = X\beta$ linear regression formula, just a different way to estimate $\beta$. Likewise, linear regression would be inappropriate for a situation like the following, where the curve is specified to follow $\mathbb E\left[y\right] = \exp\left(\beta_0 + \beta_1x\right)$.

data

However, the residuals from fitting the correct (nonlinear) regression are quite normal, as the normal quantile-quantile plot below shows.

residual qq plot

R Code:

set.seed(2024)
N <- 10000
x <- runif(N, -2, 2)
z <- 2 + x
u <- exp(z)
y <- u + rnorm(N, 0, 1)
plot(x, y)
L <- glm(y ~ x, family = gaussian(link = "log"), start = c(2, 1))
r <- resid(L)
qqnorm(r)
qqline(r)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.