0
$\begingroup$

[Edit:] My question concerns the use of a categorical predictor in linear regression specifically.

Some of the assumptions for linear regression include: normality, homoscedasticity, and autocorrelation (of residuals), among others.

Normally, these all relate to the residuals of the model, but in a previous answer, it was suggested that when using a categorical predictor, assumptions should be checked on the raw data instead, but also that all the same assumptions endure. That being said, a regression with a categorical predictor is very similar to an ANOVA, and to my knowledge, ANOVA does not have the assumption of autocorrelation of residuals.

Which brings the question: Does the assumption of autocorrelation apply to linear regression with a categorical predictor? If yes, then how do you test it in R on the raw data (and not from the model)?

Edit

I began my question by saying the "main" assumptions but perhaps I should have said, "the main assumptions of interest to check at the analysis stage (once data is already collected and respecting independence of observations) with a categorical variable (group) as predictor." I had read online (e.g., http://r-statistics.co/Assumptions-of-Linear-Regression.html) that there are 10 assumptions. I will name them below for the sake of completeness (though I think the list is missing independence and linearity proper):

  1. The regression model is linear in parameters [re: related to Linearity]
  2. The mean of residuals is zero
  3. Homoscedasticity of residuals or equal variance [re: Equal Variance]
  4. No autocorrelation of residuals
  5. The X variables and residuals are uncorrelated
  6. The number of observations must be greater than the number of Xs
  7. The variability in X values is positive
  8. The regression model is correctly specified
  9. No perfect multicollinearity
  10. Normality of residuals [re: Normality]

Some of these assumptions either have been already verified in my case or do not apply. For example, multicollinearity does not apply to my situation as I have a single predictor, and by definition when comparing two levels of a categorical predictor the relationship will be linear (example 1, example 2).

That being said, as the title of my question points out, my question relates exclusively to the autocorrelation of residuals assumption for the case of a categorical predictor. The source linked before mentions "This is applicable especially for time series data", but it doesn't say that it doesn't apply to non-time-series data, so that's why I wasn't sure (also: in relation to a categorical predictor).

$\endgroup$

1 Answer 1

4
$\begingroup$

This statement is plain wrong: "I believe the three main assumptions for linear regression are: normality, homoscedasticity, and autocorrelation.". When it comes to statistics, it's dangerous to believe things without doing your due dilligence and checking them out.

If you check any statistical book on linear regression modelling, you will see that there are 4 major assumptions underlying this type of modelling, which can be conveniently summarized via the word LINE:

L: Linearity
I: Independence
N: Normality
E: Equal Variance 

The Equal Variance assumption is what you referred to as homoscedasticity.

In practice, the most important assumption is Linearity, followed by Independence, followed by Equal Variance, followed by Normality.

The independence assumption means that the response values Y are assumed to be independent from each other. In other words, knowing one value of Y tells you nothing about any of the other values of Y.

Usually, linear regression is applied to data collected from a cross-sectional sample. As an example, this sample can be obtained by randomly sampling a population of subjects and recording the value of the response variable Y and the predictor variables X1, ..., Xp for each of these subjects at the same time point.

However, it is possible to use linear regression to a response value Y collected over time. An example of such response variable will be annual sea surface temperature at a particular ocean location. The values of this variable will be collected year after year for that same site and will likely be autocorrelated (e.g., a high value of sea surface temperature this year was likely preceded by a high value the previous year). If you formulate a linear regression model which will relate annual sea surface temperature (Y) to year (X), then you would have to check its residuals for the presence of autocorrelation. If autocorrelation is present, then you might want to consider a generalized least squares fit for your model rather than an ordinary least squares fit.

Thus, worrying about autocorrelation of the model residuals only makes sense if your response values were collected over time. If your response values were collected at regular time intervals (e.g., every year), then you could use graphical methods such as ACF or PACF plots to detect the presence of autocorrelation among the model residuals. If the data were collected at irregular time intervals, the graphical methods you use would have to reflect that irregularity.

Perhaps after reading this answer you can revise your question and make it more specific.

$\endgroup$
1
  • 2
    $\begingroup$ Thank you for your answer Isabella! I think the key statement was: "worrying about autocorrelation of the model residuals only makes sense if your response values were collected over time." Since I have an experimental design (but collected at a single point in time), then I interpret this as: I do not have to check the autocorrelation assumption in this case. I also edited my question at your request for clarity. $\endgroup$
    – rempsyc
    Dec 29, 2020 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.