0
$\begingroup$

Forgive me; I am new to learning statistics. I am trying to understand OLS simple linear regression better and gain an intuition for the associated assumptions. Upon my research, I have identified four neccesary assumptions about the $\epsilon$ term in $Y = \beta_0 + \beta_1$X + $\epsilon$:

$\epsilon$ is assumed to be a random variable that....

  1. Has a mean of 0
  2. Normally distributed
  3. Constant variance at every value of X ($\sigma^2$)
  4. Independent between observations

My problem has to do with the first two assumptions. Most textbooks and online resources do not clarify the first two assumptions. That is, is the assumption that $\epsilon$ has a mean of 0 at every value of X or across values of X? Likewise, is the assumption that $\epsilon$ is normally distributed at every value of X or accross values of X? Any help would be much appreciated.

Based on my own understanding, I believe that both of these assumptions should apply to every value of X and not across values of X, seeing as we want to use these assumptions to create a distribution where $Y = \beta_0 + \beta_1X + \epsilon$ is normally distributed with a mean of $\beta_0 + \beta_1X$ and variance of $\sigma^2$ at each X value. If the first two assumptions apply to values across X values, we cannot be certain that every value of X has a mean of 0 and is normally distributed.

Improve this question
$\endgroup$
5
  • $\begingroup$ Welcome to CV. Have you considered searching our site for answers? $\endgroup$
    – whuber ♦
    Commented Jul 10 at 19:01
  • $\begingroup$ The typical assumptions you mentioned are often written as $\varepsilon_i\overset{iid}{\sim} N(0, \sigma^2)$, where the $iid$ means "independently and identically distributed" (means exactly what it sounds like, that each $\varepsilon_i$ has the same distribution, all of which are independent of each other). Since this seems to be a self-study question, I will hold off on being explicit about your specific questions, though I have answered implicitly, in hopes you will use this to craft a self-answer to post that I suspect will be correct. $\endgroup$
    – Dave
    Commented Jul 10 at 20:47
  • $\begingroup$ @Dave I will craft a self-answer, but I still have a reservation. Namely, your response implies what I thought to be true––that each of these assumptions applies to every value of X individually. To informally verify these assumptions, online resources suggest looking at two different displays: a residual display (either x_hat or y_hat on the x-axis vs. residual on the y-axis) and a histogram of the residuals. The former makes sense (we can verify for each x value that residuals appear identically distributed), but the histogram would make it impossible to verify identical distributions. $\endgroup$
    – LateGameLank
    Commented Jul 10 at 23:24
  • $\begingroup$ @Dave In other words, why are these sources suggesting looking at a histogram of residuals, which will make it impossible to see if error is identically distributed across x values? $\endgroup$
    – LateGameLank
    Commented Jul 10 at 23:27
  • $\begingroup$ @LateGameLank There are many plots worth considering in regssion diagnostics. You're seeing that one can be valuable for a particular task yet be limited in its ability to comment on a different task. If you, for instance, plot an lm object in R, you will get four graphs, e.g., set.seed(2024); N <- 99; x <- rnorm(N); y <- x + rnorm(N); L <- lm(y ~ x); plot(L). (None of those plots will be histograms, which speak to the fact that looking at the histogram might not be as obvious of a regression diagnostic as it first seems or often is presented.) $\endgroup$
    – Dave
    Commented Jul 11 at 1:27

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.