3
$\begingroup$

I was reading the book ``Genetics and Analysis of Quantitative Traits'', by Lynch and Walsh. In chapter $5$ of the book while discussing the concept of heritability, they seem to be claiming the following:

In a linear regression (where $y_i$ is observation $i$, $A$ is the known design matrix, $x$ is slopes we're estimating) minimising $||y - Ax||_2^2$, the error term, $y_i-a_i^{T}x$ is uncorrelated with $y_j-a_j^{T}x$, if $i \neq j$ and $a_i \neq a_j$. To me it is not clear why this is the case. Am I missing something here?

Improve this question
$\endgroup$
3
  • $\begingroup$ Can you define the notation without requiring us to go read the book? What's $x$ here? What's $A$? What's $a_i$? $\endgroup$
    – Glen_b
    Commented May 16, 2015 at 2:48
  • $\begingroup$ $y_i$ is the observation, $A$ is the known design matrix, $x$ is slopes, we're estimating here. $\endgroup$
    – Devil
    Commented May 17, 2015 at 3:09
  • $\begingroup$ Thanks. I've edited this information into your question. The reason I asked is that most statisticians would not typically use a Roman letter both for data ($y$) and for population parameters ($x$) -- mostly reserving Greek letters for the latter (usually they'd tend to see $y-X\beta$ where you have $y-Ax$). Lower case $x$ might perhaps then represent a row or column from $X$. Given that $x$ could tend to be misinterpreted, I thought it better to be explicit. $\endgroup$
    – Glen_b
    Commented May 17, 2015 at 3:19

1 Answer 1

Reset to default
5
$\begingroup$

Independence of the errors is a regression assumption; it doesn't follow from anything else.

But rather than quote the list there, I'll quote Andrew Gelman. Gelman gives the following list on his blog (itself, quoting Gelman&Hill):

  1. Validity. Most importantly, the data you are analyzing should map to the research question you are trying to answer. This sounds obvious but is often overlooked or ignored because it can be inconvenient. . . .

  2. Additivity and linearity. The most important mathematical assumption of the regression model is that its deterministic component is a linear function of the separate predictors . . .

  3. Independence of errors. . . .

  4. Equal variance of errors. . . .

  5. Normality of errors. . . .

Further assumptions are necessary if a regression coefficient is to be given a causal interpretation . .

The first isn't really a statistical assumption; if we include nonstatistical assumptions I think one could argue for some others, and I'd push them all into "further assumptions", but the remaining assumptions can be found in most lists; many people would add one or two additional assumptions that Gelman doesn't present. I largely agree with his ordering of the assumptions there.

[The normality is really only of any importance if we're performing inference that relies on it -- and in large samples often not even then; the estimates are efficient if normality holds, but they're usually reasonable if the distributions aren't such that all linear estimators are fairly bad.]

Improve this answer
$\endgroup$
2
  • $\begingroup$ The discrepancy between the top answer at stats.stackexchange.com/questions/16381/… and Gelman/Hill's comments has caused me a lot of confusion in the past. Your distinction between statistical and non-statistical assumptions helps to clarify things, although I still can't fully square the two accounts. $\endgroup$
    – user1205901 - Слава Україні
    Commented May 17, 2015 at 3:46
  • 1
    $\begingroup$ @user1205901 The top answer there seems to list desirable properties of an estimator and not the assumptions at all (it deliberately sets out to do so and explains why - and makes some excellent points for that). However, in my mind that answer is mostly or at least partly missing the point - the assumptions of regression largely relate to getting the required properties for hypothesis tests, CIs and PIs -- not simply estimation with desirable properties. $\endgroup$
    – Glen_b
    Commented May 17, 2015 at 3:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.