1
$\begingroup$

Perusing various documents is see references to least squares regression that is said to be different from OLS regression(1,2,3), and comparisons between "regression" and standard ANOVA(4).

It appears the comparisons to standard ANOVA are talking about OLS regression, due to the assumption of normality and independence of the residuals, and the assumption of the homogeneity of variance.

I am posting this to check that this is correct.

The references are included to explain why a person browsing documents would ask this question. (As two comments say this is not understood.)

  1. Fomby T.B., Johnson S.R., Hill R.C. (1984) Review of Ordinary Least Squares and Generalized Least Squares. In: Advanced Econometric Methods. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8746-4_2

    "The purpose of this chapter is to review the fundamentals of ordinary least squares and generalized least squares in the context of linear regression analysis. ... In Section 2.4 we introduce the large sample concepts of convergence in probability and consistency. It is shown that convergence in quadratic mean is a sufficient condition for consistency and that the ordinary least squares estimator is consistent. In Section 2.5 the generalized least squares model is defined and the optimality of the generalized least squares estimator is established by Aitken’s theorem..."

  2. https://en.wikipedia.org/wiki/Least_squares

    "Least-squares problems fall into two categories: linear or ordinary least squares and nonlinear least squares, depending on whether or not the residuals are linear in all unknowns."

  3. https://www.quora.com/Regression-statistics-What-is-the-difference-between-Ordinary-least-square-and-generalized-least-squares

    "OLS gives the maximum likelihood estimate for β when the parameters have equal variance and are uncorrelated ... Generalized least squares allows this approach to be generalized to give the maximum likelihood estimate when the noise is colored (heteroscedasticity)..."

  4. Multiple Regression as a Flexible Alternative to ANOVA in L2 Research, Studies in Second Language Acquisition, 2017, 39, 579–592. doi:10.1017/S0272263116000231

Improve this question
$\endgroup$
16
  • 2
    $\begingroup$ Thank you. Are you inquiring about differences and similarities among OLS, ANOVA, weighted OLS, generalized LS, GLM, Maximum Likelihood, and "nonlinear least squares" (your references mention them all)? What do you want to focus on? $\endgroup$
    – whuber ♦
    Commented Aug 25, 2020 at 18:09
  • 3
    $\begingroup$ It's hard to understand what you are asking because the premise seems confused. OLS (Ordinary Least Squares) is a particular Least Squares (LS) method. Other LS methods include WLS (Weighted LS), GLS (Generalized LS), TLS (Total LS), etc. ANOVA stands for the ANalysis Of VAriance. The default ANOVA would typically use OLS, but weighted and generalized, etc., methods are possible, as well. Note further that ANOVA is a special case of regression. Thus the question reads something like, 'which is analogous to rabbit, animal or mammal?' $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 25, 2020 at 18:56
  • 1
    $\begingroup$ Both are "related to standard ANOVA". Standard ANOVA is OLS regression, which is LS regression, just as a rabbit is a mammal, which is an animal. $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 25, 2020 at 19:53
  • 1
    $\begingroup$ OK: Standard ANOVA is LS regression. $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 25, 2020 at 20:21
  • 1
    $\begingroup$ There are a few reasons: (1) opening up question is determined by a CV community vote and they may not have had a chance to vote on this yet or (2) I'm guessing you haven't updated with the question sufficient clarity since your first update which basically consisted of a dump of articles you found, some of which deal with OLS, but others that deal with generalized linear models and other models that have some still scratching their heads, or (3) both. $\endgroup$
    – StatsStudent
    Commented Aug 25, 2020 at 21:37

1 Answer 1

Reset to default
0
$\begingroup$

The comparisons of regression to standard ANOVA are referring to Ordinary Least Squares (OLS) specifically. This is based on the comments to the question, and my own reading.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.