Difference between regression analysis and analysis of variance?

Question

I am learning right now about regression analysis and the analysis of variance.

In regression analysis you have one variable fixed and you want to know how the variable goes with the other variable.

In analysis of variance you want to know for example: If this specific animal food influences the weight of animals... SO one fixed var and the influence on the others...

Is that right or wrong, pls help me...

Answer 1

Suppose your data set consists of a set $(x_i,y_i)$ for $i=1,\ldots,n$ and you want to look at the dependence of $y$ on $x$.

Suppose you find the values $\hat\alpha$ and $\hat\beta$ of $\alpha$ and $\beta$ that minimize the residual sum of squares $$ \sum_{i=1}^n (y_i - (\alpha+\beta x_i))^2. $$ Then you take $\hat y = \hat\alpha+ \hat\beta x$ to be the predicted $y$-value for any (not necessarily already observed) $x$-value. That's linear regression.

Now consider decomposing the total sum of squares $$ \sum_{i=1}^n (y_i - \bar y)^2 \qquad\text{where }\bar y = \frac{y_1+\cdots+y_n}{n} $$ with $n-1$ degrees of freedom, into "explained" and "unexplained" parts: $$ \underbrace{\sum_{i=1}^n ((\hat\alpha+\hat\beta x_i) - \bar y)^2}_{\text{explained}}\ +\ \underbrace{\sum_{i=1}^n (y_i - (\hat\alpha+\hat\beta x_i))^2}_{\text{unexplained}}. $$ with $1$ and $n-2$ degrees of freedom, respectively. That's analysis of variance, and one then considers things like F-statistics $$ F = \frac{\sum_{i=1}^n ((\hat\alpha+\hat\beta x_i) - \bar y)^2/1}{\sum_{i=1}^n (y_i - (\hat\alpha+\hat\beta x_i))^2/(n-2)}. $$ This F-statistic tests the null hypothesis $\beta=0$.

One often first encounters the term "analysis of variance" when the predictor is categorical, so that you're fitting the model $$ y = \alpha + \beta_i $$ where $i$ identifies which category is the value of the predictor. If there are $k$ categories, you'd get $k-1$ degrees of freedom in the numerator in the F-statistic, and usually $n-k$ degrees of freedom in the denominator. But the distinction between regression and analysis of variance is still the same for this kind of model.

A couple of additional points:

• To some mathematicians, the account above may make it appear that the whole field is only what is seen above, so it may seem mysterious that both regression and analysis of variance are active research areas. There is much that won't fit into an answer appropriate for posting here.
• There is a popular and tempting mistake, which is that it's called "linear" because the graph of $y=\alpha+\beta x$ is a line. That is false. One of my earlier answers explains why it's still called "linear regression" when you're fitting a polynomial via least squares.

Answer 2

The main difference is the response variable. While logistic regression deals with a binary response in linear regression analysis and also nonlinear regression the response variable is continuous. You have a variable(s) (aka covariate(s)) that have a functional relationship to the continuous response variable. In the analysis of variance the response is continuous but belongs to a few different categories (e.g. treatment group and control group). In the analysis of variance you look for difference in the mean response between groups. In linear regression you look at how the response changes as the covariates change. Another way to look at the difference is to say that in regression the covariates are continuous whereas in analysis of variance they are a discrete set of groups.

Answer 3

The analysis of variance (ANOVA) is a body of statistical method of analyzing observations assumed to be of the structure

$y_i=\beta_1x_{i1}+\beta_2x_{i2}+\dots+\beta_px_{ip}+e_i,~i=1(1)n$,which are constituted of linear combinations of $p$ unknown quantities $\beta_1,\beta_2,\dots,\beta_p$ plus errors $e_1,e_2,\dots,e_n$ and the {$x_{ij}$} are known constant coefficients with the r.v's {$e_i$} are uncorrelated and have the same mean $0$ and the variance $\sigma^2$(unknown).

i.e. $E(y^{n \times 1})=X\beta,D(y)=\sigma^2I_n$ Where D is dispersion matrix or variance-covariance matrix.

,where the coefficients {$x_{ij}$} are the values of counter variables or indicator variables which refer to the presence or absence of the effects {$\beta_j$} in the conditions under which the observations are taken:{$x_{ij}$} is the number of times $\beta_j$ occurs in the i-th observation,and this is usually $0$ or $1$.In general,in the analysis of variance all the factors are treated qualitatively.

If the {$x_{ij}$} are values taken on in the observations not by counter variables but by continuous variables like $t$=time ,$T$=temperature,$t^2,e^{-T}$,etc,then we have a case of *regression analysis.In general,in regression analysis all factors are quantitative and treated quantitatively.

Mainly,these two are two kinds of Analysis.