Why is ANOVA equivalent to linear regression? I read that ANOVA and linear regression are the same thing. How can that be, considering that the output of ANOVA is some $F$ value and some $p$-value based on which you conclude if the sample means across the different samples are same or different.
But assuming the means are not equal (reject null hypothesis), ANOVA tells you nothing about the coefficients of the linear model. So how is linear regression the same as ANOVA?
 A: Antoni Parellada and usεr11852 had very good answer. I will address your question for coding perspective with R.

ANOVA tells you nothing about the coefficients of the linear model. So how is linear regression the same as ANOVA?

In fact, we can aov function in R can be used as same as lm. Here are some examples.
> lm_fit=lm(mpg~as.factor(cyl),mtcars)

> aov_fit=aov(mpg~as.factor(cyl),mtcars)

> coef(lm_fit)
    (Intercept) as.factor(cyl)6 as.factor(cyl)8 
      26.663636       -6.920779      -11.563636 

> coef(aov_fit)
    (Intercept) as.factor(cyl)6 as.factor(cyl)8 
      26.663636       -6.920779      -11.563636 

> all(predict(lm_fit,mtcars)==predict(aov_fit,mtcars))
[1] TRUE

As you can see, not only we can get coefficient from ANOVA model, but also we can use it for prediction, just like the linear model. 
If we check the help file for aov function it says

This provides a wrapper to lm for fitting linear models to balanced or unbalanced experimental designs. The main difference from lm is in the way print, summary and so on handle the fit: this is expressed in the traditional language of the analysis of variance rather than that of linear models.

A: ANOVA and linear regression are equivalent when the two models test against the same hypotheses and use an identical encoding. The models differ in their basic aim: ANOVA is mostly concerned to present differences between categories' means in the data while linear regression is mostly concern to estimate a sample mean response and an associated $\sigma^2$.
Somewhat aphoristically one can describe ANOVA as a regression with dummy variables. We can easily see that this is the case in the simple regression with categorical variables. A categorical variable will be encoded as a indicator matrix (a matrix of 0/1 depending on whether a subject is part of a given group or not) and then used directly for the solution of the linear system described by a linear regression.
Let's see an example with 5 groups. For the sake of argument I will assume that the mean of group1 equals 1,  the mean of group2 equals 2, ... and the mean of group5 equals 5. (I use MATLAB, but the exact same thing is equivalent in R.)
rng(123);               % Fix the seed
X = randi(5,100,1);     % Generate 100 random integer U[1,5]
Y = X + randn(100,1);   % Generate my response sample
Xcat = categorical(X);  % Treat the integers are categories

% One-way ANOVA
[anovaPval,anovatab,stats] = anova1(Y,Xcat);
% Linear regression
fitObj = fitlm(Xcat,Y);

% Get the group means from the ANOVA
ANOVAgroupMeans = stats.means
% ANOVAgroupMeans =
% 1.0953    1.8421    2.7350    4.2321    5.0517

% Get the beta coefficients from the linear regression
LRbetas = [fitObj.Coefficients.Estimate'] 
% LRbetas =
% 1.0953    0.7468    1.6398    3.1368    3.9565

% Rescale the betas according the intercept
scaledLRbetas = [LRbetas(1) LRbetas(1)+LRbetas(2:5)]
% scaledLRbetas =
% 1.0953    1.8421    2.7350    4.2321    5.0517

% Check if the two results are numerically equivalent
abs(max( scaledLRbetas - ANOVAgroupMeans)) 
% ans =
% 2.6645e-15

As it can be seen in this scenario the results where exactly the same. The minute numerical difference is due to the design not being perfectly balanced as well as the underlaying estimation procedure; the ANOVA accumulates numerical errors a bit more aggressively. To that respect we fit an intercept, LRbetas(1); we could fit an intercept-free model but that would not be a "standard" linear regression. (The results would be even closer to ANOVA in that case though.)
The $F$-statistic (a ratio of the means) in the case of the ANOVA and in the case of linear regression will be also be the same for the above example: 
abs( fitObj.anova.F(1) - anovatab{2,5} )
% ans =
% 2.9132e-13 

This is because procedures test the same hypothesis but with different wordings: ANOVA will qualitatively check if "the ratio is high enough to suggest that no grouping is implausible" while linear regression will qualitatively check if "the ratio is high enough to suggest an intercept only model is possibly inadequate".
(This is a somewhat free interpretation of the "possibility to see a value equal or greater than the one observed under the null hypothesis" and it is not meant to be a text-book definition.)
Coming back to the final part of your question about "ANOVA tell(ing) you nothing about the coefficients of the linear model (assuming the means are not equal") I hope you can now see that the ANOVA, in the case that your design is simple/balanced enough, tells you everything that a linear model would. The confidence intervals for group means will be the same you have for your $\beta$, etc. Clearly when ones starts adding multiple covariate in his regression model, a simple one-way ANOVA does not have a direct equivalence. In that case one augments the information used to calculate the linear regression's mean response with information that are not directly available for a one way ANOVA. I believe that one can re-express things in ANOVA terms once more but it is mostly an academic exercise.
An interesting paper on the matter is Gelman's 2005 paper titled: Analysis of Variance - Why it is more important than ever. Some important points raised; I am not fully supportive of the paper (I think I personally align much more with McCullach's view) but it can be a constructive read.
As a final note: The plot thickens when you have mixed effects models. There you have different concepts about what can be considered a nuisance or actual information regarding the grouping of your data. These issues are outside the scope of this question but I think they are worthy of a nod. 
A: ANOVA is not a model; it is a method within a model
The analysis of variance (ANOVA) is a method that occurs within regression models.  The technique gives a particular set of outputs for the model that analyse the estimated variance of different parts, and use this to make inferences about whether or not there are relationships between the explanatory variables and the response variable.  Comparison of linear regression to ANOVA is an "apples and oranges" comparison, since the former is a model and the latter is a method of analysis that occurs within a model.  Sometimes you will see references to ANOVA as if it were a model, in which case there must be some underlying model to which the method is applied.
The ANOVA method is fairly simple to understand when you look at it holistically in the context of a general regression model.  The technique is based on the law of iterated variance.  Suppose you are working in the context of some regression model:
$$Y_i = f(\mathbf{x}_i, \theta) + \varepsilon_i
\quad \quad \quad
\varepsilon_1,...,\varepsilon_n \sim \text{IID Dist}(0, \sigma^2).$$
Using the law of iterated variance we can write the marginal variance of $Y_i$ as:
$$\begin{equation} \begin{aligned}
\mathbb{V}(Y_i)
&= \mathbb{V}(\mathbb{E}(Y_i|\mathbf{X}_i)) + \mathbb{E}(\mathbb{V}(Y_i|\mathbf{X}_i)) \\[6pt]
&= \mathbb{V}(f(\mathbf{X}_i, \theta)) + \mathbb{E}(\mathbb{V}(\varepsilon_i)) \\[6pt]
&= \mathbb{V}(f(\mathbf{X}_i, \theta)) + \mathbb{E}(\sigma^2) \\[6pt]
&= \mathbb{V}(f(\mathbf{X}_i, \theta)) + \sigma^2. \\[6pt]
\end{aligned} \end{equation}$$
Now, if the explanatory vector does not have a relationship with the response variable then the regression function does not depend on the explanatory vector, and so $\mathbb{V}(f(\mathbf{X}_i, \theta)) = 0$, which implies $\mathbb{V}(Y_i) =  \sigma^2$.  On the other hand, if the explanatory vector does have a relationship with the response variable, then we will generally have $\mathbb{V}(f(\mathbf{X}_i, \theta)) > 0$, which implies $\mathbb{V}(Y_i) >  \sigma^2$.  Thus, generally speaking, a larger gap between the estimated variance of the response variable, and the estimated variance of the error term, constitutes evidence in favour of the hypothesis that there is a relationship between the explanatory vector and the response variable.
ANOVA can be taken further than this by breaking down the variance contributions of each of the explanatory variables in the model (or groups of these variables, etc.) to allow you to further test whether there is a plausible relationship between the response variable and an individual explanatory variable or group of explanatory variables.  The lovely answer by Antoni Parellada shows you a colourful illustration of estimating the variance in three groups from a categorical explanatory variable.
The above decomposition from the law of iterated variance is the basic insight that underlies ANOVA.  It is used to construct formal ANOVA tests to determine whether or not there is evidence of a relationship between the explanatory vector and the response variable.  By conditioning on parts of the explanatory vector, this basic method can also be used to test for a relationship between particular subsets of explanatory variables and the response variable.  In summary, ANOVA is a particular method that is used within the context of regression analysis to test for relationships between variables.

When is ANOVA "equivalent" to linear regression: The technique of ANOVA gives you a breakdown of the estimated variance of the components in the data, and this is often augmented with formal F-tests that use those estimated variance components.  This is "equivalent" to performing the F-tests in a regression model, since that is what you are doing in ANOVA.  (In simple linear regression there is only one explanatory variable so the F-test gives the same p-value as the T-test for this coefficient.  In this case the ANOVA test is also equivalent to the individual T-test for the sole explanatory variable.)
You are correct that there are many aspects of the regression model that fall outside the scope of ANOVA (e.g., estimation of the coefficients for the individual explanatory variables).  Again, that is because ANOVA is a method that occurs within the context of a regression model, not a model in its own right.  When you see a whole model referred to as an "ANOVA model", it is more accurate to think of it as an ANOVA method applied to an underlying regression model.
A: If we take all data entries and arrange them into one single column Y, with the rest of the columns being indicator variables 1{ith data is element of the jth column in the original anova arrangement} then by taking a simple linear regression of Y on any of the other columns (say column B), you should obtain the same DF, SS, MS and F test statistic as in your ANOVA problem. 
Thus ANOVA can be 'treated as' Linear Regression by writing the data with binary variables. Also note that the coefficient of regression for, say, a regression of Y on B should be the same as the avg. of the column B, computed with the original data.   
