Why not use linear regression when comparing two group's means? When deciding whether group means are different, people usually use ANOVA (or T-test).
For example, comparing the height(H) of male and female(G).
We may use t.test(H~G), and see the p.value.
However, ANOVA (T-test) cannot tell how much difference they are. 
My question is that 
why not just fit a linear regression like 
H = a+bG? (a is fitted intercept and b is fitted coefficient.)
Then people can both use the p value of b and the magnitude of b at the same time. But if we only use T-test, we can only get a p value. Also I believe that the p value of linear regression and p value of T-test are equivalent.
Did I miss something?
 A: Certainly a t-test can tell you how different they are, depending on what you mean by "how different".
If you mean the standardized difference in means - the number of standard-errors the means are apart - well, that's the t-statistic.
If you mean "the raw difference in means", many stats packages will give it to you (or failing that usually gives you the means from which you can calculate the difference -- like you couldn't calculate a mean difference anyway). 
R doesn't give the raw difference in means by default, but you can get it a couple of ways:
 #gives mean2 - mean1
 as.numeric(diff( t.test(extra ~ group, data = sleep)$estimate))

 #gives mean1 - mean2
 t.test(extra ~ group, data = sleep,c=0)$conf[1]

Of course there are packages that give other implementations of the t-test that do give it automatically (and it's a matter of seconds to write one that gives exactly the information you want) but in the case of vanilla R it's slightly 
easier to use regression to obtain the difference in means. R is atypical among packages in that it tends to give relatively terse output by default.
ANOVA itself is designed for more than two groups and doesn't give you a direct measure of difference in means for that reason (at least not without some extra effort).
There's nothing stopping you using regression instead of a t=test, but (except perhaps for some package defaults) it doesn't really gain you anything either.

the American Psychological Association (APA) recommended all published statistical reports also include effect size

Effect size is one of those things that has more than one meaning (see the opening para at the Wikipedia link there). 
While to many statisticians it will often be interpreted as the raw difference in means (in particular, ones that deal with data where the response is in meaningful units), by "effect size" the APA don't generally mean "raw difference in means" - they would seek Cohen's $d$.
