For least squares estimation, what is the difference between using the estimator $\hat{\beta} = X^{T}Y$ vs $\hat{\beta} = (X^{T}X)^{-1}X^{T}Y$ For least squares estimation, the estimator $\hat{\beta} = X^{T}Y$ is an unbiased estimator while $\hat{\beta} = (X^{T}X)^{-1}X^{T}Y$ is also an unbiased estimator given that $X$ is well-defined. Is there a difference between these two estimators through adding on the $(X^{T}X)^{-1}$ term? Thanks.
 A: Your first estimator doesn't work. It's not only biased, but the bias increases with the sample size.
Imagine a simple intercept model:
$$y_i=c+\varepsilon_i$$
here your design matrix is a simple vector of ones, $x_i=1$. 
Your estimator is $$\hat c = X'Y\equiv\sum_{i=1}^n y_i$$
If you add more data your estimator keeps increasing without bound: $$\hat c\to\infty$$ when $n\to\infty$
This doesn't make a sense, because, $E[y_i]=c$, assuming $E[\varepsilon_i]=0$, which suggests that maybe $\hat c=\bar y_i$ would be a good estimator.
However, here $X'X=n$, and with the second estimator you get what you'd expect from a reasonable looking estimator: 
$$\hat c=\frac{1}{n}\sum_{i=1}^n y_i\equiv \bar y$$
UPDATE:
Your second estimator will work in one case: if the design matrix is orthonormal and square, i.e. forms orthonormal basis.
The simplest case is when you have exactly one observation in my example. You have $X=1$, so $X'Y=y_1$, then you get:
$$\hat c=y_1$$
A: I don't believe that the first estimator is unbiased.
Under the linear model assumption
$$ Y = X \beta + \epsilon $$
the expectation of the first estimator is
$$ E[X^{T} Y] = E[X^{T}X \beta] + E[X^{T} \epsilon] = \underbrace{X^{T}X \beta + X^{T} E[\epsilon]}_{\text{linearity of expectation}} = X^{T}X \beta$$
From which we conclude that the proposed estimator is unbiased if and only if $X^{T} X \beta = \beta$.
This computation also clearly shows why the second estimator is always unbiased.
