Sufficiency of least squares estimator?

Question

How to prove the sufficiency of the least squares estimator?

Let $Y|X=Y_{1},...,Y_{n}$ a sequence of i.i.d random variables conditionated on the values of (fixed) $\mathbf{X}$, where \begin{equation} \mathbf{X}=\left[\begin{array}{cccc} 1 & x_{11} & ... & x_{k1} \\ 1 & x_{12} & ... & x_{k2} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{1n} & ... & x_{kn} \end{array}\right] \end{equation} The least squares estimator is defined by \begin{equation} \boldsymbol{\hat{\beta}}=(\mathbf{X'X})^{-1}\mathbf{X'}\mathbf{Y}\sim \mathcal{N}(\boldsymbol\beta,\boldsymbol\Omega=\sigma^{2}(X'X)^{-1}) \end{equation}

Then the densify function of the estimator is \begin{equation} f_{\hat{\boldsymbol\beta}}(\hat{\boldsymbol\beta}|\boldsymbol\beta,\boldsymbol\Omega)=\frac{1}{(2\pi)^{p/2}|\boldsymbol\Omega|^{1/2}}\exp\left\{-\frac{1}{2}(\boldsymbol{\hat{\beta}}-\boldsymbol\beta)'\boldsymbol\Omega^{-1}(\boldsymbol{\hat{\beta}}-\boldsymbol\beta)\right\} \end{equation}

and the likelihood of the sample is

\begin{align} f_{Y|X,\boldsymbol{\beta}\sigma^{2}I_{k}}(Y|X,\boldsymbol{\beta},\sigma^{2}I_{k})=\left(\frac{1}{2\pi\sigma^{2}}\right)^{n/2}\exp\left\{-\frac{1}{2\sigma^{2}}(Y-X\boldsymbol{\beta})'(Y-X\boldsymbol{\beta})\right\} \end{align}

I know that the factorization theorem states that

Let $X_{1}, X_{2}, ..., X_{n}$ denote random variables with joint probability density function $f(x_{1}, x_{2}, ..., x_{n}; θ)$, which depends on the parameter $θ$.

Then, the statistic $Y=u(X1,X2,...,Xn)$

is sufficient for θ if and only if the p.d.f can be factored into two components, that is:

\begin{equation} f(x_{1},x_{2},...,x_{n};θ)=\phi[u(x_{1},x_{2},...,x_{n});θ]h(x1,x2,...,xn) \end{equation}

How to proceed in this case?

Answer 1

Starting with the joint density (/ likelihood)

1. try expanding the exponent, collecting terms (in the parameter, so you get terms of the form $\beta'A\beta$ and $B\beta$ for example )

2. "complete the square", rewriting the terms in the form $(\beta-d)'A(\beta-d)+C$

3. You should then be able to get it to be a product of two terms and use the theorem you mention