# Sufficiency of least squares estimator?

### How to prove the sufficiency of the least squares estimator?

Let $Y|X=Y_{1},...,Y_{n}$ a sequence of i.i.d random variables conditionated on the values of (fixed) $\mathbf{X}$, where $$\mathbf{X}=\left[\begin{array}{cccc} 1 & x_{11} & ... & x_{k1} \\ 1 & x_{12} & ... & x_{k2} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{1n} & ... & x_{kn} \end{array}\right]$$ The least squares estimator is defined by $$\boldsymbol{\hat{\beta}}=(\mathbf{X'X})^{-1}\mathbf{X'}\mathbf{Y}\sim \mathcal{N}(\boldsymbol\beta,\boldsymbol\Omega=\sigma^{2}(X'X)^{-1})$$

Then the densify function of the estimator is $$f_{\hat{\boldsymbol\beta}}(\hat{\boldsymbol\beta}|\boldsymbol\beta,\boldsymbol\Omega)=\frac{1}{(2\pi)^{p/2}|\boldsymbol\Omega|^{1/2}}\exp\left\{-\frac{1}{2}(\boldsymbol{\hat{\beta}}-\boldsymbol\beta)'\boldsymbol\Omega^{-1}(\boldsymbol{\hat{\beta}}-\boldsymbol\beta)\right\}$$

and the likelihood of the sample is

\begin{align} f_{Y|X,\boldsymbol{\beta}\sigma^{2}I_{k}}(Y|X,\boldsymbol{\beta},\sigma^{2}I_{k})=\left(\frac{1}{2\pi\sigma^{2}}\right)^{n/2}\exp\left\{-\frac{1}{2\sigma^{2}}(Y-X\boldsymbol{\beta})'(Y-X\boldsymbol{\beta})\right\} \end{align}

I know that the factorization theorem states that

Let $X_{1}, X_{2}, ..., X_{n}$ denote random variables with joint probability density function $f(x_{1}, x_{2}, ..., x_{n}; θ)$, which depends on the parameter $θ$.

Then, the statistic $Y=u(X1,X2,...,Xn)$

is sufficient for θ if and only if the p.d.f can be factored into two components, that is:

$$f(x_{1},x_{2},...,x_{n};θ)=\phi[u(x_{1},x_{2},...,x_{n});θ]h(x1,x2,...,xn)$$

How to proceed in this case?

• This deserves a self-study tag, I believe, and you have to detail why you cannot establish the sufficiency on your own. Aug 26 '16 at 0:07
• Okay. I will edit the question. Aug 26 '16 at 0:09
• What do you know about sufficiency? What have you tried? Aug 26 '16 at 0:42
• @Glen_b I have edited the question Aug 26 '16 at 0:48

1. try expanding the exponent, collecting terms (in the parameter, so you get terms of the form $\beta'A\beta$ and $B\beta$ for example )
2. "complete the square", rewriting the terms in the form $(\beta-d)'A(\beta-d)+C$