Sufficient Statistic for $\beta$ in OLS I have the classical regression model
$$y = \beta X + \epsilon$$
$$\epsilon \sim N(0, \sigma^2)$$
where $X$ is taken to be fixed (not random), and $\hat\beta$ is the OLS estimate for $\beta$.
It is known that $(y^T y, X^T y)$ pair is a complete sufficient statistic for $x_0^T \beta$, for some input $x_0$.
Can we conclude that $(y^T y, X^T y)$ is also a sufficient statistic for $\beta$, and why? I think for this to work $X^T X$ should be full rank. I mean a 1 to 1 transformation of a sufficient statistic is still a sufficient statistic, but it is still a sufficient statistic for $x_0^T \beta$. Based on what are we going to conclude the sufficiency of $\hat\beta$ for $\beta$ itself?
 A: Sometimes the simplest way to look at sufficiency is by looking directly at the log-likelihood and using the factorisation theorem.  For a linear regression model with Gaussian error term the log-likelihood function can be written as:
$$\begin{equation} \begin{aligned}
\ell_{\mathbf{y}, \mathbf{x}}(\boldsymbol{\beta}, \sigma) 
&= - n \ln \sigma -\frac{1}{2 \sigma^2} || \mathbf{y} - \mathbf{x} \boldsymbol{\beta} ||^2 \\[6pt]
&= - n \ln \sigma -\frac{1}{2 \sigma^2} (\mathbf{y} - \mathbf{x} \boldsymbol{\beta})^\text{T} (\mathbf{y} - \mathbf{x} \boldsymbol{\beta} ) \\[6pt]
&= - n \ln \sigma -\frac{1}{2 \sigma^2} (\mathbf{y}^\text{T} \mathbf{y} - \mathbf{y}^\text{T} \mathbf{x} \boldsymbol{\beta} - \boldsymbol{\beta}^\text{T} \mathbf{x}^\text{T} \mathbf{y} + \boldsymbol{\beta}^\text{T} \mathbf{x}^\text{T}  \mathbf{x} \boldsymbol{\beta} ) \\[6pt]
&= - n \ln \sigma -\frac{1}{2 \sigma^2} \mathbf{y}^\text{T} \mathbf{y} -\frac{1}{2 \sigma^2} ( 2 \boldsymbol{\beta}^\text{T} \mathbf{T}_1 - \boldsymbol{\beta}^\text{T} \mathbf{T}_2 \boldsymbol{\beta} ) \\[6pt]
&= h(\mathbf{y}, \sigma) + g_\boldsymbol{\beta}(\mathbf{T}_1, \mathbf{T}_2, \sigma), \\[6pt]
\end{aligned} \end{equation}$$
where $\mathbf{T}_1 \equiv \mathbf{T}_1(\mathbf{x}, \mathbf{y}) \equiv \mathbf{x}^\text{T}  \mathbf{y}$ and $\mathbf{T}_2 \equiv \mathbf{T}_2(\mathbf{x}, \mathbf{y}) \equiv \mathbf{x}^\text{T} \mathbf{x}$.  This shows that the statistic $\mathbf{T} \equiv (\mathbf{T}_1, \mathbf{T}_2)$ is sufficient for the coefficient parameter $\boldsymbol{\beta}$.  There is no requirement that the design matrix be of full rank for sufficiency, but if it is not of full rank then these statistics are not minimal sufficient (and you obtain a minimal sufficient statistic by reducing the design matrix to full rank).
From the above form we can also see that the OLS estimator $\hat{\boldsymbol{\beta}}$ is not sufficient for $\boldsymbol{\beta}$.  Sufficiency also requires knowledge of the matrix $\mathbf{T}_2 = \mathbf{x}^\text{T} \mathbf{x}$, which arises as part of the covariance of the OLS estimator.  This tells us that, in the case where the design matrix is of full rank, the OLS estimator and its covariance matrix are jointly sufficient for the unkonwn coefficient parameter.  (Of course, it is worth noting that regression problems always condition on $\mathbf{x}$, so in this context we get the required sufficiency.)
