Help: Partitioned samples efficiency in OLS compared to one sample regression

Question

As usual, we can estimate by OLS the model (in matrix form) $Y=\alpha+\beta*X+u$ with a sample of $n+m$ observations. The OLS estimator is $\hat{\beta}=(X^{T}X)^{-1}X^{T}Y$. Now, if we partition our sample in two subsamples A and B of sizes n and m, respectively, we could get the OLS estimators for each sumbsample: $\hat{\beta_A}=(X_A^{T}X_A)^{-1}X_A^{T}Y_A$ and $\hat{\beta_B}=(X_B^{T}X_B)^{-1}X_B^{T}Y_B$. We know both of these estimators are unbiased, and their variance is (assuming homoscedasticity) equal to $\sigma^2(X_i^{T}X_i)^{-1}$. Now let's define $\tilde{\beta}=\frac{\hat{\beta_A}+\hat{\beta_B}}{2}$, which is also unbiased, and, assuming no serial correlation, its variance is $V(\tilde{\beta})=\frac{V(\hat{\beta_A})+V(\hat{\beta_B})}{2}$.

How could we prove this estimator $\tilde{\beta}$ is less efficient/worse than $\hat{\beta}$?

Answer 1

This follows from some basic properties of quadratic forms. Let's get those out of the way first.

Fix a finite dimension $k$ and suppose $A$ and $B$ are both symmetric positive-definite (SPD) $k\times k$ matrices. This (easily) implies $A+B$ is symmetric positive-definite and all three of $A,$ $B,$ and $A+B$ are invertible, whence their inverses are SPD, too.

Lemma: For all vectors $u,$ $$u^\prime (A+B)^{-1} u \le \frac{1}{2} u^\prime(A^{-1}+B^{-1})u.$$

Proof: Let $v = (A+B)^{-1}u.$ Because $A^{-1}$ and $B^{-1}$ are SPD, $v^\prime BA^{-1}Bv = (Bv)^\prime (A^{-1}) (Bv) \ge 0$ and likewise $v^\prime A B^{-1} A v \ge 0.$

Therefore

$$\begin{aligned} u^\prime (A+B)^{-1} u &= v^\prime (A+B)(A+B)^{-1}(A+B) v\\ &= v^\prime(A+B)v\\ &\le v^\prime(A+B) v + \frac{1}{2}\left(v^\prime(A+B) v + v^\prime BA^{-1}B v + v^\prime AB^{-1}Av\right)\\ &= \frac{1}{2}v^\prime(A+B)(A^{-1}+B^{-1})(A+B)v\\ &= \frac{1}{2}u^\prime (A^{-1} + B^{-1})u, \end{aligned}$$

QED.

Apply this to $A = X_A^\prime X_A$ and $B = X_B^\prime X_B.$ Because $u$ represents an arbitrary linear combination of parameter estimates, conclude that the variance of any linear combination of parameter estimates using the second estimator $\tilde \beta$ is never less than its variance using the OLS estimator $\hat\beta.$

Answer 2

Consider the simple case of inference for the mean (see whuber's answer or the ideas from the comments for the general case), aka regression on a constant. Take $\sigma^2=1$ for simplicity.

Then, $V(\hat\beta_A)=1/n$, $V(\hat\beta_B)=1/m$, $V(\hat\beta)=1/(n+m)$. Write $T=n+m$ and $n=cT$, $m=(1-c)T$, $c\in(0,1)$. Then, and there is a little mistake in what you write, as constants leave variances in squares, $$ V(\tilde\beta)=\frac{1}{4}\left(\frac{1}{cT}+\frac{1}{(1-c)T}\right)=\frac{1}{4}\left(\frac{T}{cT(1-c)T}\right)=\frac{1}{4}\left(\frac{1}{c(1-c)T}\right) $$ We have that $c(1-c)$ is largest at $c=1/2$, so that $$ V(\tilde\beta)\geq\frac{1}{4}\left(\frac{1}{0.5(1-0.5)T}\right)=1/T=Var(\hat\beta) $$