# Help: Partitioned samples efficiency in OLS compared to one sample regression

As usual, we can estimate by OLS the model (in matrix form) $$Y=\alpha+\beta*X+u$$ with a sample of $$n+m$$ observations. The OLS estimator is $$\hat{\beta}=(X^{T}X)^{-1}X^{T}Y$$. Now, if we partition our sample in two subsamples A and B of sizes n and m, respectively, we could get the OLS estimators for each sumbsample: $$\hat{\beta_A}=(X_A^{T}X_A)^{-1}X_A^{T}Y_A$$ and $$\hat{\beta_B}=(X_B^{T}X_B)^{-1}X_B^{T}Y_B$$. We know both of these estimators are unbiased, and their variance is (assuming homoscedasticity) equal to $$\sigma^2(X_i^{T}X_i)^{-1}$$. Now let's define $$\tilde{\beta}=\frac{\hat{\beta_A}+\hat{\beta_B}}{2}$$, which is also unbiased, and, assuming no serial correlation, its variance is $$V(\tilde{\beta})=\frac{V(\hat{\beta_A})+V(\hat{\beta_B})}{2}$$.

How could we prove this estimator $$\tilde{\beta}$$ is less efficient/worse than $$\hat{\beta}$$?

• The Gauss-Markov Theorem asserts this.
– whuber
Sep 10, 2022 at 19:07
• I understand, but I'm not seeing how to prove it in this example without invoking Gauss-Markov; would that be possible? Sep 10, 2022 at 19:17

This follows from some basic properties of quadratic forms. Let's get those out of the way first.

Fix a finite dimension $$k$$ and suppose $$A$$ and $$B$$ are both symmetric positive-definite (SPD) $$k\times k$$ matrices. This (easily) implies $$A+B$$ is symmetric positive-definite and all three of $$A,$$ $$B,$$ and $$A+B$$ are invertible, whence their inverses are SPD, too.

Lemma: For all vectors $$u,$$ $$u^\prime (A+B)^{-1} u \le \frac{1}{2} u^\prime(A^{-1}+B^{-1})u.$$

Proof: Let $$v = (A+B)^{-1}u.$$ Because $$A^{-1}$$ and $$B^{-1}$$ are SPD, $$v^\prime BA^{-1}Bv = (Bv)^\prime (A^{-1}) (Bv) \ge 0$$ and likewise $$v^\prime A B^{-1} A v \ge 0.$$

Therefore

\begin{aligned} u^\prime (A+B)^{-1} u &= v^\prime (A+B)(A+B)^{-1}(A+B) v\\ &= v^\prime(A+B)v\\ &\le v^\prime(A+B) v + \frac{1}{2}\left(v^\prime(A+B) v + v^\prime BA^{-1}B v + v^\prime AB^{-1}Av\right)\\ &= \frac{1}{2}v^\prime(A+B)(A^{-1}+B^{-1})(A+B)v\\ &= \frac{1}{2}u^\prime (A^{-1} + B^{-1})u, \end{aligned}

QED.

Apply this to $$A = X_A^\prime X_A$$ and $$B = X_B^\prime X_B.$$ Because $$u$$ represents an arbitrary linear combination of parameter estimates, conclude that the variance of any linear combination of parameter estimates using the second estimator $$\tilde \beta$$ is never less than its variance using the OLS estimator $$\hat\beta.$$

Consider the simple case of inference for the mean (see whuber's answer or the ideas from the comments for the general case), aka regression on a constant. Take $$\sigma^2=1$$ for simplicity.

Then, $$V(\hat\beta_A)=1/n$$, $$V(\hat\beta_B)=1/m$$, $$V(\hat\beta)=1/(n+m)$$. Write $$T=n+m$$ and $$n=cT$$, $$m=(1-c)T$$, $$c\in(0,1)$$. Then, and there is a little mistake in what you write, as constants leave variances in squares, $$V(\tilde\beta)=\frac{1}{4}\left(\frac{1}{cT}+\frac{1}{(1-c)T}\right)=\frac{1}{4}\left(\frac{T}{cT(1-c)T}\right)=\frac{1}{4}\left(\frac{1}{c(1-c)T}\right)$$ We have that $$c(1-c)$$ is largest at $$c=1/2$$, so that $$V(\tilde\beta)\geq\frac{1}{4}\left(\frac{1}{0.5(1-0.5)T}\right)=1/T=Var(\hat\beta)$$