$\hat{\beta}^{(M)}_i\sim \hat{\beta}^{(N)}_i$ for linear regression?

Question

Consider an i.i.d. sample $(X_1, Y_1), \dots, (X_N, Y_N)$, where each $X_i$ and $Y_i$ are $n$-dimensional column vectors, let $M \leq N$ and denote by $\hat{\beta}^{(M)}$ and $\hat{\beta}^{(N)}$ the coefficients resulting from applying linear regression to the sample comprising the first $M$/$N$ pairs, respectively. Is it true or false that $$ \hat{\beta}^{(M)}_i \sim \hat{\beta}^{(N)}_i $$ where $\hat{\beta}^{(K)}_i$ ($K\in\{M,N\}$) is the $i$th component of the vector $\hat{\beta}^{(K)}$.

Answer 1

Note that if $Y_i$'s are $n$-dimensional vectors, than $\hat\beta$ will be a matrix rather than a vector, if I understand your formulation correctly.

Never mind, the simple answer is no, the more samples, the more concentrated the value of $\hat\beta$ will be, so distributions will not be the same (I assume that's what $\hat\beta^{(M)} \sim \hat\beta^{(N)}$ means). You can try simulating an example:

import numpy as np
import statsmodels.api as sm
import matplotlib.pyplot as plt

N, M = 100, 10
b_N, b_M = [], []

for i in range(5000):
    x = np.random.random(N)
    y = 2*x + 1 + np.random.random(N)
    _, b_1 = sm.OLS(y, sm.add_constant(x)).fit().params
    b_N.append(b_1)
    _, b_1 = sm.OLS(y[:M], sm.add_constant(x[:M])).fit().params
    b_M.append(b_1)

plt.hist(b_M)
plt.hist(b_N)
plt.show()

In theory it is possible to derive the distribution for $\hat\beta$ given the joint distribution for $(X, Y)$, arranging $n$ samples in matrices $\mathbf{X}, \mathbf{Y}$ and noting that $\hat\beta$ is a random variable $\hat\beta = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}$.

I think that writing $\mathbf{X}^T \mathbf{X} = n \mathbf{Z}$ we can intuitively argue that increasing $n$ we get $\mathbf{Z}$ more and more concentrated around $E(X^T X)$ (and the same holds for $\mathbf{X^TY}$), so after inversion and cancellation of $n$ we are left with product of two random variables, which are more concentrated as $n$ grows.