Consider a linear system $y_i=Wx_i+v_i$, where $x_i\in R^{d\times1} \sim N(0,\Sigma_x)$, $v_i\in R^{p\times1} \sim N(0,\Sigma_v)$, $y_i\in R^{p\times1}$ and $W\in R^{p\times d}$.

Now we consider a linear estimator $\hat{x}_i=\beta_1y_i+\beta_0$. We know that the optimal theoretical least square solution of is given by:



On the other hand, suppose we have a set consisting of $N$ ovservations pairs $\{(x_1,y_1),(x_2,y_2),..,(x_N,y_N)\}$. We denote $Y_N=[y_1,y_2,...,y_N]$, $X_N=[x_1,x_2,...,x_N]$ and $V_N=[v_1,v_2,...,v_N]$. the abover system and the linear estimator can be written as


$\hat{X}_N=[\beta_1, \beta_0]\begin{bmatrix}Y_N\\1\end{bmatrix}$.

Therefore, the least square solution is given by :

$[\beta^{\ast\ast}_1, \beta^{\ast\ast}_0]=X_N[Y^\top_N, 1^\top](\begin{bmatrix}Y_N\\1\end{bmatrix}[Y^\top_N, 1^\top])^{-1}$.

where $\beta^{\ast\ast}_1$ and $\beta^{\ast\ast}_0$ can be considered as random variables.

We know that when $N\to \inf$, $\beta^{\ast\ast}_1\to \beta^{\ast}_1$.

My question is: when N is finite, can we describe the relationship between $\beta^{\ast\ast}_1$ and $\beta^{\ast}_1$, e.g., the pdf of $\beta^{\ast\ast}_1-\beta^{\ast}_1$, for a given $N$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.