# Sampling Distribution (Variance) of Weight Estimates

I am currently facing an issue regarding the sampling distribution of weight estimates.

# Problem Statement

Given an estimate of a $$n \times n$$ covariance matrix $$\hat{\Sigma}$$ of $$n$$ random variables $$X_1 \sim N(0,\sigma_1^2),\dots,X_n \sim N(0, \sigma_n^2)$$, I calculate $$\hat{w} = \frac{\hat{\Sigma}^{-1} \vec 1}{{\vec 1}^T \hat{\Sigma}^{-1}\vec 1}$$

$$\hat{w}$$ is obviously a vector of lenght $$n$$ and has a sum of 1. It gives a weight to each random variable so that the variance of the weighted sum of the random variables is minimized.

Assuming that the true covariance matrix $$\Sigma$$ is known, the true value of $$w$$ is also known.

I am now looking for the sampling distribution of $$\hat{w}$$. The expected value of $$\hat{w}$$ is clear: $$E[\hat{w}] = w$$. The sampling variance is however unclear.

$$\hat{\Sigma}$$ clearly follows an Inverse-Wishart distribution: $$\hat{\Sigma} \sim W_p^{-1}(\Sigma^{-1},n)$$

Although the variance of the elements of $$\Sigma^{-1}$$ is consequently defined, calculating the sampling variance of the elements of $$\hat{w}$$ starting from here seems quite tedious, especially since a ratio is involved.

I therefore wonder whether there is a known simple formulation for the sampling variance I am looking for.

# Alternative Regression Formulation

The problem can also be stated in a regression context.

Assuming that $$X$$ is a matrix of $$m$$ realizations of the $$n$$ random variables. We can then define $$Y'=X_1$$ and $$X'=(X_1-X_2,\dots,X_1-X_n)$$ where $$X_i$$ is the i-th column of the data matrix.

Regressing $$Y'= X \beta + \epsilon$$ results in $$n-1$$ coefficients $$\hat{w}'$$ which are equal to $$\hat{w}_2,\dots,\hat{w}_n$$. $$\hat{w}_1$$ then equals $$1- \sum\hat{w}'$$.

The sampling variance of the simple regression coefficient ($$m=2$$) is known from papers from Pearson (1926) and Romanovsky (1926): $$Var(\hat{\beta_1}) = \frac{\sigma_Y^2}{\sigma_X^2} \frac{1-\rho^2}{n - 3}$$ which can be transferred to the specific setting where $$X_1$$ is the dependent variable and $$X_1 - X_2$$ is the independent variable.

I have however not found a generalization of the results of Pearson and Romanovsky to $$m > 2$$. A results regarding the sampling distribution of the multiple regression coefficients would consequently also be of great help for my problem and could be transfered in a second step.

I would appreciete any help. Thank you in advance.

Edit: Interesingly, the sampling distribution of the elements of $$\hat{w}$$ seems to be normal. In the following plot, I have generated the sampling distribution of one element of $$\hat{w}$$ for an example $$\Sigma$$ in 50,000 simulation runs. The black line is the resulting density whereas the red line is the corresponding normal distribution with the estimated mean and variance.

$\hat{w}$" />

• Presumably a "unit vector" is not one with unit length, but one all of whose coefficients are $1$. Even so, why is it obvious that $\mathbb{E}(\hat w)=w$? In some cases $\hat w$ is not even defined with positive probability--what then?
– whuber
Commented Jun 25, 2015 at 13:28
• Thank you for point out the mistake regarding the "unit vector". I have edited the question accordingly. Regarding your second comment, I am trying to imagine cases where $\hat{w}$ is not defined with positive probability. Could you elaborate with an example when this could be the case? In my problem, all the random variables are normally distributed with mean zero and different variances. In my simulations, I have not yet observed a case where issues with $\hat{w}$ occurred. Furthermore, in all of the simulated cases, I found that $E[\hat{w}] = w$. Commented Jun 25, 2015 at 13:38
• Your post refers to "random variables" without any restriction. When they have finite support, there will be some positive chance that $\hat\Sigma$ is not invertible. For instance, when there are two independent Bernoulli(1/2) variables that are sampled three times, the chance that $\hat\Sigma$ is not invertible is $5/8$. More generally, because $\hat w$ is a nonlinear function of the data, it would be surprising that its expectation would agree with $w$ itself, so I am interested in understanding how you concluded that would be the case even with Normally distributed variables.
– whuber
Commented Jun 25, 2015 at 13:59
• You are definitely correct regarding other distributions with finite support. I have corrected the initial question to explicitly state that the random variables follow a normal distribution. The asumption regarding the expectation $E[\hat{w}] = w$ stems from the connection to linear regression which I described in my question. The regression estimator should be unbiased, resulting in said assumption of mine. Just to avoid misunderstandings: I assume that $E[\hat{w}] = w = \frac{\Sigma^{-1} \vec 1 }{{\vec 1}^T \Sigma^{-1} \vec 1 }$, which has been so far confirmed by my simulation experiments. Commented Jun 25, 2015 at 18:56