2
$\begingroup$

Suppose we are interested in the covariance matrix $\Sigma$ of a few MLE estimators $\hat \theta_1,\hat \theta_2,\cdots,\hat \theta_n$. For each $j$, $\hat \theta_j$ is normally distributed and estimated from data. The data is multivariate normal with known covariance and mean $\vec 0$.

The problem is, I obtained the covariance matrix $\Sigma$ heuristically because it was impossible to compute directly. Now I want to prove that I have found the correct expression. What are some methods which would prove that I have found the correct covariance matrix?

EDIT: I was asked to explain how the covariance matrices was were obtained heuristically. A few dozens of covariance matrices of interest were generated numerically. This is possible since $\hat \theta_i$ has closed form and therefore $\Sigma$ has closed form. By staring at the geometrical pattern they formed it was possible to guess the formula for the elements of the covariance matrices. Next, numerics were used again to compute thousands of such matrices to check that the guess was correct.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Can you write a little bit about how you actually found the covariance matrix? $\endgroup$
    – eric_kernfeld
    Oct 9, 2017 at 15:39
  • $\begingroup$ Thanks! Now I have more questions. For the formula you found, are the inputs the $\theta$'s? If $\Sigma$ has a known closed form, why do you need to ask this question? In your simulations, do you generate thousands of samples and compute a single sample covariance matrix, or do you generate thousands of sample covariance matrices for various values of ${\theta}_{j=1}^n$? $\endgroup$
    – eric_kernfeld
    Oct 10, 2017 at 12:56
  • $\begingroup$ It's true that $\Sigma$ has closed form, but it's unpractical to write down apriori because each element in $\Sigma$ it is the trace of a product of 4 matrices which vary with every element in Sigma. Ie $\mathrm{Cov}(\hat \theta_i, \hat \theta_j) = \mathrm{Tr}(A_jBC_iD)$. Suppose $X_1,\cdots,X_n$ are the observations with variance $\sigma^2$. The formula found depend on $n$ and $\sigma^2$. In simulations I generate $A_i,B,C_j,D$,for various dimensions $n \times n$ and indecies $i$ and $j$ then generate the trace. Recall that each element of $\Sigma$ is such a trace $\endgroup$
    – Mikkel Rev
    Oct 10, 2017 at 13:33
  • $\begingroup$ Do you want to look for a proof of truth (which requires getting the closed form)? Or did you mean by proof some way to improve it's plausibility (which can be done by some statistical model)? $\endgroup$
    – Sextus Empiricus
    Oct 11, 2017 at 1:09
  • $\begingroup$ Proof of truth. $\endgroup$
    – Mikkel Rev
    Oct 11, 2017 at 19:53

1 Answer 1

Reset to default
1
+50
$\begingroup$

The covariance is a function of some matrices that you know: $$\text{Cov}(\hat\theta_i,\hat\theta_j)=\text{Tr}(A_jBC_iD) = Q(A_j, B_i)$$.

You came up with a formula $f(i, j, n, \sigma^2, ... ?)$ and you want to show that it equals your trace formula ($Q$) for each $i$, $j$, and $n$. The trace formula $Q$ is linear in each of $A_j$ and $B_i$, and I assume that your $f$ is also bilinear in some corresponding parameters. My go-to tactic is to show equality on pairs of basis functions for the linear spaces inhabited by $A$ and $B$. If you can show that your formula is correct for a sufficient number of linearly independent special cases, you can argue that they determine its behavior for all possible inputs.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.