# Has anyone studied linear regression where the covariance matrix of the error is a function of the parameters being estimated?

Consider the multivariate linear model: $$y = X\beta + e$$ $$y$$ is the measured output, $$X$$ is the model matrix, $$\beta$$ is the parameter vector, and $$e$$ is the zero-mean error vector: $$E[e] = 0 \qquad E[ee^T] = V$$ The goal is to estimate $$\beta$$ from $$y$$. It is well known that the minimum variance, affine unbiased estimator of $$\beta$$ exits if and only if $$X$$ has linear independent columns, in which case it is $$\hat\beta = [X^T (V + XX^T)^+ X]^{-1} X^T (V + XX^T)^+y$$ Many statisticians, mathematicians, econometricians, etc. have studied this problem where the error covariance matrix $$V$$ is partially or completely unknown. There are plenty of approaches on how to estimate $$\beta$$ and/or $$V$$ in this case.

My situation: I have an application where $$V$$ is unknown, but $$V$$ is a known function of $$\beta$$: $$V = f(\beta)$$ I have conducted a literature search looking for sources that have studied this problem, but I haven't found anything. It seems this situation has not been widely considered, if at all, but I am not a statistician; the literature is vast and I could just be searching for the wrong thing. However, if those sources are out there, I would like to cite them.

My question: Does anyone know of any papers/books/whatever that consider this problem?

• Being a non-statistician myself I suggest that you look for the literature referring to this specific problem in the relevant field, rather than for a general solution by statisticians/mathematicians/etc. The problem might be well explored by the practitioners for the realistic choices of $f(\beta)$, even if a general solution is impossible. – Vadim Nov 14 at 17:02
• @Vadim I appreciate the comment. Your suggestion is exactly what I am trying to do; my issue is that I don't really know where to start looking for such sources, and that is where I need help. – teerav42 Nov 14 at 21:18
• I suppose that there is an actual (not purely mathematical) problem in biology/economics/physics or another field that is behind the question? – Vadim Nov 14 at 21:25
• @Vadim actually, not really. The problem I am studying is noise covariance estimation for discrete time, time invariant linear systems. The work is not motivated by any particular physical system in particular. Essentially, $\beta$ is the covariance matrices of the noises which drive the system. This is the source of all of the uncertainty in the system, so this is why the case $V = f(\beta)$ occurs, in a nutshell. – teerav42 Nov 14 at 21:53

• Thanks! As a non-statistician, that is a bit of a mouthful for me. Without going into too much detail, what I am doing does involve a Gaussian AR model (though I'd just call it a discrete time linear model, i.e. $x_{k+1} = Ax_k + w_k$ and $y_k = Cx_k + v_k$. Can you suggest a reference I can start with? I know how I want to tackle the estimation problem for my application, so I don't care a ton about the details. Rather, I was about to write "the problem where $V = f(\beta)$ seems not to have been considered by many authors", so if that's not actually true, I just want to cite an example. – teerav42 Nov 14 at 6:37