3
$\begingroup$

Consider the following linear Gaussian system:

enter image description here

where $p(x)$ is our prior. The Bayesian inference problem can be expressed in closed-form as1:

enter image description here

Where can I find an equivalent "frequentist" estimation procedure? Can they also be expressed in closed form?


1 "Machine Learning - A Probabilistic Perspective", 2012, pp. 119. Kevin Murphy.

$\endgroup$
3
$\begingroup$

It is unclear what you mean by frequentist estimation, but I'll take a stab.

This problem is regression with a known covariance matrix where $$ Y-b \sim N(Ax,\Sigma_y). $$

The only unknown here is $x$, so the maximum likelihood estimator is $$ \hat{x}_{MLE} = \text{argmax}_{x} p(y|x) = [A^T\Sigma_y^{-1}A]^{-1}A^T \Sigma_y^{-1}(y-b) $$ which is the posterior mode in the limit as $\Sigma_x^{-1}\to 0$, i.e. the improper uniform distribution.

The sampling distribution for $\hat{x}_{MLE}$ is normal with mean $$ [A^T\Sigma_y^{-1}A]^{-1}A^T \Sigma_y^{-1}A x $$ and covariance matrix $$ A[A^T\Sigma_y^{-1}A]^{-1}[A^T\Sigma_y^{-1}A]^{-1}A^T. $$ The sampling distribution can be used to construct confidence, i.e. frequentist, intervals.

$\endgroup$
  • $\begingroup$ Thanks @jaradniemi. This is great; right what I was looking for. Any particular papers or textbooks you recommend that cover sampling distributions of these types of systems? (e.g. I am interested on similar derivations for a mixture of Beta-Binomial variables). $\endgroup$ – Amelio Vazquez-Reina Feb 20 '16 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.