Are these Bayesian Inference?

Question

I am trying to understand what is and what is not considered Bayesian inference. Let say I am to estimate a parameter or a vector of parameters say $\theta$ and I have data on some features of the distribution say the first two moments $\hat{m_1}$ and $\hat{m_2}$. I think it is OK to use Bayes theorem to derive a sort of a posterior as follows

$p(\theta|\hat{m_1},\hat{m_2})\propto p(\hat{m_1},\hat{m_2}|\theta)p(\theta)$

where $p(\hat{m_1},\hat{m_2}|\theta)$ is the joint distribution of $\hat{m_1}$ and $\hat{m_1}$ conditional on $\theta$. Which one of these is not Bayesian inference?

1. use the actual distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula
2. use the Asymptotic distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula
3. How about the following when I have a frequentist estimate of $\theta$ and its distribution?

$p(\theta|\hat{\theta})\propto p(\hat{\theta}|\theta)p(\theta)$

Addendum: Here is an example to clarify the question: Suppose I want to learn about the parameters of the distribution of income given my data and let's assume it follows a parametric distribution with parameter $\theta$ . Often the data on income distribution is available only in the form of some summary statistics (e.g. deciles or Lorenz ordinates). for simplicity let say only 2 summary statistics of $\hat{m_1}$ and $\hat{m_2}$ are available. It is possible to obtain either the actual or more often asymptotic distribution of these summary statistics as a function of unknown $\theta$. Let's denote this with $p(\hat{m_1},\hat{m_2}|\theta)$. (1) and (2) above asks whether this can be used like a likelihood and proceed with the Bayesian inference. I personally think this is a legitimate Bayesian analysis and I have seen papers doing such things.

Now if the above is legitimate Bayesian inference and the objective of Bayesian analysis is to incorporate prior information then I can use the above logic and easily make any frequentist estimator Bayesian without MCMC or any high power methods. I can calculate $\hat{\theta}$ (the frequentist estimate of $\theta$) and the frequentist analysis often gives me the (asymptotic) distribution of $\hat{\theta}$ as a function of $\theta$ which I denote with $p(\hat{\theta}|\theta)$. I can combine this (often a normal distribution) with prior $p(\theta)$ and do a simple Bayesian analysis. This does not seem right but what is wrong with it or what is that I am missing?

Addendum2: The latter may not as simple as I thought since the variance of the asymptotic normal distribution could depend on $\theta$ in a complicated way.

Answer 1

Let me split the answer into two parts.

Moments in the likelihood considered as regular data

First, I'll make it more general, and instead of considering

$$ p(\theta|\hat{m_1},\hat{m_2})\propto p(\hat{m_1},\hat{m_2}|\theta)\,p(\theta) $$

I'll consider the general case

$$ p(\theta|X)\propto p(X|\theta) \,p(\theta) $$

where $X$ can be any data. If you can define the likelihood function in terms of the moments $\hat{m_1},\hat{m_2}$ as the data (same as data about $n$ i.i.d. Bernoulli trials can be simplified to considering it as a binomial random variable for the total number of successes), then it is the same. In many cases, however, it might be hard to come up with such a likelihood function.

1. use the actual distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula

This question is confusing. Are you talking about the likelihood $p(\hat{m_1},\hat{m_2}|\theta)$ or the unconditional distribution $p(\hat{m_1},\hat{m_2})$?

If it is the latter, you cannot put unconditional distribution in the place of the conditional distribution, they are different things. It is hard to discuss it as a general case because it is not clear what you mean here and how exactly it refers to the initial problem. Say that you are interested only in the mode of the posterior (maximum a posterior estimation), so you are just maximizing the posterior probability, $p(\hat{m_1},\hat{m_2})$ cannot be maximized with regard of $\theta$ because it is not a function of $\theta$.

If you meant conditional distribution then lucky you! Usually, when deciding on a statistical model, you need to somehow come up with what the likelihood function should be for the problem. If you know it, you don't have to make this step.

2. use the Asymptotic distribution of $p(\hat{m_1},\hat{m_2})$ in the above formula

Same as above. Unusually we don't "know" the likelihood function, but make some assumptions.

3. How about the following when I have a frequentist estimate of $\theta$ and its distribution?

First, in the frequentist approach, you would not have the estimate for the distribution of the parameter, because the parameter is not considered a random variable. You would have the point estimate. But answering the question, deciding on priors based on in-data statistics is called empirical Bayesian approach, it is controversial, but sometimes used.

Can we use moments in the likelihood function?

In some cases, it might be hard to come up with a likelihood function, but instead, the distribution can be characterized in other terms. In such cases, we can use likelihood-free inference and things like approximate Bayesian computation (ABC) (see also other threads for approximate-bayesian-computation). Your problem with using only the moments rather than the data might fall into this category. In ABC, the likelihood is replaced with a distance function that serves as a proxy for it.

Finally, "what is Bayesian inference?" is a variant of the Who Are The Bayesians? question that already got some nice answers. As you can learn from the linked thread, Bayesian statistics is not about using the Bayes theorem, but is more general than this. That is how ABC falls into this category, though it does not exactly use the Bayes theorem, just takes from the general idea behind it.