Questions tagged [map-estimation]

Estimation by maximizing the posterior density function

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Bayesian Inference Intuition: Beta and Binomial vs Gamma and Poisson

When the data is assumed to be binomial distributed, and the prior probability is assumed to be a beta distribution, the posterior follows the distribution $Beta (\alpha - 1 + k , \beta - 1 + n- k $). ...
0
votes
1answer
12 views

MLE and MAP with Naive Bayes

From what I understand, Naive Bayes classifies by doing: $$ y \leftarrow argmax_{y_k}P(Y=y_k)\prod_{i}P(X_i|Y=y_k) $$ There are two things there we need to know: $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$ ...
1
vote
0answers
18 views

Bayesian inference MAP question

Hello I am stuck with this problem and was wondering if someone can solve it for me. Thanks
2
votes
1answer
26 views

MAP: estimate 2 parameters

I have some data x and I want to estimate the mu and sigma of this data according to model $x \sim N(\mu, \sigma)$ where I have priors $\mu \sim N(0, 1)$ and $\sigma \sim \Gamma(1, 1)$. Assume $\theta ...
2
votes
1answer
37 views

Deriving the posterior distribution over the model parameters: are the model parameters and data independent?

We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
0
votes
2answers
54 views

The prior in MAP and Bayesian interference

We can use a Normal distribution as a prior when handling a Normal distribution as likelihood in Bayesian inference However if we want to do MAP given a Bernoulli as likelihood can we use Normal ...
3
votes
1answer
46 views

Does it make sense to condition on fixed values of some parameters before doing MCMC on the other parameters?

I have a Bayesian model with a large number of parameters (around 50), and as usual my goal is to infer the posterior distribution for the parameters, with MCMC. However, I am only interested in the ...
1
vote
1answer
117 views

Shouldn't log likelihood always be normalized by data size in bayesian estimation?

This is very interesting problem. I wonder if the whole bayesian statistics is neglecting it or if I am super confused. I will illustrate it on a bayesian Maximum a Posteriori (MAP) estimation and ...
1
vote
0answers
16 views

Finding the MAP for a function whose conditioning depends on an exponential integral

Let $X$ be such that $X \sim exp( \lambda = 1)$ and let $Y$ be such that $Y \sim U[0,x]$, where $x$ is the realization of $X$. Given that information I know that: $f_{X}(x) = e^{-x}$ for $x \geq 0$...
0
votes
0answers
45 views

MAP Estimation of Covariance Matrix of a Multivariate Normal Distribution

I have a general prior multivariate normal distribution and I want to update it with new samples which are more local for my case. I want to do it with MAP estimation. With MAP estimation it is ...
0
votes
0answers
18 views

Practical Estimation of Maximum a Posteriori Values

Suppose I have samples from a posterior density function, generated using MCMC methods. I wish to summarize the posterior marginally for any given parameter in the model (of which there may be many) - ...
1
vote
1answer
21 views

MAP, MLE and parametrised data

It is often said that maximum likelihood is used to obtain estimates of distrubtion's parameters. However, what is unclear is whether it will produce consistent estimate parameters other than those of ...
3
votes
0answers
103 views

Bias correcting penalized maximum likelihood / maximum a posteriori estimates

Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing: $$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$ where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\...
0
votes
0answers
53 views

Naive Bayes - why MAP?

According to Wikipedia on (Classic) Naive Bayes The naive Bayes classifier combines this model with a decision rule. One common rule is to pick the hypothesis that is most probable; this is known ...
1
vote
0answers
24 views

Logistic regression fitting methods clarification

Each book I read propose a different fitting method for Logistic Regression. The general idea is to maximize this expression. $$ Pr\left(\beta|y,X,M\right) = \frac{Pr\left(y|\beta,X,M\right) Pr(\...
1
vote
1answer
30 views

reference request for the impact of priors in bayesian statistics

It is well known that in bayesian statistics, the prior believe can have a large impact on the estimation result. For example if you flip a coin ten times to determine whether it is loaded, a prior $...
1
vote
1answer
26 views

Assign an error to the parameters of MAP estimate

Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration. ...
1
vote
0answers
37 views

Discriminative Models with Class Priors

In discriminative models, we model $p(Y|X)$ directly while in generative models we model $p(X|Y)p(Y)$ where $X$ is the input and $Y$ is the output variable. I am confused when the parameters and ...
3
votes
3answers
196 views

Bayesian parameter estimation with proportion data

I am trying to do a Bayesian analysis using a model that comes from the literature in non-Bayesian form: $y = \Phi\Bigg(\frac{1}{\alpha} * log(A/\beta)\Bigg)$. Because the model uses the function $\...
0
votes
0answers
116 views

Can you find the posterior mode of an unknown distribution without MCMC?

I was wondering if you wanted to compute the MAP estimate of an unknown posterior distribution, is there a non-sampling based method that would suffice? As in, if you don’t need to know anything more ...
0
votes
0answers
29 views

How to choose estimates after Bayesian regression?

In a Bayesian logistic regression with two predictor variables $x_{1}$ and $x_{2}$, I did MCMC (2000 samples) to estimate posterior distribution. Now it's done, how can I choose the final estimates ...
0
votes
1answer
62 views

Finding MAP estimate

I think after all the reading I've done I still don't fully understand MAP estimation. I came across a problem that's leaving me dumbfounded. Suppose $A$ ~ $N(0,\sigma^2_1) $ and $\epsilon$ ~ $N(0,\...
1
vote
1answer
156 views

What are the possible estimates of the parameters of the multinomial distribution?

The expected value of the parameters of the multinomial distribution (taking into account the Dirichlet prior $D(\alpha)$ and the posterior Dirichlet-Multinomial) is: $\pi_i = α_i+ x_i / \sum_{j} α_j+...
0
votes
1answer
88 views

Example of maximum a posteriori that does not match the mean of a marginalized posterior

Given a N-parameter likelihood and prior, I can obtain the marginalized posterior for each parameter through Bayesian MCMC. I can also estimate the maximum a posteriori (MAP) of the N-parameter ...
0
votes
1answer
52 views

Confused about maximum a posteriori estimation [closed]

I am new to Bayesian statistics, and I just came across MAP. When our prior is a continuous distribution (pdf) on $\theta$ how can we calculate $g(\theta)$ in the numerator? Edit: I assumed $g(\...
2
votes
1answer
61 views

Multidimensional Bayes point estimates

Consider the posterior distribution $p(\theta|x)$. We aim to find a "good" estimate of the random variable $\theta$. The Bayes risk associated with the loss function $L(\hat{\theta}, \theta)$ is ...
10
votes
3answers
543 views

MAP is a solution to $L(\theta) = \mathcal{I}[\theta \ne \theta^{*}]$

I have come across these slides (slide # 16 & #17) in one of the online courses. The instructor was trying to explain how Maximum Posterior Estimate(MAP) is actually the solution $L(\theta) = \...
1
vote
1answer
757 views

How can (L1 / L2) regularization be equivalent to using a prior when priors can't be changed?

I understand the argument for how training with an L1/L2 regularizer is the same thing as finding the MAP estimate when the prior is Gaussian/Laplace. But there's a crucial difference. In Bayes' ...
1
vote
1answer
493 views

MAP estimation for multiple parameters

Consider $N$ observed data points $x_i$ ($i=1,..,N$), and a likelihood that depends on $p$ parameters: $f(x_i|\theta_n)$ ($n=1,..p$). From Bayes' theorem $$p(\theta_n|x_i) = \frac{f(x_i|\theta_n)g(\...
3
votes
2answers
808 views

What is an example of a transformation on a posterior distribution such that the MAP estimate will be non-invariant?

Suppose that we have a posterior distribution $p(\theta\mid y)$ and we wish to define a transformation on $\theta$ such that $\phi = f(\theta)$. I know that generally such transformations will not ...
0
votes
1answer
189 views

For a posterior $p(\theta |y)$, if I specify a one-to-one transformation $\phi = g(\theta)$, how can I apply the transformation? [duplicate]

Suppose I have a posterior distribution, $p(\theta \mid y)$, where $y$ was my data and $\theta$ is a random variable with some prior distribution. If I specify a one-to-one transformation $\phi = g(\...
3
votes
1answer
107 views

The explanation for the need to compute (rather then optimize) the posterior of latent variables

The most common usage of the variational inference looks like to be in computing the marginal distribution $P(X)$ in the denominator of the Bayes formula when computing the posterior probability of ...
1
vote
0answers
208 views

confusion related to maximum a posteriori estimation [duplicate]

I was reading this article in wikipedia related to MAP http://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation. However, I had this confusion when it says MAP estimation is a limit of Bayes ...