Skip to main content

# Questions tagged [map-estimation]

Estimation by maximizing the posterior density function

66 questions
Filter by
Sorted by
Tagged with
3 votes
1 answer
219 views

### Basic question about deriving MAP estimator

Say we have a random process $X(t, u)$ parametrized by $t$ and $u$ that generates data $x$. We also have a prior on $u$, $p(u)$. Am I correct in stating that the expression to find the maximum a ...
0 votes
0 answers
15 views

### How are the MLE/MAP distinction and the generative/discriminative distinction related?

What is the relationship between Maximum Likelihood Estimation versus Maximum A Posteriori Estimation and generative modeling versus discriminative modeling? Is MLE an example of a generative model ...
• 123
3 votes
0 answers
283 views

### Minimum Description Length, Normalized Maximum Likelihood, and Maximum A Posteriori Estimation

TL;DR: I believe MDL using NML is a special case of the joint MAP of model and parameters, and need to verify this and find sources that have acknowledges this. This is how I understand Minimum ...
• 197
1 vote
0 answers
21 views

### MAP when the prior of the parameter is defined piecewise

As defined on Wikipedia, $$\hat{\theta}_{MAP} = \underset{\theta}{\mathrm{argmax}} f(x | \theta) g(\theta)$$ Then, to actually obtain theta-hat-MAP, we could set the derivative of the above (or their ...
3 votes
0 answers
42 views

### Does the mode of MCMC samples equal the MAP of the posterior?

If I had millions of MCMC samples from a posterior, should the most frequent value among those samples (i.e., the peak of a histogram of those samples) at least in principle always equal the maximum-a-...
• 1,352
1 vote
0 answers
39 views

### Laplace approximation, MAP vs MLE and wiki's notations

I was trying to understand Laplace approximation in statistics and so I was going through the wikipedia article. I don't know much about statistics and I am already getting a bit confused by the ...
• 111
2 votes
1 answer
95 views

• 241
2 votes
1 answer
40 views

### Bayes prior in MAP estimation corresponding to $\ell^0$ penalization

I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$ Is this also true for $\ell^0$ ...
• 543
7 votes
2 answers
245 views

### How do we derive the conditional mode as the solution to linear regression, for uniform cost function?

I know that if the cost functions are respectively the least squares ($L^2$) and the absolute deviation ($L^1$), the solution to linear regression is the conditional mean and the conditional median ...
• 629
3 votes
1 answer
417 views

### Why maximum a posterior, not maximum posterior?

Is the additional "a" mean that different priors may lead to different posterior, MAP is a result of many possible results? And similar to MLE, why the abbreviation of maximum a posterior ...
• 93
2 votes
1 answer
290 views

### Computing the Gaussian posterior from likelihood and prior

Say I have a gaussian likelihood and prior, $$p(\theta) = \mathcal{N}(\theta|\theta_0, \Sigma_\theta)$$ $$p(y|\theta) = \mathcal{N}(y| \Phi \theta, \Sigma_\eta)$$ I would like to compute the ...
• 217
4 votes
1 answer
589 views

### How to prove that the posterior of the regression coefficients $\mathbf{w}$ is roughly gaussian in MAP regularized logistic regression?

The logistic regression model is $$p(y=\pm 1 \mid \mathbf{x}, \mathbf{w})=\sigma\left(y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)=\frac{1}{1+\exp \left(-y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)}$$...
3 votes
1 answer
638 views

### Bayesian MAP Estimates

Suppose you have a simple linear regression problem (y = bo + b1x) and you decide to use Bayesian Estimation to estimate the value pf bo and b1. Using Bayesian Estimation, you obtain a list of ...
2 votes
1 answer
607 views

### Parameters in Naive Bayes

This is from https://scikit-learn.org/stable/modules/naive_bayes.html In the last line it says and we can use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i|y)$; the former is ...
• 21
0 votes
1 answer
53 views

### Where does the denominator vanish to in the MAP derivation?

According to MAP estimator: $$\hat\theta_\text{MAP}=\arg\max_\theta P(\theta|D) = \arg\max_\theta \frac{P(D|\theta)P(\theta)}{P(D)}=\arg\max_\theta {P(D|\theta)P(\theta)}$$ The denominator $P(D)$ ...
• 125
1 vote
0 answers
104 views

### Probabilty estimation for Bernoulli with number of trials as random variable

Problem description Suppose we have fixed number of people that are the test population, let's say $t=200$ persons. For each one of them $\mathbf{r}_j$ we know about $m=300$ features that describes ...
• 11
1 vote
0 answers
199 views

### Is there a reason to use variational inference for point estimates?

I have seen Bayesian hierarchical models, particularly in computational biology, that use variational inference, but do not use the uncertainty provided by a variational solution. For example, MOFA is ...
• 1,228
2 votes
1 answer
143 views

### MCMC with using MAP as starting value

let $X$ be a random variable from my target distribution $\pi(x)$, which I know up to a normalizing constant, and I want to calculate $Ef(X)$ for some know function $f$. The dimensions of $X$ are ...
• 657
5 votes
1 answer
269 views

### Why should MAP be invariant under reparameterization?

I learned why MAP suffers from being reparametrization invariance while MLE not from this answer, but I don't know why reparametrization invariance even matters? What is the non-linear mapping ...
• 6,810
3 votes
1 answer
84 views

### Why is MAP and ML widely accepted? [closed]

(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori) I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
• 143
4 votes
3 answers
9k views

### Differences between MLE and MAP estimators

Generally speaking, what are the differences between an MLE and a MAP estimator? If I wanted to improve the performance of a model, how would these differences come into play? Are there specific ...
• 141
1 vote
0 answers
38 views

### Quantifying uncertainty in MAP nonlinear regression

I am interested in clinical pharmacokinetics, where we have a given medicine's population model, with its parameters (mean and standard deviation), and our goal is take one or two blood samples from a ...
• 11
2 votes
1 answer
2k views

### MLE and MAP with Naive Bayes

From what I understand, Naive Bayes classifies by doing: $$y \leftarrow argmax_{y_k}P(Y=y_k)\prod_{i}P(X_i|Y=y_k)$$ There are two things there we need to know: $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$ ...
• 183
1 vote
0 answers
30 views

### Bayesian inference MAP question

Hello I am stuck with this problem and was wondering if someone can solve it for me. Thanks
2 votes
1 answer
95 views

• 535
1 vote
0 answers
36 views

### Logistic regression fitting methods clarification

Each book I read propose a different fitting method for Logistic Regression. The general idea is to maximize this expression.  Pr\left(\beta|y,X,M\right) = \frac{Pr\left(y|\beta,X,M\right) Pr(\...
1 vote
1 answer
94 views