Skip to main content

Questions tagged [map-estimation]

Estimation by maximizing the posterior density function

Filter by
Sorted by
Tagged with
3 votes
1 answer
219 views

Basic question about deriving MAP estimator

Say we have a random process $X(t, u)$ parametrized by $t$ and $u$ that generates data $x$. We also have a prior on $u$, $p(u)$. Am I correct in stating that the expression to find the maximum a ...
DangerousTim's user avatar
0 votes
0 answers
15 views

How are the MLE/MAP distinction and the generative/discriminative distinction related?

What is the relationship between Maximum Likelihood Estimation versus Maximum A Posteriori Estimation and generative modeling versus discriminative modeling? Is MLE an example of a generative model ...
user10478's user avatar
  • 123
3 votes
0 answers
283 views

Minimum Description Length, Normalized Maximum Likelihood, and Maximum A Posteriori Estimation

TL;DR: I believe MDL using NML is a special case of the joint MAP of model and parameters, and need to verify this and find sources that have acknowledges this. This is how I understand Minimum ...
Feri's user avatar
  • 197
1 vote
0 answers
21 views

MAP when the prior of the parameter is defined piecewise

As defined on Wikipedia, $$\hat{\theta}_{MAP} = \underset{\theta}{\mathrm{argmax}} f(x | \theta) g(\theta)$$ Then, to actually obtain theta-hat-MAP, we could set the derivative of the above (or their ...
sanssouci's user avatar
3 votes
0 answers
42 views

Does the mode of MCMC samples equal the MAP of the posterior?

If I had millions of MCMC samples from a posterior, should the most frequent value among those samples (i.e., the peak of a histogram of those samples) at least in principle always equal the maximum-a-...
Durden's user avatar
  • 1,352
1 vote
0 answers
39 views

Laplace approximation, MAP vs MLE and wiki's notations

I was trying to understand Laplace approximation in statistics and so I was going through the wikipedia article. I don't know much about statistics and I am already getting a bit confused by the ...
roi_saumon's user avatar
2 votes
1 answer
95 views

For discrete $X_n$ if $X_n \stackrel{d}{\to} U$, does $P_e \to 1$ where $P_e$ is optimal Bayes error and $U$ is uniform

Consider the following setting. Let $ \{X_n\}_{n=1}^\infty \subseteq [-1,1] $ be a discrete random variables that converges to a continuous uniform random variable in distribution. Let $Y_n = X_n +...
Boby's user avatar
  • 195
2 votes
2 answers
223 views

MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula

I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
ummg's user avatar
  • 145
0 votes
0 answers
90 views

Estimating the parameter of a Bernoulli distribution using probabilistic modeling and the MAP estimation

Suppose you tossed a coin multiple times. Sometimes you got heads and other times you got tails. You recorded your experiment in a dataset $ X$. Now you want to estimate the parameter θ (which ...
Mosab Shaheen's user avatar
2 votes
1 answer
176 views

Why the prophet time series model uses MAP and not MLE?

I'm using prophet model for one of my time series analysis. I learnt that it optimizes the parameters by MAP approach. The fundamental idea of when to use MAP vs MLE is that when we have a strong ...
Patrick Priyadharshan's user avatar
2 votes
1 answer
64 views

How can I pool Bayesian parameter estimates after multiple imputation?

After multiple imputation (imputed dataset = 20), I would like to conduct Bayesian Model Estimation with Adaptive Metropolis Hastings Sampling (amh) -- using the MCMC method. How can I pool the ...
conner's user avatar
  • 21
7 votes
2 answers
617 views

Why does the MAP differ from the MLE for the uniform prior in Laplace's Rule?

Laplace's Rule of Succession produces an estimate for the probability $p$ of a Bernoulli distribution. It starts with a $Beta(1,1)$ prior (equivalent to a uniform distribution prior on $(0,1)$), and ...
Legendre's user avatar
  • 217
1 vote
1 answer
34 views

Repeated Bayesian inference to track a time-varying parameter online

I have trouble finding the name of the problem (and algorithms to solve it) where one needs to repeatedly estimate the value of a continuous, time-varying parameter online based on incoming ...
user15546's user avatar
0 votes
1 answer
253 views

Is the posterior maximum always the same as the marginal's?

When I see plots of the conditional probabilities, marginals and joint distributions together they are mostly plot using Gaussians. It is not clear to me if this applies to every other distribution. ...
ElPotac's user avatar
  • 61
3 votes
0 answers
110 views

For log-concave densities, are joint and marginal modes consistent?

Suppose I have a probability density function $\pi(x_1, \ldots, x_n)$, which is the density of a vector-valued random variable $X$ in $\mathbb{R}^n$. Assume that $\pi$ is strongly log-concave, i.e., ...
Jonathan Lindbloom's user avatar
1 vote
1 answer
199 views

Does the beta negative binomial (BNB) distribution have a conjugate prior?

BNB distribution is constructed using negative binomial and beta distributions, which are both exponential family, so my guess would be yes, there shoudl exist a conjugate prior in theory. But what is ...
user1747134's user avatar
2 votes
0 answers
33 views

Estimating parameters for a set of related random variables

Suppose I have some random variables $$X_i \sim Dist(\theta_i)$$ for $i = 1, ..., n$ where $Dist$ is some known probability distribution family and $\theta_i$ are some parameters which may vary ...
user1747134's user avatar
1 vote
1 answer
525 views

MLE ≠ MAP under Gaussian Prior?

I saw a post on why MLE and MAP yield the same result when under uniform prior. But, I was wondering about the case when they are under Gaussian Prior. I suppose they are different in this case but I ...
jimmy1998's user avatar
1 vote
1 answer
130 views

Is MAP a point estimator? Why?

In Bishop's Pattern Recognition and Machine Learning book, page 30, section 1.2.6, it says that the MAP estimator for the curve fitting model derived in section 1.2.5 is a point estimator of the model ...
vicky's user avatar
  • 13
0 votes
1 answer
42 views

Bayesian statistics

Assuming I have that $Y_i\mid \mu$ is an iid ~ $N(\mu,\sigma^2)$, for $i \in (1,\dotsc,n)$ with $\sigma_i$ known and improper prior $\pi(\mu)=1$ for all $\mu$. i. How can I derive a formula for the ...
user354604's user avatar
0 votes
0 answers
59 views

MAP estimation with hierarchical prior

I have a hierarchical prior with the probabilistic model as $$p(Y,H,Z|G)=p(Y|H,Z)p(H|Z,G)p(Z|G)$$ and I was wondering if MAP estimation for H and Z together is possible here by maximizing the log ...
NKJ's user avatar
  • 1
2 votes
2 answers
492 views

Weighted likelihood in Bayes' rule

This question is about whether we can treat a weighted likelihood as a likelihood inside Bayes' rule. Let $\theta$ be a parameter and let $x$ be some observed random variable. Given a prior over $\...
Student's user avatar
  • 241
2 votes
1 answer
40 views

Bayes prior in MAP estimation corresponding to $\ell^0$ penalization

I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$ Is this also true for $\ell^0$ ...
Golden_Ratio's user avatar
7 votes
2 answers
245 views

How do we derive the conditional mode as the solution to linear regression, for uniform cost function?

I know that if the cost functions are respectively the least squares ($L^2$) and the absolute deviation ($L^1$), the solution to linear regression is the conditional mean and the conditional median ...
Bravo's user avatar
  • 629
3 votes
1 answer
417 views

Why maximum a posterior, not maximum posterior?

Is the additional "a" mean that different priors may lead to different posterior, MAP is a result of many possible results? And similar to MLE, why the abbreviation of maximum a posterior ...
Alex's user avatar
  • 93
2 votes
1 answer
290 views

Computing the Gaussian posterior from likelihood and prior

Say I have a gaussian likelihood and prior, $$ p(\theta) = \mathcal{N}(\theta|\theta_0, \Sigma_\theta) $$ $$ p(y|\theta) = \mathcal{N}(y| \Phi \theta, \Sigma_\eta) $$ I would like to compute the ...
DarkLink's user avatar
  • 217
4 votes
1 answer
589 views

How to prove that the posterior of the regression coefficients $\mathbf{w}$ is roughly gaussian in MAP regularized logistic regression?

The logistic regression model is $$ p(y=\pm 1 \mid \mathbf{x}, \mathbf{w})=\sigma\left(y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)=\frac{1}{1+\exp \left(-y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)} $$...
user avatar
3 votes
1 answer
638 views

Bayesian MAP Estimates

Suppose you have a simple linear regression problem (y = bo + b1x) and you decide to use Bayesian Estimation to estimate the value pf bo and b1. Using Bayesian Estimation, you obtain a list of ...
stats_noob's user avatar
2 votes
1 answer
607 views

Parameters in Naive Bayes

This is from https://scikit-learn.org/stable/modules/naive_bayes.html In the last line it says and we can use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i|y)$; the former is ...
Esha's user avatar
  • 21
0 votes
1 answer
53 views

Where does the denominator vanish to in the MAP derivation?

According to MAP estimator: $$\hat\theta_\text{MAP}=\arg\max_\theta P(\theta|D) = \arg\max_\theta \frac{P(D|\theta)P(\theta)}{P(D)}=\arg\max_\theta {P(D|\theta)P(\theta)} $$ The denominator $P(D)$ ...
user3668129's user avatar
1 vote
0 answers
104 views

Probabilty estimation for Bernoulli with number of trials as random variable

Problem description Suppose we have fixed number of people that are the test population, let's say $t=200$ persons. For each one of them $\mathbf{r}_j$ we know about $m=300$ features that describes ...
Alex's user avatar
  • 11
1 vote
0 answers
199 views

Is there a reason to use variational inference for point estimates?

I have seen Bayesian hierarchical models, particularly in computational biology, that use variational inference, but do not use the uncertainty provided by a variational solution. For example, MOFA is ...
alan ocallaghan's user avatar
2 votes
1 answer
143 views

MCMC with using MAP as starting value

let $X$ be a random variable from my target distribution $\pi(x)$, which I know up to a normalizing constant, and I want to calculate $Ef(X)$ for some know function $f$. The dimensions of $X$ are ...
user1292919's user avatar
5 votes
1 answer
269 views

Why should MAP be invariant under reparameterization?

I learned why MAP suffers from being reparametrization invariance while MLE not from this answer, but I don't know why reparametrization invariance even matters? What is the non-linear mapping ...
Lerner Zhang's user avatar
  • 6,810
3 votes
1 answer
84 views

Why is MAP and ML widely accepted? [closed]

(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori) I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
Cookie's user avatar
  • 143
4 votes
3 answers
9k views

Differences between MLE and MAP estimators

Generally speaking, what are the differences between an MLE and a MAP estimator? If I wanted to improve the performance of a model, how would these differences come into play? Are there specific ...
MLNewbie's user avatar
  • 141
1 vote
0 answers
38 views

Quantifying uncertainty in MAP nonlinear regression

I am interested in clinical pharmacokinetics, where we have a given medicine's population model, with its parameters (mean and standard deviation), and our goal is take one or two blood samples from a ...
raululm's user avatar
  • 11
2 votes
1 answer
2k views

MLE and MAP with Naive Bayes

From what I understand, Naive Bayes classifies by doing: $$ y \leftarrow argmax_{y_k}P(Y=y_k)\prod_{i}P(X_i|Y=y_k) $$ There are two things there we need to know: $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$ ...
user3629892's user avatar
1 vote
0 answers
30 views

Bayesian inference MAP question

Hello I am stuck with this problem and was wondering if someone can solve it for me. Thanks
James Dean's user avatar
2 votes
1 answer
95 views

MAP: estimate 2 parameters

I have some data x and I want to estimate the mu and sigma of this data according to model $x \sim N(\mu, \sigma)$ where I have priors $\mu \sim N(0, 1)$ and $\sigma \sim \Gamma(1, 1)$. Assume $\theta ...
Dirk N's user avatar
  • 305
2 votes
1 answer
125 views

Deriving the posterior distribution over the model parameters: are the model parameters and data independent?

We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
orthonormal-stice's user avatar
0 votes
2 answers
352 views

The prior in MAP and Bayesian inference

We can use a Normal distribution as a prior when handling a Normal distribution as likelihood in Bayesian inference However if we want to do MAP given a Bernoulli as likelihood can we use Normal ...
Linear Algebra fans's user avatar
3 votes
1 answer
85 views

Does it make sense to condition on fixed values of some parameters before doing MCMC on the other parameters?

I have a Bayesian model with a large number of parameters (around 50), and as usual my goal is to infer the posterior distribution for the parameters, with MCMC. However, I am only interested in the ...
segej733's user avatar
2 votes
1 answer
1k views

Shouldn't log likelihood always be normalized by data size in bayesian estimation?

This is very interesting problem. I wonder if the whole bayesian statistics is neglecting it or if I am super confused. I will illustrate it on a bayesian Maximum a Posteriori (MAP) estimation and ...
Tomas's user avatar
  • 6,211
1 vote
0 answers
23 views

Finding the MAP for a function whose conditioning depends on an exponential integral

Let $X$ be such that $X \sim exp( \lambda = 1)$ and let $Y$ be such that $Y \sim U[0,x]$, where $x$ is the realization of $X$. Given that information I know that: $f_{X}(x) = e^{-x}$ for $x \geq 0$...
Jxson99's user avatar
  • 664
0 votes
1 answer
29 views

MAP, MLE and parametrised data

It is often said that maximum likelihood is used to obtain estimates of distrubtion's parameters. However, what is unclear is whether it will produce consistent estimate parameters other than those of ...
Gidefi's user avatar
  • 9
4 votes
0 answers
213 views

Bias correcting penalized maximum likelihood / maximum a posteriori estimates

Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing: $$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$ where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\...
cfp's user avatar
  • 535
1 vote
0 answers
36 views

Logistic regression fitting methods clarification

Each book I read propose a different fitting method for Logistic Regression. The general idea is to maximize this expression. $$ Pr\left(\beta|y,X,M\right) = \frac{Pr\left(y|\beta,X,M\right) Pr(\...
Francesco Boi's user avatar
1 vote
1 answer
94 views

reference request for the impact of priors in bayesian statistics

It is well known that in bayesian statistics, the prior believe can have a large impact on the estimation result. For example if you flip a coin ten times to determine whether it is loaded, a prior $...
safex's user avatar
  • 171
1 vote
1 answer
121 views

Assign an error to the parameters of MAP estimate

Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration. ...
Johnpiton's user avatar