Questions tagged [map-estimation]
Estimation by maximizing the posterior density function
58
questions
7
votes
2
answers
464
views
Why does the MAP differ from the MLE for the uniform prior in Laplace's Rule?
Laplace's Rule of Succession produces an estimate for the probability $p$ of a Bernoulli distribution. It starts with a $Beta(1,1)$ prior (equivalent to a uniform distribution prior on $(0,1)$), and ...
- 217
0
votes
0
answers
17
views
Is it ok to just use Add One Smoothing for any sufficient statistics on Counts/Probabilities?
I know that Add-one smoothing is due to $$
\theta_{\text{MAP}}=\arg\max_\theta \log(P(\theta|D))
$$ when the posterior is a Binomial/Bernoulli with a $\text{Beta}(2,2)$ prior.
Now, I am implementing ...
- 581
1
vote
1
answer
30
views
Repeated Bayesian inference to track a time-varying parameter online
I have trouble finding the name of the problem (and algorithms to solve it) where one needs to repeatedly estimate the value of a continuous, time-varying parameter online based on incoming ...
- 31
0
votes
1
answer
57
views
Is the posterior maximum always the same as the marginal's?
When I see plots of the conditional probabilities, marginals and joint distributions together they are mostly plot using Gaussians. It is not clear to me if this applies to every other distribution. ...
- 31
2
votes
0
answers
79
views
For log-concave densities, are joint and marginal modes consistent?
Suppose I have a probability density function $\pi(x_1, \ldots, x_n)$, which is the density of a vector-valued random variable $X$ in $\mathbb{R}^n$. Assume that $\pi$ is strongly log-concave, i.e., ...
0
votes
2
answers
90
views
Does the beta negative binomial (BNB) distribution have a conjugate prior?
BNB distribution is constructed using negative binomial and beta distributions, which are both exponential family, so my guess would be yes, there shoudl exist a conjugate prior in theory. But what is ...
- 143
2
votes
0
answers
28
views
Estimating parameters for a set of related random variables
Suppose I have some random variables
$$X_i \sim Dist(\theta_i)$$
for $i = 1, ..., n$ where $Dist$ is some known probability distribution family and $\theta_i$ are some parameters which may vary ...
- 143
1
vote
1
answer
189
views
MLE ≠ MAP under Gaussian Prior?
I saw a post on why MLE and MAP yield the same result when under uniform prior. But, I was wondering about the case when they are under Gaussian Prior. I suppose they are different in this case but I ...
- 31
1
vote
1
answer
69
views
Is MAP a point estimator? Why?
In Bishop's Pattern Recognition and Machine Learning book, page 30, section 1.2.6, it says that the MAP estimator for the curve fitting model derived in section 1.2.5 is a point estimator of the model ...
- 13
0
votes
0
answers
55
views
Hessian of log marginal likelihood is rank deficient
I am implementing the Bayesian MAP estimation algorithm presented in [1, Sec. 4.4]. More specifically, I am trying to estimate the hyper-parameters of the main MAP estimator by optimizing the Bayes ...
- 385
1
vote
0
answers
26
views
Is there an analog of Lehman-Scheffe theorem for Bayes/MAP/biased estimators?
In statistics, the Lehmann-Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. ${ }^{[1]}$ The theorem states ...
user318514
0
votes
1
answer
41
views
Bayesian statistics
Assuming I have that $Y_i\mid \mu$ is an iid ~ $N(\mu,\sigma^2)$, for $i \in (1,\dotsc,n)$ with $\sigma_i$ known and improper prior $\pi(\mu)=1$ for all $\mu$.
i. How can I derive a formula for the ...
0
votes
0
answers
34
views
MAP estimation with hierarchical prior
I have a hierarchical prior with the probabilistic model as
$$p(Y,H,Z|G)=p(Y|H,Z)p(H|Z,G)p(Z|G)$$
and I was wondering if MAP estimation for H and Z together is possible here by maximizing the log ...
- 1
1
vote
0
answers
224
views
Weighted likelihood in Bayes' rule
This question is about whether we can treat a weighted likelihood as a likelihood inside Bayes' rule.
Let $\theta$ be a parameter and let $x$ be some observed random variable. Given a prior over $\...
- 221
2
votes
1
answer
35
views
Bayes prior in MAP estimation corresponding to $\ell^0$ penalization
I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$
Is this also true for $\ell^0$ ...
- 503
6
votes
1
answer
158
views
How do we derive the conditional mode as the solution to linear regression, for uniform cost function?
I know that if the cost functions are respectively the least squares ($L^2$) and the absolute deviation ($L^1$), the solution to linear regression is the conditional mean and the conditional median ...
- 597
2
votes
1
answer
381
views
Why maximum a posterior, not maximum posterior?
Is the additional "a" mean that different priors may lead to different posterior, MAP is a result of many possible results? And similar to MLE, why the abbreviation of maximum a posterior ...
- 31
2
votes
1
answer
182
views
Computing the Gaussian posterior from likelihood and prior
Say I have a gaussian likelihood and prior,
$$
p(\theta) = \mathcal{N}(\theta|\theta_0, \Sigma_\theta)
$$
$$
p(y|\theta) = \mathcal{N}(y| \Phi \theta, \Sigma_\eta)
$$
I would like to compute the ...
- 217
4
votes
1
answer
454
views
How to prove that the posterior of the regression coefficients $\mathbf{w}$ is roughly gaussian in MAP regularized logistic regression?
The logistic regression model is
$$
p(y=\pm 1 \mid \mathbf{x}, \mathbf{w})=\sigma\left(y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)=\frac{1}{1+\exp \left(-y \mathbf{w}^{\mathrm{T}} \mathbf{x}\right)}
$$...
user318514
3
votes
1
answer
459
views
Bayesian MAP Estimates
Suppose you have a simple linear regression problem (y = bo + b1x) and you decide to use Bayesian Estimation to estimate the value pf bo and b1.
Using Bayesian Estimation, you obtain a list of ...
- 7,126
2
votes
1
answer
403
views
Parameters in Naive Bayes
This is from https://scikit-learn.org/stable/modules/naive_bayes.html
In the last line it says
and we can use Maximum A Posteriori (MAP) estimation to estimate
$P(y)$ and $P(x_i|y)$; the former is ...
- 21
0
votes
1
answer
50
views
Where does the denominator vanish to in the MAP derivation?
According to MAP estimator:
$$\hat\theta_\text{MAP}=\arg\max_\theta P(\theta|D) = \arg\max_\theta \frac{P(D|\theta)P(\theta)}{P(D)}=\arg\max_\theta {P(D|\theta)P(\theta)} $$
The denominator $P(D)$ ...
- 125
1
vote
0
answers
90
views
Probabilty estimation for Bernoulli with number of trials as random variable
Problem description
Suppose we have fixed number of people that are the test population, let's say $t=200$ persons. For each one of them $\mathbf{r}_j$ we know about $m=300$ features that describes ...
- 11
1
vote
0
answers
169
views
Is there a reason to use variational inference for point estimates?
I have seen Bayesian hierarchical models, particularly in computational biology, that use variational inference, but do not use the uncertainty provided by a variational solution. For example, MOFA is ...
- 1,178
2
votes
1
answer
88
views
MCMC with using MAP as starting value
let $X$ be a random variable from my target distribution $\pi(x)$, which I know up to a normalizing constant, and I want to calculate $Ef(X)$ for some know function $f$. The dimensions of $X$ are ...
- 637
4
votes
1
answer
189
views
Why should MAP be invariant under reparameterization?
I learned why MAP suffers from being reparametrization invariance while MLE not from this answer, but I don't know why reparametrization invariance even matters? What is the non-linear mapping ...
- 5,948
3
votes
1
answer
76
views
Why is MAP and ML widely accepted? [closed]
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
- 143
4
votes
3
answers
4k
views
Differences between MLE and MAP estimators
Generally speaking, what are the differences between an MLE and a MAP estimator?
If I wanted to improve the performance of a model, how would these differences come into play? Are there specific ...
- 121
1
vote
0
answers
36
views
Quantifying uncertainty in MAP nonlinear regression
I am interested in clinical pharmacokinetics, where we have a given medicine's population model, with its parameters (mean and standard deviation), and our goal is take one or two blood samples from a ...
- 11
2
votes
1
answer
2k
views
MLE and MAP with Naive Bayes
From what I understand, Naive Bayes classifies by doing:
$$
y \leftarrow argmax_{y_k}P(Y=y_k)\prod_{i}P(X_i|Y=y_k)
$$
There are two things there we need to know: $P(Y=y_k)$ and all the $P(X_i|Y=y_k)$
...
- 173
1
vote
0
answers
28
views
Bayesian inference MAP question
Hello I am stuck with this problem and was wondering if someone can solve it for me. Thanks
- 11
2
votes
1
answer
65
views
MAP: estimate 2 parameters
I have some data x and I want to estimate the mu and sigma of this data according to model $x \sim N(\mu, \sigma)$ where I have priors $\mu \sim N(0, 1)$ and $\sigma \sim \Gamma(1, 1)$.
Assume $\theta ...
- 263
2
votes
1
answer
112
views
Deriving the posterior distribution over the model parameters: are the model parameters and data independent?
We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
0
votes
2
answers
289
views
The prior in MAP and Bayesian inference
We can use a Normal distribution as a prior when handling a Normal distribution as likelihood in Bayesian inference
However if we want to do MAP
given a Bernoulli as likelihood can we use Normal ...
3
votes
1
answer
70
views
Does it make sense to condition on fixed values of some parameters before doing MCMC on the other parameters?
I have a Bayesian model with a large number of parameters (around 50), and as usual my goal is to infer the posterior distribution for the parameters, with MCMC.
However, I am only interested in the ...
- 33
2
votes
1
answer
904
views
Shouldn't log likelihood always be normalized by data size in bayesian estimation?
This is very interesting problem. I wonder if the whole bayesian statistics is neglecting it or if I am super confused. I will illustrate it on a bayesian Maximum a Posteriori (MAP) estimation and ...
- 6,033
1
vote
0
answers
23
views
Finding the MAP for a function whose conditioning depends on an exponential integral
Let $X$ be such that $X \sim exp( \lambda = 1)$ and let $Y$ be such that $Y \sim U[0,x]$, where $x$ is the realization of $X$.
Given that information I know that:
$f_{X}(x) = e^{-x}$ for $x \geq 0$...
- 640
1
vote
1
answer
26
views
MAP, MLE and parametrised data
It is often said that maximum likelihood is used to obtain estimates of distrubtion's parameters. However, what is unclear is whether it will produce consistent estimate parameters other than those of ...
- 11
4
votes
0
answers
201
views
Bias correcting penalized maximum likelihood / maximum a posteriori estimates
Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing:
$$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$
where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\...
- 495
1
vote
0
answers
33
views
Logistic regression fitting methods clarification
Each book I read propose a different fitting method for Logistic Regression.
The general idea is to maximize this expression.
$$
Pr\left(\beta|y,X,M\right) = \frac{Pr\left(y|\beta,X,M\right) Pr(\...
- 601
1
vote
1
answer
78
views
reference request for the impact of priors in bayesian statistics
It is well known that in bayesian statistics, the prior believe can have a large impact on the estimation result. For example if you flip a coin ten times to determine whether it is loaded, a prior $...
- 171
1
vote
1
answer
87
views
Assign an error to the parameters of MAP estimate
Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration.
...
- 11
1
vote
0
answers
222
views
Discriminative Models with Class Priors
In discriminative models, we model $p(Y|X)$ directly while in generative models we model $p(X|Y)p(Y)$ where $X$ is the input and $Y$ is the output variable. I am confused when the parameters and ...
- 473
3
votes
3
answers
437
views
Bayesian parameter estimation with proportion data
I am trying to do a Bayesian analysis using a model that comes from the literature in non-Bayesian form: $y = \Phi\Bigg(\frac{1}{\alpha} * log(A/\beta)\Bigg)$. Because the model uses the function $\...
0
votes
0
answers
288
views
Can you find the posterior mode of an unknown distribution without MCMC?
I was wondering if you wanted to compute the MAP estimate of an unknown posterior distribution, is there a non-sampling based method that would suffice? As in, if you don’t need to know anything more ...
- 71
0
votes
0
answers
56
views
How to choose estimates after Bayesian regression?
In a Bayesian logistic regression with two predictor variables $x_{1}$ and $x_{2}$, I did MCMC (2000 samples) to estimate posterior distribution. Now it's done, how can I choose the final estimates ...
- 393
0
votes
1
answer
232
views
Finding MAP estimate
I think after all the reading I've done I still don't fully understand MAP estimation. I came across a problem that's leaving me dumbfounded.
Suppose $A$ ~ $N(0,\sigma^2_1) $ and $\epsilon$ ~ $N(0,\...
- 167
1
vote
1
answer
460
views
What are the possible estimates of the parameters of the multinomial distribution?
The expected value of the parameters of the multinomial distribution (taking into account the Dirichlet prior $D(\alpha)$ and the posterior Dirichlet-Multinomial) is:
$\pi_i = α_i+ x_i / \sum_{j} α_j+...
- 262
0
votes
1
answer
241
views
Example of maximum a posteriori that does not match the mean of a marginalized posterior
Given a N-parameter likelihood and prior, I can obtain the marginalized posterior for each parameter through Bayesian MCMC.
I can also estimate the maximum a posteriori (MAP) of the N-parameter ...
- 3,440
0
votes
1
answer
59
views
Confused about maximum a posteriori estimation [closed]
I am new to Bayesian statistics, and I just came across MAP.
When our prior is a continuous distribution (pdf) on $\theta$ how can we calculate $g(\theta)$ in the numerator?
Edit:
I assumed $g(\...
- 123