New answers tagged maximum-likelihood
1
vote
Simulated annealing on regression model
The objective function in non-linear least squares for fitting $y\approx f(\vec{x})$ is $\sum_i(f(\vec{x}_i) - y_i)^2$, which is in your case
$$Q(\alpha,\beta,\gamma) = \sum_{i=1}^n \left(\alpha x_{i1}...
- 3,161
4
votes
Accepted
EM algorithm on discrete random variable
I'm going to generalise your problem a little bit, to deal with a broader class of distributions. Specifically, I will add an additional parameter $\alpha$ to the problem; your question follows as a ...
- 97.7k
1
vote
Does Fisher scoring exist as such?
As mentioned before, you start with a random value of $\theta_0$, and then you (hopefully) CAN calculate the expectation. So the expectation is w.r.t. the current parameter value. Without a starting ...
- 2,277
0
votes
MLE vs MAP estimation, when to use which?
Theoretically, if you have the information about the prior probability, use MAP; otherwise MLE.
However, as the amount of data increases, the leading role of prior assumptions (which used by MAP) on ...
0
votes
Accepted
MLE of parameters for a difference of two Exponential IID
Following from the direction at the end of the post above, the solution is now
\begin{align*}
\theta_1 - \theta_2 &= S_p + S_n \\
\theta_1 + \theta_2 &= S_p - S_n + 2\sqrt{-S_n S_p} \\
&\...
- 69
2
votes
Standard Errors for Numerical Optimization using Chi-Square Objective Function
There are various Chi-square minimization methods described in the literature.
All of them boil down to minimizing a sum (or average) of normalized squared residuals (or Chi-square distances).
The ...
- 904
1
vote
Accepted
Bayesian model averaging
If you have two candidate likelihoods $P_1(\text{data}=\{x_1\dots x_n\}|\theta_1)$ and $P_2(\text{data}=\{x_1\dots x_n\}|\theta_2)$ with two different parametrizations $\theta_1$ and $\theta_2$, the ...
- 1,810
0
votes
Accepted
MLE for distribution having most general form
I don't know how you tried to differentiate the log-likelihood, but I tried to proceed in following way:
The pmf is given by,
$f(x_i;\theta)=(\frac{x_i}{\theta})^{\theta A'(\theta)}e^{A(\theta)+C(x_i)}...
- 91
4
votes
What is the justification of using the first-order Taylor expansion in the proof of asymptotic normality of MLEs?
The proofs I am familiar with
a) first prove consistency
b) then use a mean-value expansion, not a Taylor expansion, so they do not have a remainder, and the unknown value at which the Hessian must be ...
- 53.4k
0
votes
What's special about moments that allows "method of moments" to work?
The idea of the "method of moments" is indeed not restricted to moments, but can be applied to any estimator for a summary statistic. For instance, Elo derived an estimator for chess ratings ...
- 3,161
1
vote
Accepted
Sampling distribution of GBM Maximum-Likelihood estimator
It is common knowledge in asymptotic statistics that the asymptotic distribution of MLE in an exponential family is the normal distribution with mean being the MLE, and variance being the inverse ...
- 126
9
votes
Accepted
MLE for a modified German tank problem
Given your specified distributions, the marginal density of $X$ is:
$$\begin{align}
f_X(x)
&= \int \limits_0^a \text{N}(x |\mu,\sigma^2) \cdot \text{U}(\mu|0,a) \ d \mu \\[6pt]
&= \frac{1}{a} ...
- 97.7k
0
votes
How to choose between mean squared error and likelihood?
Notice that the mean squared error of the first approach can also arise from likelihood maximization, but for a model that assumes the same variance for the two normal distributions. Namely if you ...
- 4,157
2
votes
Should we really search for the model for which the probability of the data is maximal?
The disadvantage of the maximum a posteriori (MAP) estimator that you mention can be illustrated with a bi-modal distribution like the following
The maximum is in the point 2, but there is is not a ...
- 48.8k
Top 50 recent answers are included