New answers tagged

1 vote

Simulated annealing on regression model

The objective function in non-linear least squares for fitting $y\approx f(\vec{x})$ is $\sum_i(f(\vec{x}_i) - y_i)^2$, which is in your case $$Q(\alpha,\beta,\gamma) = \sum_{i=1}^n \left(\alpha x_{i1}...
user avatar
  • 3,161
4 votes
Accepted

EM algorithm on discrete random variable

I'm going to generalise your problem a little bit, to deal with a broader class of distributions. Specifically, I will add an additional parameter $\alpha$ to the problem; your question follows as a ...
user avatar
  • 97.7k
1 vote

Does Fisher scoring exist as such?

As mentioned before, you start with a random value of $\theta_0$, and then you (hopefully) CAN calculate the expectation. So the expectation is w.r.t. the current parameter value. Without a starting ...
user avatar
0 votes

MLE vs MAP estimation, when to use which?

Theoretically, if you have the information about the prior probability, use MAP; otherwise MLE. However, as the amount of data increases, the leading role of prior assumptions (which used by MAP) on ...
user avatar
0 votes
Accepted

MLE of parameters for a difference of two Exponential IID

Following from the direction at the end of the post above, the solution is now \begin{align*} \theta_1 - \theta_2 &= S_p + S_n \\ \theta_1 + \theta_2 &= S_p - S_n + 2\sqrt{-S_n S_p} \\ &\...
user avatar
  • 69
2 votes

Standard Errors for Numerical Optimization using Chi-Square Objective Function

There are various Chi-square minimization methods described in the literature. All of them boil down to minimizing a sum (or average) of normalized squared residuals (or Chi-square distances). The ...
user avatar
  • 904
1 vote
Accepted

Bayesian model averaging

If you have two candidate likelihoods $P_1(\text{data}=\{x_1\dots x_n\}|\theta_1)$ and $P_2(\text{data}=\{x_1\dots x_n\}|\theta_2)$ with two different parametrizations $\theta_1$ and $\theta_2$, the ...
user avatar
0 votes
Accepted

MLE for distribution having most general form

I don't know how you tried to differentiate the log-likelihood, but I tried to proceed in following way: The pmf is given by, $f(x_i;\theta)=(\frac{x_i}{\theta})^{\theta A'(\theta)}e^{A(\theta)+C(x_i)}...
user avatar
  • 91
4 votes

What is the justification of using the first-order Taylor expansion in the proof of asymptotic normality of MLEs?

The proofs I am familiar with a) first prove consistency b) then use a mean-value expansion, not a Taylor expansion, so they do not have a remainder, and the unknown value at which the Hessian must be ...
user avatar
0 votes

What's special about moments that allows "method of moments" to work?

The idea of the "method of moments" is indeed not restricted to moments, but can be applied to any estimator for a summary statistic. For instance, Elo derived an estimator for chess ratings ...
user avatar
  • 3,161
1 vote
Accepted

Sampling distribution of GBM Maximum-Likelihood estimator

It is common knowledge in asymptotic statistics that the asymptotic distribution of MLE in an exponential family is the normal distribution with mean being the MLE, and variance being the inverse ...
user avatar
9 votes
Accepted

MLE for a modified German tank problem

Given your specified distributions, the marginal density of $X$ is: $$\begin{align} f_X(x) &= \int \limits_0^a \text{N}(x |\mu,\sigma^2) \cdot \text{U}(\mu|0,a) \ d \mu \\[6pt] &= \frac{1}{a} ...
user avatar
  • 97.7k
0 votes

How to choose between mean squared error and likelihood?

Notice that the mean squared error of the first approach can also arise from likelihood maximization, but for a model that assumes the same variance for the two normal distributions. Namely if you ...
user avatar
  • 4,157
2 votes

Should we really search for the model for which the probability of the data is maximal?

The disadvantage of the maximum a posteriori (MAP) estimator that you mention can be illustrated with a bi-modal distribution like the following The maximum is in the point 2, but there is is not a ...
user avatar

Top 50 recent answers are included