30
votes
Why is method of moments (MoM) not unique? What is uniqueness?
Let's take a different example, a random variable $X$ having a Poisson distribution with parameter $\lambda$.
You can say $\mathbb E[X] = \lambda$, so $\hat \lambda_1 = \frac1n\sum X$ is a method of ...
- 42.2k
20
votes
Accepted
When do maximum likelihood and method of moments produce the same estimators?
A general answer is that an estimator based on a method of moments is not invariant by a bijective change of parameterisation, while a maximum likelihood estimator is invariant. Therefore, they almost ...
- 108k
19
votes
Accepted
A description of the mean of the Geometric Distribution - is it unorthodox or just incorrect?
$\exp(\mathbb E[\log(X)])$
is the geometric mean of a positive random variable $X$
not the mean of a geometric random variable.
So either the homework directions put the words in the wrong order, or ...
- 42.2k
15
votes
For Gamma distribution, use MLE or MoM?
Use MLE.
Gamma distributions start approximating Normal distributions for large shape parameters (where both MoM and MLE ought to be equally fine), and the scale parameter merely establishes a unit of ...
- 334k
14
votes
Accepted
Principle of Analogy and Method of Moments
Least squares estimator in the classical linear regression model is a Method of Moments estimator.
The model is
$$\mathbf y = \mathbf X\beta + \mathbf u$$
Instead of minimizing the sum of ...
- 60.8k
14
votes
What is the Method of Moments and how is it different from MLE?
What is the method of moments?
There is a nice article about this on Wikipedia.
It means that you are estimating the population parameters by selecting the parameters such that for certain specific ...
- 86.5k
13
votes
What is the Method of Moments and how is it different from MLE?
In MoM, the estimator is chosen so that some function has conditional expectation equal to zero. E.g. $E[g(y,x,\theta)] = 0$. Often the expectation is conditional on $x$. Typically, this is converted ...
- 822
10
votes
Real life uses of Moment generating functions
You are right that mgf's can seem somewhat unmotivated in introductory courses. So, some examples of use. First, in discrete probability problems often we use the probability generating function, ...
- 82.8k
9
votes
Why is OLS related to Moment Estimation?
You get the equations you quote by differentiating the squared error.
If
$$\mathrm{RSS}=\sum_i (y-\beta_0-\beta_1x)^2$$
then
$$\frac{\partial\mathrm{RSS}}{\partial\beta_0} = -2\sum_i (y-\beta_0-\...
- 46.8k
8
votes
What is the Method of Moments and how is it different from MLE?
The MLE is invariant by transformation of the data $(X_1,...,X_n)$ by a strictly increasing transformation. Concretely, if $g$ is strictly increasing and $Y_i=g(X_i)$ and the $X_i$'s have density $f_{\...
- 596
7
votes
Accepted
Hyper-parameter estimation for Beta-Binomial Empirical Bayes
The hierarchical model
You don't actually even need the marginal probability mass function $m()$, you actually only need the marginal moments of $Y$.
In this tutorial, Casella (1992) is assuming the ...
- 13.5k
7
votes
Accepted
"Appropriate conditions" for method of moments estimator to exist, be consistent, and asymptotically normal?
Almost all arguments to asymptotic normality of a sequence of statistics hinge on arguments using Taylor series, and thus, the "appropriate conditions" are generally smoothness conditions required to ...
- 133k
7
votes
Why is OLS related to Moment Estimation?
The idea of method of moments says that you
take the population moment conditions (here, after substitution, the last two equations you present)
replace expected values with sample analogons (sample ...
- 34.8k
7
votes
Why is method of moments (MoM) not unique? What is uniqueness?
The method of moments means
that you are estimating the population parameters by selecting the parameters such that for certain specific moments the population distribution has the moments that are ...
- 86.5k
7
votes
Method of Moments of Uniform Distribution
The mean of this distribution is always zero so it does not depend on the parameter $\theta$. For that reason, if you were using MOM estimation you could use the second raw moment, which is:
$$\...
- 133k
5
votes
Accepted
Method of moment estimates for n Bernoulli trials
Hint: "method of moments" means you set sample moments equal to population/theoretical moments.
For example, the first sample moment is $\bar{X} = n^{-1}\sum_{i=1}^n X_i$, and the second sample ...
- 21.5k
5
votes
Parameter estimates for the triangular distribution
Using the extreme-order statistics as estimators for the boundaries $a,b$ and then using
$$E(X) = \frac {a+b+c}{3}$$
to estimate $c$ by method of moments is so ...maddeningly easy,
$$\hat a = X_{...
5
votes
Accepted
Empirical Bayes: method of moments
First, note that the combination of a Binomial distribution for $X_i | n_i, \theta_i$ and a Beta distribution for $\theta_i | \alpha, \beta$ leads directly to a Beta-Binomial distribution for $X_i | ...
- 41.1k
5
votes
Accepted
Truncated Beta parameters - method of moments
Your data is drawn from a censored Beta distribution, with the censoring point unknown as well as how many observations were censored. The PDF of the distribution is:
$$p(x; a, b, c) = {x^{a-1}(1-x)^...
- 41.1k
5
votes
Accepted
By conditioning on $N$, show that the moment generating function of $Y$ is given by $m_Y(t)=m_N(\ln(m_X(t)))$
The mgf of $Y$ conditional on $N=n$ is
$$
M_{Y|N=n}(t)=M_X(t)^n,
$$
since $Y$ is a sum of independent random variables each with mgf $M_X(t)$. Using the law of total expectation and the definition of ...
- 11.8k
5
votes
Reasons for different parameters via MoM and MLE
One reason is, that the data does not seem to follow a gamma distribution. Comparing the ECDF with the one predicted by the gamma distribution using either parameter set shows this quite clearly (blue ...
- 106
5
votes
Accepted
Method of moments and MLE estimates for Lomax (Pareto Type 2)
The issue appears to be the greatly different scales of the two parameters and how that interacts with BFGS. When I try optim using BFGS on the raw data, I get ...
- 41.1k
5
votes
An example of continuous random variable X > 0 with finite second moment but Infinite third moment
So, you want
$$\int_0^\infty x^2f(x),dx$$ to exist, but
$$\int_0^\infty x^3f(x),dx$$ to be infinite.
We know that the integral $\int_1^\infty x^{-n}\,dx$ is finite if $n>1$ and infinite if $n\leq ...
- 46.8k
5
votes
Is this estimator biased or unbiased?
It's unbiased.
$$E\left[\frac{X}{\lambda}\right]=E\left[\frac{X}{N}\cdot\frac{N}{\lambda}\right]=E_N\left[E\left[\frac{X}{N}\cdot\frac{N}{\lambda}\middle| N\right] \right]$$
Now
$$E\left[\frac{X}{N}\...
- 46.8k
5
votes
Finding method of moments estimate for density function $f(x|\alpha) = \frac {\Gamma(2\alpha)} {\Gamma(\alpha)^2}[x(1-x)]^{\alpha - 1}$
Your answer is equivalent to the solution provided, so there is no problem there. It is also worth noting that there is an alternative MOM estimator that is obtained by equating the sample variance ...
- 133k
4
votes
Accepted
Does an optimal linear classifier perform no better then chance iff class distributions have the same mean?
The answer is no. I'll show a simple counterexample where $p(x \mid y=0)$ and $p(x \mid y=1)$ have the same mean, but it's possible to construct a linear classifier with misclassification rate better ...
- 33.2k
4
votes
What is the Method of Moments and how is it different from MLE?
Sorry, I can't post comments..
MLE makes stricter assumptions (the full density) and is thus
typically less robust but more efficient if the assumptions are met
Actually, at MITx "Fundamentals ...
- 403
4
votes
Accepted
What is the limiting distribution of $Y_n = \sqrt{n}(\bar{X}_n-1)$ as $n \to \infty$?
In your updated version, you will get $\left[1 - \frac{t^2/2}{n} +o(\frac{1}{n})\right]^{-n} = \left[1 + \frac{t^2/2}{n} +o(\frac{1}{n})\right]^{n}$
and the limit of that is $e^{t^2/2}$, which is ...
- 42.2k
4
votes
Accepted
Method of moments estimator, $P_\theta(X = x) = \frac{1}{\theta}$
This is a discrete distribution, so it does not have a density $f(x)$ but instead a probability mass function. Its expectation is $$E[X]= 1 \times \frac1\theta + 2 \times \frac1\theta + \cdots + \...
- 42.2k
4
votes
Accepted
Negative-Binomial Method of moments with an offset
Your model is $\log\mu=\beta_0+\log t$, since for an offset (which you log-transform since you are working with a log link) you constrain the corresponding parameter to be $1$. On the original scale (...
- 131k
Only top scored, non community-wiki answers of a minimum length are eligible