I know this is a very basic question, but I am getting a bit confused due to some variation between resources I'm using for a statistics course.

Say you have some $iid$ random samples $X_1$,$X_2$,...,$X_n$ that have some pdf $f(x|\beta,\delta)$. When we are trying to determine the method of moments estimators $\hat \beta$ and $\hat \delta$, I am confused as to what we equate to determine these estimators. I know that you let $E(X)$=$\bar x$, but for the second equation that you need, do you let $Var(X)$=$s^2$ or are you meant to use the second moment $E(X^2)$ somehow?

  • $\begingroup$ What you need to do is to create set of equations which one side you have the moment derived analytically from the distribution and depends on the parameters you're after and on the other side the empirical value of the moment. Then just solve it. $\endgroup$ – Royi Mar 17 '18 at 9:02
  • $\begingroup$ I will give an example using Gaussian Distribution. $\endgroup$ – Royi Mar 17 '18 at 9:14

As I wrote in comments, all you need is to generate set of solvable equations system.
Since you're after 2 parameters, you need at least 2 equations.

Let's try doing so using Normal Distribution.

The model is:

$$ {X}_{i} \sim f \left( x \mid \mu, {\sigma}^{2} \right) = \frac{1}{ \sqrt{2 \pi {\sigma}^{2}} } \exp \left( - \frac{ \left( x - \mu \right)^{2} }{2 {\sigma}^{2} } \right) $$

Now, in the Normal Distribution the Moments are function of the Mean and the Variance as following - Normal Distribution Moment:

enter image description here

Given $ n $ samples the empirical $ p $ moment is given by:

$$ {m}_{i} = \frac{1}{n} \sum_{j = 1}^{n} {x}_{j}^{i} $$

Now it is easy to create set of equations.
In this case even solving them by hand.

In general case using Non Linear Solver.
I used MATLAB's lsqnonlin() as following:

%% Generate Data

vX = paramMu + (sqrt(paramSigmaSq) * randn([numSamples, 1]));
vEmpMoment = zeros([numMoments, 1]);

%% Generating Empirical Moments

for ii = 1:numMoments
    vEmpMoment(ii) = mean(vX .^ ii);

hResFun = @(vTheta) ResFun(numMoments, vTheta(1), vTheta(2), vEmpMoment);
vTheta = [0, 1];

sSolverOptions = optimoptions('lsqnonlin', 'Algorithm', 'trust-region-reflective', 'FunctionTolerance', 1e-9, 'StepTolerance', 1e-9, 'FiniteDifferenceType', 'central', 'Display', 'iter');

vTheta = lsqnonlin(hResFun, vTheta, [], [], sSolverOptions);

I must say it didn't do a very well job solving it.

The code is available (Including validation by CVX) at my StackExchange Cross Validated Q334017 GitHub Repository.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy