Suppose I have this MA(1) model:

$y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$

The marginal distribution of $y_t$ for all $t$ is $\mathcal{N}(\mu,\sigma^2(1 + \theta^2))$ but the classical theory says that since the $y_t$ variables are not independent, the likelihood of the sample is not the product of the marginal densities. One has to use conditional densities instead (given $\epsilon_0 = 0$):

$y_1 \sim \mathcal{N}(\mu,\sigma^2)$
$y_2 \sim \mathcal{N}(\mu + \theta \epsilon_1,\sigma^2)$
$y_t \sim \mathcal{N}(\mu + \theta \epsilon_{t-1},\sigma^2)$

And the likelihood is the product of these conditional densities, right?

Now, I wrote a small program in R simulating a MA(1) process and displaying the distribution of the generated $y_t$ variables:

n = 100000
mu = 5
theta = 3
sigma = 2

y = rep(1, n)
eps.1 = 0

for (t in 1:n)
    eps = rnorm(1, mean = 0, sd = sigma)
    y[t] = mu + eps + theta * eps.1
    eps.1 = eps

hist(y, probability = TRUE)
curve(dnorm(x, mean = mu, sd = sigma * sqrt(1 + theta^2)), col = "blue", add = TRUE)

And I find out that the joint distribution of $y_t$ is $\mathcal{N}(\mu,\sigma^2(1 + \theta^2))$. How come?

  • 1
    $\begingroup$ The joint distribution is a multivariate density -so the phrase "the joint distribution of $y_t$" makes no sense. Do you mean "the joint distribution of $t$ $y$-random variables"? And what you found, is it perhaps the marginal distribution of the single r.v. $y_t$? Because it appears to be univariate, not multivariate. $\endgroup$ – Alecos Papadopoulos Aug 15 '14 at 12:52
  • $\begingroup$ Yeah, I meant $y_t$ as a process. The correct notation would be $(y_t)_t$. I also think I am plotting the marginal distribution of $y_t$. So, the question is: how do you plot a joint distribution from a sample? $\endgroup$ – Mark Morrisson Aug 15 '14 at 14:48
  • $\begingroup$ If it is more than 2-dimensional, you cannot fully plot it (you need the 3d dimension for the value of the joint density). $\endgroup$ – Alecos Papadopoulos Aug 15 '14 at 14:54
  • $\begingroup$ Of course, but I was wondering why the marginal distribution (easily obtained) is not used for estimation purposes. If \sigma is known, the variance of the sample immediately gives the value of $\theta$ (except for the sign). And in case $\sigma$ is unknown, higher moments could be used to determine both $\sigma$ and $\theta$. But when it comes to estimating a MA(1) process, books always present the MLE method with a numerical maximization and this is much more complicated. Am I missing something? $\endgroup$ – Mark Morrisson Aug 15 '14 at 15:22
  • $\begingroup$ I believe you are, in essence, wondering "why use maximum likelihood and not method-of-moments", so perhaps you should contemplate this question more generally, since one can almost always apply method-of-moments estimation: why do we use ML at all, when we have method-of-moments? $\endgroup$ – Alecos Papadopoulos Aug 15 '14 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.