0
$\begingroup$

Suppose I have this MA(1) model:

$y_t = \mu + \epsilon_t + \theta \epsilon_{t-1}$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$

The marginal distribution of $y_t$ for all $t$ is $\mathcal{N}(\mu,\sigma^2(1 + \theta^2))$ but the classical theory says that since the $y_t$ variables are not independent, the likelihood of the sample is not the product of the marginal densities. One has to use conditional densities instead (given $\epsilon_0 = 0$):

$y_1 \sim \mathcal{N}(\mu,\sigma^2)$
$y_2 \sim \mathcal{N}(\mu + \theta \epsilon_1,\sigma^2)$
$...$
$y_t \sim \mathcal{N}(\mu + \theta \epsilon_{t-1},\sigma^2)$

And the likelihood is the product of these conditional densities, right?

Now, I wrote a small program in R simulating a MA(1) process and displaying the distribution of the generated $y_t$ variables:

n = 100000
mu = 5
theta = 3
sigma = 2

y = rep(1, n)
eps.1 = 0

for (t in 1:n)
{
    eps = rnorm(1, mean = 0, sd = sigma)
    y[t] = mu + eps + theta * eps.1
    eps.1 = eps
}

hist(y, probability = TRUE)
curve(dnorm(x, mean = mu, sd = sigma * sqrt(1 + theta^2)), col = "blue", add = TRUE)

And I find out that the joint distribution of $y_t$ is $\mathcal{N}(\mu,\sigma^2(1 + \theta^2))$. How come?

$\endgroup$
8
  • 1
    $\begingroup$ The joint distribution is a multivariate density -so the phrase "the joint distribution of $y_t$" makes no sense. Do you mean "the joint distribution of $t$ $y$-random variables"? And what you found, is it perhaps the marginal distribution of the single r.v. $y_t$? Because it appears to be univariate, not multivariate. $\endgroup$ Commented Aug 15, 2014 at 12:52
  • $\begingroup$ Yeah, I meant $y_t$ as a process. The correct notation would be $(y_t)_t$. I also think I am plotting the marginal distribution of $y_t$. So, the question is: how do you plot a joint distribution from a sample? $\endgroup$ Commented Aug 15, 2014 at 14:48
  • $\begingroup$ If it is more than 2-dimensional, you cannot fully plot it (you need the 3d dimension for the value of the joint density). $\endgroup$ Commented Aug 15, 2014 at 14:54
  • $\begingroup$ Of course, but I was wondering why the marginal distribution (easily obtained) is not used for estimation purposes. If \sigma is known, the variance of the sample immediately gives the value of $\theta$ (except for the sign). And in case $\sigma$ is unknown, higher moments could be used to determine both $\sigma$ and $\theta$. But when it comes to estimating a MA(1) process, books always present the MLE method with a numerical maximization and this is much more complicated. Am I missing something? $\endgroup$ Commented Aug 15, 2014 at 15:22
  • $\begingroup$ I believe you are, in essence, wondering "why use maximum likelihood and not method-of-moments", so perhaps you should contemplate this question more generally, since one can almost always apply method-of-moments estimation: why do we use ML at all, when we have method-of-moments? $\endgroup$ Commented Aug 15, 2014 at 16:28

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.