2
$\begingroup$

I am going over the following slides where the likelihood for time series is formulated as (page 5):

$f(y) = \prod_{i=2}^{n} f_{y_i|y_{i-}}(y_i|y_{i-}) f_{y_1}(y_1)$

  1. I am not quite sure what the subscript $y_i|y_{i-}$ and $y_1$ is for. Could someone please explain?

  2. From my perspective, the formulation should look be:

$L(y;\theta) = \prod_{i=2}^{n} f_{i}(y_i|y_{i-};\theta) f_{1}(y_1)$.

Does this look reasonable? Can one assume each term has a distinct conditional distribution and the MLE will converge?

Improve this question
$\endgroup$
7
  • $\begingroup$ This notation is defined immediately before the equation. $y_1$ was defined on p. 2: it's the first data value in the time series. The vertical bar is the standard notation for a conditional probability (or density). See stats.stackexchange.com/…. But surely you know that, because you employed that notation in your post at stats.stackexchange.com/q/644953/919. What, then, do you need explained? $\endgroup$
    – whuber ♦
    Commented Jun 23 at 16:56
  • $\begingroup$ @whuber thank you, I do understand the vertical bar notation is for conditional probability. Why is it on the subscript though given that they it is already in the function param? From my post, I would have expected something like $f(y) = \prod_{i=2}^{n} f_i(y_i|y_{i-}) f_1(y_1)$ and here the $i$ would indicate that each conditional distribution could be different. $\endgroup$
    – spie227
    Commented Jun 23 at 17:05
  • $\begingroup$ Are you really just complaining about the notation adopted in that paper? If so, I'm sure many would agree that it could be better. But is there anything ambiguous about it? $\endgroup$
    – whuber ♦
    Commented Jun 23 at 18:41
  • $\begingroup$ I wanted to check whether this notation has some special meaning (to add to my confusion, there is no parameter specified) Would formulating it as $f(y) = \prod_{i=2}^{n} f_i(y_i|y_{i-}; \theta) f_1(y_1)$ be reasonable ? Where subscript $i$ indicates possibility of non-identical distribution and $\theta$ is the parameter to be learnt. $\endgroup$
    – spie227
    Commented Jun 23 at 18:51
  • 1
    $\begingroup$ I suggest you not fret over the notation and consider what is being said in context. In this context the point concerns factoring the joint density of $Y$ as a product of conditionals, so there is no need to mention any parameter: the concept and the factorization have nothing to do with whether the distribution is part of a family. $\endgroup$
    – whuber ♦
    Commented Jun 23 at 20:02

1 Answer 1

Reset to default
1
$\begingroup$

The key difference in your econometrics reference with the usual MLE/CE is the non-IID assumption in econometrics time series as mentioned on the same page.

There is a subscript $i$ on $f$ to allow for the possibility that each $Y_i$ has a distinct probability distribution... In time series and panel data problems there is often dependence among the $Y_i$’s.

Therefore there's a need to introduce a new random vector $Y_{i−}=\{Y_1, . . . , Y_{i−1}\}$ to account for such temporal dependence where each actually observed value for this new random vector is notated as $y_{i−}$.

Based on this background knowledge, it's now straightforward by simple substitution of above definition to your concerned RHS of the equation to arrive at LHS's joint density (mass) function $f(y)$ of the random vector $Y$, where $y$ is a generic outcome from the sample space $\Omega$ of the random vector $Y=\{Y_1, . . . , Y_i\}$ in measure-theoretic formal language of probability space.

Finally $L$ is usually the notation for likelihood function in statistics. Since the above LHS is nothing but joint distribution of the random series $Y$, it of course makes sense to notate as (joint) likelihood with the global parameters $\theta$ which is already employed on the same page of your reference. Because it's a non-IID time series, each conditional distribution term $f_i$ in your quoted equation may be different and more importantly as discussed in your another recent post to form meaningful MLE/CE loss function in non-IID cases for inference purpose you need to sample many cross-sectional data vector to get IID MLE. Then the final IID MLE for a very random and long time series in reality based on all those cross-sectional data could be very small and thus bounded above. With further simplified state-space model assumptions such as $f_i(y_i|x_i;θ)=f(y_i|x_i;θ)$ mentioned on your same referenced page and universal function approximators such as deep neural networks MLE will likely converge to find at least some local minimum in the parameters landscape to minimize its equivalent NLL/CE approximately.

To summarise, the notation $L(y;\theta) = \prod_{i=2}^{n} f_{i}(y_i|y_{i-};\theta) f_{1}(y_1)$ for non-IID sequential data and $L(y;\theta) = \prod_{i=1}^{n} f_{i}(y_i;\theta)$ for INID cross sectional data can be simplified $L(y;\theta) = \prod_{i=2}^{n} f(y_i|y_{i-};\theta)g(y_1)$ for a stationary sequence and $L(y;\theta) = \prod_{i=1}^{n} f_{i}(y_i;\theta)$ for IID cross-sectional data.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thanks @cinch. So just to summarise, $L(y;\theta) = \prod_{i=2}^{n} f_{i}(y_i|y_{i-};\theta) f_{1}(y_1)$ this formulation of MLE for a generic sequential DGP does make sense right? (similar to $L(y;\theta) = \prod_{i=1}^{n} f_{i}(y_i;\theta)$ formulation of a generic cross-sectional DGP) And the $i$ subscript can only be ignored if 1) time series is stationary and 2) cross-sectional data is identically distributed ? $\endgroup$
    – spie227
    Commented Jul 27 at 11:54
  • $\begingroup$ Yes, likelihood given in my reference is just parameterized inferred probability mass or density given a specific observed data outcome which could be a dependent sequential vector as in your econ time series case (we also discussed this in your previous question). It's just usually for the sake of using MLE as an optimization goal we usually demand a IID (cross-sectional) data since then its NLL/CE loss objective is a simple sum of log functions which is usually convex and easy to optimize for simple models with their corresponding parameters. Hope this completely clarifies. $\endgroup$
    – cinch
    Commented Jul 28 at 0:45
  • 1
    $\begingroup$ Thanks a lot. I edited your answer above to reflect this. Do let me know if that makes sense before I accept the answer. $\endgroup$
    – spie227
    Commented Jul 28 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.