3
$\begingroup$

When looking at papers about long memory they tend to analyze data sets whose length is in the thousands, see http://www.math.canterbury.ac.nz/~m.reale/pub/Reaetal2011.pdf for an example.

My question is to the long memory researchers and practitioners out there. What rule of thumb do you use to decide whether a data set is too small to be able to appropriately detect/estimate long memory?

[Naturally the smaller the long memory parameter the more observations you will require to detect it, but i'm after a general rule of thumb rather than exact notions around the number of observations required to detect a specific effect size.]

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ One thing you might want to try is lookup the $n$ used in simulation studies in a few theory papers. If you find that they generally use a sample size over a certain value, you can probably assume that for $n$ smaller than that there is decent power loss. $\endgroup$
    – David Veitch
    Jun 12 at 22:27
  • $\begingroup$ The problem is that what happens in simulation studies and how models behave in practice are very different things. My experience is that people choose round numbers that work and/or reviewers ask for longer/large data sets. It could provide an upper-lower bound though. Thanks for the suggestion! $\endgroup$
    – adunaic
    Jun 14 at 0:43
  • $\begingroup$ There is no hard or soft $n$. My rule of thumb is how much better can we do compared to a simple model. An OLS $y = \beta_0 + \beta_1t$ and then add the world's simplest periodic term so: $y= \dots + \beta_2sin(2\pi\frac{t}{T})+ \beta_3cos(2\pi\frac{t}{T})$ if there is obvious periodicity $T$. Takes ~15' to set up the data for - maybe throw an AR(1) error term if I think it can get estimated. In that sense, the question is on some false premises. There is no reason to use a GBM if a simple binary tree provides adequate performance or the GBM provides inadequate gains. Same thing with the LSTM. $\endgroup$
    – usεr11852
    Jun 15 at 19:54
  • $\begingroup$ I appreciate the premise that simpler models can often perform very well but I can't see how this comment is relevant to the question at hand. $\endgroup$
    – adunaic
    Jun 16 at 10:16

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.