Suppose I wanted to simulate the data generating process of a non-linear regression with ma(1) errors. So, without going into many unnecessary details, the model is

$$y_t = f(x_t,x_t-1,..., x_{t_0}, \beta_1, \beta_2) + \epsilon_t + \theta*\epsilon_{t-1}$$

where $\epsilon_t \sim N(0, \sigma^2)$.

The reason for doing this is that I want to test some non-linear estimation algorithms to compare their robustness and speed to each other. There are probably 4 of those but that's not important either.

My confusion is the following. Given some $\beta_1$ and $\beta_2$ and some $\theta$, I can generate the DGP of pretty straightforwardly because I have the $x_t$s.

But how do I deal with scaling the noise. By this I mean, how can I know what reasonable values for $\sigma^2$ are? For example, if I use too small of a $\sigma^2$, then my error term could end up being negligible and I ended with a deterministic relationship.

On the other hand, if I use too large of a $\sigma^2$, it could end up dominating the two other terms so that I end up generating an MA(1) model. It's not clear to me how to know what the scale for $\sigma^2$ should be.

  • $\begingroup$ As presented in your question, there is no indication that you should scale $\sigma$ one way or the other. Do you have actual data to relate with? $\endgroup$ – Xi'an Dec 18 '14 at 18:25
  • $\begingroup$ @Xi'an : I do have data so I plan on using something similar to what Aksakal proposed. $\endgroup$ – mlofton Dec 19 '14 at 17:38
  • $\begingroup$ @Aksakal: I think it's just notational sematics. I could have said $y_t = f(.) + \gamma_t$ where $\gamma_t = \epsilon_t + \theta* \epsilon_{t-1}$. An MA(1) has the same noise term occurring twice on the RHS. $\endgroup$ – mlofton Dec 19 '14 at 17:41
  • $\begingroup$ @Aksakal: I would rather use the $\epsilon_t+\theta\epsilon_{t-1}$ to keep with the traditional ARMA representation. $\endgroup$ – Xi'an Dec 19 '14 at 17:41
  • $\begingroup$ @mlofton, I withdraw my comment, agree with you. $\endgroup$ – Aksakal Dec 19 '14 at 18:34

I would start with fraction of variance unexplained, and think of what would be a reasonable value for your domain. Maybe in your field you expect the models have VFU 20%. In this case you can use $\sigma^2\sim0.2Var[y_t]$ for errors.

  • $\begingroup$ interesting: one idea I had which sounds similar is the following: take one of the data sets, estimate it blindly to get the estimated parameters. then back out the estimated variance using rss/n-k and use that as my sigmasquared hat and then set that sigmasquared hat = var(e_t) + gammasquared var(e_t) to back out var(e_t). is that similar to what you are saying ? if so, I don't understand the 0.2 but I'll click on link. thanks. $\endgroup$ – mlofton Dec 18 '14 at 15:52
  • $\begingroup$ just clicked. I think I get what you are saying now. Is it the following: Calculate SSTOT. Then back out SSERR so that SSERR /SSTOT = 0.2 say. Then, calculate SSERR/n-k as the estimate of simgasquared. Then, set the estimate of sigmasquared to var(e_t) + gamma^2* var(e_t) to back out var(e_t) ? I hope that's right. thanks. $\endgroup$ – mlofton Dec 18 '14 at 16:01
  • $\begingroup$ @mlofton, yes, something like that. The main idea is to base the variance of errors on some fraction of the variance of the dependent variable. The rest is details. I wouldn't worry about backing out exact sigma of errors from MA terms. All you need is to get the sensible sigma's range for simulation tests. $\endgroup$ – Aksakal Dec 18 '14 at 16:17
  • $\begingroup$ great stuff. if this answer is any indication of the quality of this list, then it's an amazing list. thanks again. $\endgroup$ – mlofton Dec 18 '14 at 16:23

Just saw your question ... Please look at how to generate random time series for a given one, including all trends? for my suggestion as to how to simulate a series given its DGF ( read model form ! ) .

I determine the error variance from a useful model and then use it in conjunction with estimated model parameters to launch the simulation.

regards to a fellow time series enthusiast ...

  • $\begingroup$ Thanks. I will check out the link. I don't remember exactly what I did at the time but it was something along the lines of what Aksakal recommended. $\endgroup$ – mlofton Sep 15 '19 at 4:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.