0
$\begingroup$

I have a 250-time points series of actual data and I want to generate a similar time series including all its curves

first, I was interested to know its probability distribution and see if it fits other distributions, after fitting, it was Normal and Lognormal seem better. So I generated 250 points of Lognormal but they came absolutely random resembling no properties of the original time series (Lognormal are illustrated in the second chart)

The original time series it looks like this

enter image description here

My generated Lognormal according to the data parameters looks like this

enter image description here

the data is as follows 1800 1650 1850 2050 2150 1950 1700 1850 2200 2270 1950 1730 1850 2200 2350 1950 1780 1930 2200 2400 2270 1920 2080 2340 2470 2300 1890 2180 2340 2490 2300 1900 2130 2370 2440 2430 2050 2220 2400 2550 2060 1960 1890 2240 2490 2070 1920 1870 2330 2490 2170 1840 1970 2240 2570 2140 1810 1930 2270 2530 2080 1720 1970 2340 2510 2070 1680 1990 2310 2410 1990 1710 1970 2260 2470 2010 1620 1910 2270 2430 1840 1720 1960 2090 2240 1820 1680 2030 2070 2190 1820 1710 1970 2010 2160 1770 1670 1980 2050 2080 1620 1440 1710 1870 2120 1610 1470 1720 1820 2070 1690 1340 1770 1920 2080 1610 1540 1740 1960 2140 1730 1620 1710 1910 2230 1690 1580 1770 1870 2170 1740 1690 1730 1910 2210 1750 1590 1760 1930 2170 1840 1880 2030 2110 2240 1940 1740 2030 2090 2390 1910 1840 2190 2120 2270 1940 1840 2090 2160 2370 1930 1720 1870 1980 2210 1990 1670 1830 1920 2110 1910 1740 1890 1920 2170 1840 1760 1820 1990 2190 2080 1990 2230 2340 2470 2080 1910 2150 2310 2460 2120 1950 2200 2280 2520 2140 1970 2220 2320 2550 2610 2240 2387 2520 2560 2470 2210 2490 2450 2630 2510 2170 2420 2440 2580 2490 2270 2380 2520 2620 2610 2460 2520 2540 2760 2690 2530 2590 2630 2750 2680 2480 2630 2690 2770 2630 2420 2570 2580 2740 2170 2420 2440 2580 2490 2270 2380 2520 2620 2580

Improve this question
$\endgroup$
11
  • 3
    $\begingroup$ How did you find the parameters for the first series? From a rough visual look at the series, it seems like it has some persistence that your second simulated series does not have (plus some short-term ciclicality that is close to seasonality because it seems fairly regular, but this is not the main point here and should be confirmed by something else than visual inspection). What seems clear is that is either a very persistent stationary process that keeps coming back to 2000 or even a I(1). In any case it is far more persistent than yours, that seems a stationary process with not much memory $\endgroup$
    – Fr1
    Sep 9, 2019 at 10:18
  • 1
    $\begingroup$ So I suspect that the problem may be that you have not estimated the parameters very well. Was the first series a simulated series as well? or was it a real-world example with real data? consider that you are focusing on the shape of the pdf in your question, but here the estimated parameters are very important. Once you get an accurate estimate of the parameters, then, you can use them with a random innovation (possibly from the same marginal pdf as the original series 1) to simulate. $\endgroup$
    – Fr1
    Sep 9, 2019 at 10:24
  • 1
    $\begingroup$ At that point, depending on the properties of the series (firstly depending on stationarity and secondly depending on the presence of trends etc.) you may or may not have a similar graph. Consider the case of a random walk without drift, which is by definition fully dependent on innovations. In that case, since the the future level of the process is just a function of current level and innovations, you may have a completely different graph in the simulation as the simulated innovations change compared to the historical innovations of series 1. If a trend exist, it may reduce the discrepancy. $\endgroup$
    – Fr1
    Sep 9, 2019 at 10:27
  • 2
    $\begingroup$ This is a question where I gotta ask: why? What is the goal that you are trying to achieve? There is a lot of different ways to interpret the stated goal to "generate a similar time series". The most "similar" time series is the original series itself. If you want a similar time series, what should be different? I would also echo the sentiment of @Fr1 which is that your data is clearly not a stationary random process, so modelling it as a stationary lognormal process will clearly not capture the high- and low-frequency trends that are obvious in your oringinal data. $\endgroup$
    – timchap
    Sep 9, 2019 at 10:44
  • 2
    $\begingroup$ "the required was to plug the 250 days data into an optimization model. that is a decision has been made." could you explain more about this optimization model and the 'decision' that has to be made. It seems to me like you sort of want to generate 'made up data' in order to test those optimization models. This seems like a valid approach. But details are lacking. In addition it is complex as this question mixes two concepts: 1 how to correctly fit the data 2 how to simulate 'made up data' for a specific purpose. $\endgroup$
    – Sextus Empiricus
    Sep 10, 2019 at 8:34

1 Answer 1

Reset to default
1
$\begingroup$

"I was interested to know its probability distribution and see if it fits other distributions" ... there is no reason to model the observed series which can be reduced to simpler structure. Failing modelling attempts one simple uses monte-carlo procedures using the cdf of the observed series to generate values for the composite BUT not time series traces.

There is no need to transform your data see the following for clues on power transforms When (and why) should you take the log of a distribution (of numbers)?

What I propose here is a 2 step procedure 1) characterize the series ..separating observed to signal and noise AND then 2) simulating white noise and using it to render a data realization of the model .

It is important to characterize the series i.e identify the nature of the "curves" . In this way you can separate the data into signal and noise and focus on simulating trial/candidate noise processes leading to "similar time series" i.e. time series with nearly identical acf.

In this way after a useful model has been identified one can then use simulated error processes and then "reverse" the model by injecting/inputting the simulated error processes to generate a trial realization of the data. This is essentially how one generates a forecast and the probability distribution to assess prediction limits for a given interval of time BUT we will not be forecasting here.

Please post your actual 250 data points and I and others might be able to help you further.

EDITED AFTER RECEIPT OF YOUR 250 OBSERVATIONS :

One of the reasons I asked you to post your data is that I have long been a fan of simulation (dating back to the mid-sixties) as an objective way of evaluating strategies and suggested procedures. As I developed and improved model identification strategies , I conducted simulation of data in order to evaluate signal detection. I always have maintained a currency in simulation as a possible lithmus test.

I took your 250 values and used AUTOBOX , a piece of software that not only has model identification BUT data simulation capabilities. I used it to take your data and identify the following model in order to separate signal and noise and then to use the model as the basis for simulations.

Here is the ACF of the origenter image description hereinal series . AUTOBOX rendered a model here enter image description here with residuals here enter image description here and acf here enter image description here

The Actual & Fitted values are plotted hereenter image description here

Now the Histogram of the original 250 values is here and is shown for completeness sake enter image description here while the histogram of the error series from our model is here enter image description here and much more important in our exercise.

We now proceed to use the simulation option in AUTOBOX which requires a model , presented here in a structured manner enter image description here

We ask the software to generate 0,1 random numbers and use them as the basis (input) into the model form to create a 250 period realization . This can then be repeated for as many replications of the 250 simulated values.

I show here the acf of the original 250 values enter image description here and the acf of the first replication of the 250 simulated values enter image description here . The simulated series "resembles" the original series which is what I think you want to do.

Your question could have/should have been worded more correctly. I guess you now how to modify your question given my answer. I suggest you do so.

I suggest that you get hold of a good simulation piece of software ( and there are a few !) to allow you to specify the required equation for the generated simulation OR write your own code to do this .

enter image description here presents the relationship between an error process (perhaps the result of simulation) and the realization in terms of Y

Improve this answer
$\endgroup$
4
  • $\begingroup$ I posted the data, can you guide me? $\endgroup$
    – Mostafa Hussien
    Sep 9, 2019 at 16:51
  • $\begingroup$ Wow! Now this will need some serious understanding from my side as this is my first time to do time series. But at least, where in this is the method to simulate different scenarios of the same time series without forecasting $\endgroup$
    – Mostafa Hussien
    Sep 9, 2019 at 20:34
  • $\begingroup$ if you can characterize thus separating signal & noise via a FILTER AND then vary the noise BUT not it's nature & then use it you can get a "new series" .Simulation is the CONVERSE of modelling . With modelling you convert or filter the original series to render a "simpler series" . With simulation you do tjhe converse you take a set of a set of values and inject it via the INVERSE FILTER read model in order to generate a new series with similar characteristics as the original series. I have often characterized myself as a wannabe "noise maker" via efficient characterizarion/modelling. $\endgroup$
    – IrishStat
    Sep 9, 2019 at 20:42
  • $\begingroup$ there was no forecasting done at all .. The underlying model was detected and then useful in separating signal to fit and a white noise set of errors. time series modelling is a bi-directional approach i.e. y via a filter leads to a stochastic process white noise ... white noise (possibly simulated !) via an inverse filter leads to y. $\endgroup$
    – IrishStat
    Sep 9, 2019 at 21:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.