Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

this is my first question on this site, so please be patient with me. I am doing a random walk, where I build a timeseries curve. I do that a preset number of times ( let's say 100 times ). Now I was wondering what should I do with all the generated curves. Eventually I want to have 1 curve that is the best representation. I tried taking the mean and median of the values for each point of time, but that gives me a rather tame and flat curve. What other options do I have? Your input is appreciated!

share|improve this question
What do you expect? If you want a typical Brownian motion, just generate one. If you want the mean behaviour, generate a lot and take the mean, like you did. Maybe you can add the variance as extra information. An idea would be to combine the three. Plot one realization against the mean and variances of many realization. You'll get 4 curves: the one from 1 realization, the mean curve, the mean+standard deviation and the mean-standard deviation. – Raskolnikov Dec 3 '10 at 13:24
The process could indeed have in theory some component of Brownian motion. The idea of the 4 curves sounds good to me. Although it seems that then you have to assume a normal distribution, which is certainly not the case. I was also thinking of using geometric mean, harmonic mean, median absolute deviation or something more exotic? – Navi Dec 3 '10 at 13:59
Sure, when you said random walk, I assumed Brownian motion because the term is sometimes used synonymously, although strictly speaking, you are right random can be refer to any type of randomness a priori. You could still use moving averages and variances over a certain time window if the process is more exotic. But you can avoid that by transforming to the proper scale sometimes. Say, you have a geometrical Brownian motion, just plot the log of it. – Raskolnikov Dec 3 '10 at 14:26

3 Answers

up vote 5 down vote accepted

I'm somewhat confused - if these are random walks, isn't the expectation a flat, uninteresting line?

set.seed(123)
nWalks <- 1000
nTimes <- 100
mat <- matrix(c(rep(0, nWalks), rnorm(nWalks*(nTimes-1))),
              ncol = nTimes)
rwalks <- apply(mat, 1, cumsum)
matplot(rwalks, type = "l")
## Stack the data for fitting a smoother
df <- stack(data.frame(rwalks))[, 2:1]
names(df) <- c("Walk","Xt")
df <- within(df, Time <- rep(seq_len(nTimes), nWalks))
## fit smother and predict
mod <- with(df, smooth.spline(Time, Xt))
pred <- predict(mod, x = 1:100)
lines(pred, col = "red", lwd = 4)

This gives:

1000 simulated random walks from origin 0, with summary smoothing spline (think, red line)

Whilst the expectation doesn't do an awful lot, the range and the variance do slightly more-interesting things:

layout(1:3)
plot(apply(rwalks, 1, mean), type = "l", main = "Mean",
     ylab = "Mean")
plot(apply(rwalks, 1, var), type = "l", main = "Variance",
     ylab = "Variance")
plot(apply(rwalks, 1, max) - apply(rwalks, 1, min), type = "l",
     main = "Range", ylab = "Range")
layout(1)

Expectation, variance and range of simulated random walks

The tiny fluctuations in the mean trace are due to sampling variation for the set of random walks I generated. it should settle down to be 0 if you up the number of random walks generated. (In any case, the value of the mean is effectively zero given the magnitude of the values the walks take.)

Perhaps you could explain a bit more what you mean by "best representation"?

share|improve this answer
These plots look really spectacular. Looking at the first plot, I realize that what I want is to show the most probable paths, because there are certain paths that are just too improbable. The problem is how choose the most probable paths in an elegant manner and summarize them. Eventually I want to display a reference "real" curve of the measured observed data, next to the simulated ones. – Navi Dec 3 '10 at 14:44
Perhabs you could make a histogramm after the 100 steps? – EDi Dec 3 '10 at 15:52
@Navi: by definition, the most probably path is a horizontal line around zero. These random walks are just cumulative sums of Gaussian noise with mean 0 and unit variance – Gavin Simpson Dec 3 '10 at 17:13
up vote 2 down vote

Plotting the mean or median for each timepoint sounds a sensible start. You could also plot a reference range for each timepoint to show the variability across curves at each timepoint. You could also add a few (perhaps 5 or 10) randomly-chosen curves to illustrate the variability across timepoints within each curve. Should be perfectly possible to show all of those things on the same plot with a suitable choice of colours and line weights.

That should give a graphical depiction of the process's behaviou but doesn't really answer your requirement for 'one curve that is the best representation'. But to answer that we need to know what you mean by 'best' -- how will you use this 'best curve'? The mean or median may look too flat and boring to use it as the sole graphical display but may be the 'best' summary curve for quite a few purposes.

share|improve this answer
I will look into the reference range. Best means approaching the values distribution as bes as possible. What I get are flat and boring curves, almost straight lines. The expectation is to get more wild curves with expononential growth/decline, sine wave like or something in between. Some amount of random noise has also to be thrown in for good measure. – Navi Dec 3 '10 at 13:54
up vote 0 down vote

If you're simply looking for a way of representing a process for readers, look up "diffusion modelling" in the psychological literature and you'll find how those folks typically attempt to represent a stochastic process and typical exemplar.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.