4
$\begingroup$

I have simulation data from 1000 runs, plotting some measurable (in this case convergence of the algorithm) as a function of simulation time. Each run produces a discrete set of points (t, f(t)) and the superimposed image of all the runs looks like:

enter image description here

Clearly there is some structure here. How can I generate a curve that somehow captures the average value of f(t) for a given t? In addition I'd like to be able to estimate the spread as well, though I'm not sure what the underlying distribution is.

Let's say that this plot was generated from simulation parameters A. The goal of this project is to show that another set of simulation parameters B, produces a different set of curves, and have a nice way to visualize this.

$\endgroup$
  • $\begingroup$ In this particular example there is no need for an average line, the semi-transparent traces are quite informative. Simply plot the traces for the different parameter estimates as separate colors. For similar examples see Hsiang (2013) or Lofgren (2013). $\endgroup$ – Andy W Jun 11 '14 at 14:53
  • $\begingroup$ @AndyW Perhaps in this picture there is no need to project the data onto a single line, however, I have three reasons why I'd like to know how to do so. 1] I have many parameter sets to compare A,B,C, etc..., 2] I'd like to apply some qualitative analysis to the different "average" curves and I think a projection might be the best place to start. 3] This can't be a unique problem, I think I'd learn something from other approaches. $\endgroup$ – Hooked Jun 11 '14 at 15:03
  • $\begingroup$ @AndyW Thank you for the refs though, the first one looks like something I'd be willing to implement. I'm not sure what to take from the second one however. $\endgroup$ – Hooked Jun 11 '14 at 15:04
  • $\begingroup$ A simple non-parametric approach would be for each t just calculate the median, and then plot that median trace. For error intervals you could similarly calculate specific quantiles for each t and use those as estimates of the error. Other ways may just use the mean at each t point and the standard error of that mean, or try to approximate f(t) through some flexible function (e.g. using regresssion with splines for t). $\endgroup$ – Andy W Jun 11 '14 at 15:24
  • $\begingroup$ @AndyW The t values for each function are distinct, in my case they represent a switching time. I'm guessing I could simply bin them on the log scale and apply the median calculation the same way? $\endgroup$ – Hooked Jun 11 '14 at 15:30
3
$\begingroup$

This can be addressed by using a GAM to model $f(t)$ ~ $t$, and plotting the fitted response curve with its confidence intervals.

$\endgroup$
  • $\begingroup$ Thanks for the response. Being unfamiliar with GAMs, what does the ~ mean in the equation? $f(t)$ is similar to $t$? $\endgroup$ – Hooked Jun 20 at 14:13
  • $\begingroup$ @Hooked Sorry, that's how R says 'is a function of' i.e. y ~ x reads as 'y is a function of x'. That's not specific to GAMs, it's just how you would tell R the dependent and independent variables. $\endgroup$ – mkt - Reinstate Monica Jun 20 at 14:28
  • 1
    $\begingroup$ Thanks! I'm going to accept this as it seems like it will work but if you could provide a starting point or reference to get started I'd appreciate it. $\endgroup$ – Hooked Jun 21 at 15:39
  • 1
    $\begingroup$ @Hooked Happy to help! They are pretty easy to work with, here's a couple of tutorials to get you started: environmentalcomputing.net/intro-to-gams & multithreaded.stitchfix.com/blog/2015/07/30/gam $\endgroup$ – mkt - Reinstate Monica Jun 21 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.