Skip to main content
Notice removed Draw attention by eithompson
Bounty Ended with user20160's answer chosen by eithompson
Tweeted twitter.com/StackStats/status/1284004928747446277
Notice added Draw attention by eithompson
Bounty Started worth 50 reputation by eithompson
deleted 4 characters in body
Source Link
eithompson
  • 376
  • 1
  • 10

I am looking at time-series event data and need to visualize how the arrival rate $\lambda$ changes over time. I do not want to assume any underlying distribution that the data comes from (it is certainly not a Poisson process as the rate changes, and neither the Weibull nor the log-logistic distribution seem to describe what's going on). What I want is something between the following two extremes:

  1. Assume that the rate does not change with the time since the previous event (Poisson), and simply calculate the overall rate of events per time. Problem: just a single number, i.e., no ability for the rate to change over time.
  2. Calculate the empirical instantaneous rate between all sets of neighboring points (as in $\frac{1}{t_i - t_{i-1}}$) and plot this over time. Problem: too jumpy, not very informative.

I want something in between these two extremes, in the sense that a kernel density estimate is "in between" calculating the mean of observations, vs. looking at a rugplot showing the individual values of the data. Ultimately, I want to see a smoothed curve that shows a locally-weighted average of the rate of arrivals at each pointover time.

Two classes of approach come to mind: some kind of kernel density estimate of the values from extreme #2, or some use of an (inverse) exponential function of the previous interarrival times to provide an exponentially-weighted average of the rate.

Is there a right way to do this? If so, is there also a right way to select an appropriate bandwidth/weighting value (which I'm assuming will be necessary, whatever the solution is)?

PS This is not a failure analysis, but the concept of hazard rates seems somewhat relevant. However, I'm not finding much on how to get the hazard rate function without assuming a distribution.

I am looking at time-series event data and need to visualize how the arrival rate $\lambda$ changes over time. I do not want to assume any underlying distribution that the data comes from (it is certainly not a Poisson process as the rate changes, and neither the Weibull nor the log-logistic distribution seem to describe what's going on). What I want is something between the following two extremes:

  1. Assume that the rate does not change with the time since the previous event (Poisson), and simply calculate the overall rate of events per time. Problem: just a single number, i.e., no ability for the rate to change over time.
  2. Calculate the empirical instantaneous rate between all sets of neighboring points (as in $\frac{1}{t_i - t_{i-1}}$) and plot this over time. Problem: too jumpy, not very informative.

I want something in between these two extremes, in the sense that a kernel density estimate is "in between" calculating the mean of observations, vs. looking at a rugplot showing the individual values of the data. Ultimately, I want to see a smoothed curve that shows a locally-weighted average of the rate of arrivals at each point.

Two classes of approach come to mind: some kind of kernel density estimate of the values from extreme #2, or some use of an (inverse) exponential function of the previous interarrival times to provide an exponentially-weighted average of the rate.

Is there a right way to do this? If so, is there also a right way to select an appropriate bandwidth/weighting value (which I'm assuming will be necessary, whatever the solution is)?

PS This is not a failure analysis, but the concept of hazard rates seems somewhat relevant. However, I'm not finding much on how to get the hazard rate function without assuming a distribution.

I am looking at time-series event data and need to visualize how the arrival rate $\lambda$ changes over time. I do not want to assume any underlying distribution that the data comes from (it is certainly not a Poisson process as the rate changes, and neither the Weibull nor the log-logistic distribution seem to describe what's going on). What I want is something between the following two extremes:

  1. Assume that the rate does not change with the time since the previous event (Poisson), and simply calculate the overall rate of events per time. Problem: just a single number, i.e., no ability for the rate to change over time.
  2. Calculate the empirical instantaneous rate between all sets of neighboring points (as in $\frac{1}{t_i - t_{i-1}}$) and plot this over time. Problem: too jumpy, not very informative.

I want something in between these two extremes, in the sense that a kernel density estimate is "in between" calculating the mean of observations, vs. looking at a rugplot showing the individual values of the data. Ultimately, I want to see a smoothed curve that shows a locally-weighted average of the rate of arrivals over time.

Two classes of approach come to mind: some kind of kernel density estimate of the values from extreme #2, or some use of an (inverse) exponential function of the previous interarrival times to provide an exponentially-weighted average of the rate.

Is there a right way to do this? If so, is there also a right way to select an appropriate bandwidth/weighting value (which I'm assuming will be necessary, whatever the solution is)?

PS This is not a failure analysis, but the concept of hazard rates seems somewhat relevant. However, I'm not finding much on how to get the hazard rate function without assuming a distribution.

edited tags
Link
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663
edited tags
Link
eithompson
  • 376
  • 1
  • 10
Source Link
eithompson
  • 376
  • 1
  • 10
Loading