10
$\begingroup$

Cross posting my question from mathoverflow to find some stats specific help.

I am studying a physical process generating data which projects nicely into two dimensions with non-negative values. Each process has a (projected) track of $x$-$y$ points -- see the image below.

The sample tracks are blue, a troublesome type of track has been hand drawn in green, and a region of concern drawn in red: tracks and region of concern

Each track is the result of an independent experiment. Twenty million experiments have been conducted across several years, but from those only two thousand exhibit the feature which we plot as a track. We are only concerned with the experiments which generate a track, so our data set is the (approximately) two thousand tracks.

It is possible for a track to enter the region of concern, and we expect on the order of $1$ in $10^4$ tracks to do so. Estimating that number is the question at hand:

How can we calculate the likelihood of an arbitrary track entering the region of concern?

It is not possible to conduct experiments quickly enough to see how often tracks are generated which enter the region of concern, so we need to extrapolate from the available data.

We have fitted for example $x$ values given $y\ge200$, but this does not sufficiently handle data such as the green track -- it seems necessary to have a model encompassing both dimensions.

We have fitted the minimum distance from each track to the region of concern, but we are unconvinced this is producing a justifiable result.

1) Is there a known way to fit a distribution to this type of data for extrapolation?

-or-

2) Is there an obvious way to use this data to create a model for generating tracks? E.g., use principal component analysis on the tracks as points in a large space, then fit a distribution (Pearson?) to the tracks projected onto those components.

$\endgroup$
  • 1
    $\begingroup$ I must not give more information about the particular process, but I will update the question to speak to how data are collected. $\endgroup$ – Jeff Snider Feb 4 '14 at 15:45
  • 1
    $\begingroup$ I have updated the language to try and be more concrete. Imagine we're throwing pebbles at an open top floor window, and we only care how the pebbles which go through the window bounce on the floor inside. We've thrown millions of pebbles, and about 2000 have gone through the window. When a pebble goes through the window we draw a track of its progress across the floor. Given a pebble goes through the window, we want to estimate the likelihood it goes through the region of concern. $\endgroup$ – Jeff Snider Feb 4 '14 at 16:36
1
$\begingroup$

It sounds like you want to simulate the formation of tracks and then conduct a Monte Carlo simulation to see how many tracks fall into the red region. To do this, I would first convert the lines to two functions, one giving direction and the other distance from one point to the next on that track. Now you can study the probability distributions associated with those two functions. For example, you might find that distance travelled follows a specific distribution (be careful that the distribution doesn't change over time). If either variable does change over time then you have to delve into time series analysis (not my field, sorry).

Another thought that comes to mind is that, since the direction of movement in x-y changes gradually in most of the tracks, you might do better examining the change in direction vs time for the tracks.

You will also need to estimate the probability of a track starting at a given x-y co-ordinate with a given direction. You may want to consider using kernel density estimation to smooth the resulting PDF or, if it appears to follow a distribution for which there is an analytical model then expectation maximisation could be used to fit that distribution to the data.

The Monte Carlo simulation would then draw random samples from these distributions to simulate the shapes of the tracks. Then you have to simulate a large number of tracks and see how often those tracks pass through the red region. This could be thousands or millions of tracks, you'll have to experiment to see when the distribution stops changing as you add more tracks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.