3
$\begingroup$

Let's assume I have a frequency distribution for train travel times, estimated based on one month of empirical data:

  • [1:00h, 1:20[ : 20%
  • [1:20h, 1:40h[ : 50%
  • [1:40h, 2:00] : 30%

Edit: The data is put into bins, because it is very unlikely that we have two exact data points and a difference from e.g. one second isn't relevant. For someone looking at the distribution it is such easier to see relevant details. The 20mins bins are just an example.

And I know at time t that the train already travels for 1:25h. Does this information help me to update my knowledge about how long the train will travel? Maybe conditional probabilities, but I don't know how to use them in this case.

$\endgroup$
  • $\begingroup$ (+1) I would guess you aren't really thinking about this as a discrete distribution, but rather as a discrete summary of a continuous distribution. That's a little different, because it opens up the possibility of interpolating within each of the three bins. Is this the sort of approach you are looking for? $\endgroup$ – whuber Apr 29 '15 at 19:02
  • $\begingroup$ @whuber Let's assume the only information you have about the distribution within a bin is a uniform distribution of the values. So each value in the range of a bin is equally likely. However, I'm also interested in your answer if it would be a continuous distribution. $\endgroup$ – user2653422 Apr 29 '15 at 19:22
  • $\begingroup$ If it's uniform within each bin, then you have a well-defined continuous distribution: there's nothing discrete about the situation. The real issue here, as I see it, concerns estimating the true underlying continuous distribution that has been described by these limited summary data. The best approach to that problem depends on what you think the underlying distribution family ought to be. $\endgroup$ – whuber Apr 29 '15 at 19:29
  • $\begingroup$ @whuber What if it is not a theoretic distribution like exponential, but a true empirical distribution which couldn't be approximated by a theoretic distribution in a proper way. It could have been generated by counting the occurrences within one month of train travels, so that you have a frequency distribution or histogram (maybe discrete distribution was the wrong vocabulary). Is it possible to answer my initial question under these assumptions or is it only possible with theoretic distributions? $\endgroup$ – user2653422 Apr 29 '15 at 19:40
  • $\begingroup$ Yes, it is possible. Please edit your post to include that additional information. When you do that, please explain why you want to use the data summarized into 20-minute bins rather than the full empirical distribution. $\endgroup$ – whuber Apr 29 '15 at 20:06
1
+50
$\begingroup$

$T:$Random Variable denoting travel time with limits $60\le t\le120$

we need $E[T|T>85]$

we have,

$T|60\le T\le80\sim U(60,80)~and~P(60\le T\le80)=0.2$

$T|80\le T\le100\sim U(80,100)~and~P(80\le T\le100)=0.5$

$T|100\le T\le120\sim U(100,120)~and~P(100\le T\le120)=0.3$

Now, for the train that travelled $85~min$, that is train survived up to 85.

$E[T|T>85]=\frac{100+85}{2}\times\frac{.5}{.5+.3}+\frac{120+110}{2}\times\frac{.3}{.5+.3}=99.0625$

So, expected time that the train will travel is $\sim14~min$ more.

Probability:- we can find it from available conditional distributions, say, you need $P(100\le T\le120|T>85)$ here we care about distribution for $T>85$ only.... Hence the Pr will be $\frac{.3}{.3+.5}$

For example, $P(90\le T\le110|T>85)=\frac{90-85}{100-85}\times\frac{5}{8}+\frac{110-100}{120-100}\times\frac{3}{8}$

$\endgroup$
  • $\begingroup$ +1 For explaining it with the cond. probabilities I've mentioned in my question. Is it also possible to calculate e.g. P(T = 102 min | T > 85) and if so how? $\endgroup$ – user2653422 May 11 '15 at 17:42
  • $\begingroup$ @user2653422 Actually Random Variable is continuous here, so you can get probability for some range. $\endgroup$ – Hemant Rupani May 11 '15 at 17:53
  • $\begingroup$ Do you have an example please for e.g. the range 100-120? So P(100 <= T <= 120 | T > 85). $\endgroup$ – user2653422 May 11 '15 at 18:04
2
$\begingroup$

This question could be posed in a different way in the form of survival analysis. To make the example clearer, we can phrase "travel time" as "survival time until termination".

With respect to your problem,

And I know at time t that the train already travels for 1:25h. Does this information help me to update my knowledge about how long the train will travel? Maybe conditional probabilities, but I don't know how to use them in this case.

In terms of "does this information matter", it will all boil down to the assumptions you have on your data. If we use nonparametric approaches which do not make any assumptions about distribution of failure times, and just attach Kaplan Meier estimator, then using the information above (assuming 10 cases for simplicity)

+----------+-------------+------------------+---------------------------+-------------+
| Time (m) | total cases | number of events | expected number of events | Hazard rate |
+----------+-------------+------------------+---------------------------+-------------+
| 60-80    | 10          | 2                | 0.2                       | 0.2         |
+----------+-------------+------------------+---------------------------+-------------+
| 80-100   | 8           | 5                | 0.62                      | 0.82        |
+----------+-------------+------------------+---------------------------+-------------+
| 100+     | 3           | 3                | 1.00                      | 1.82        |
+----------+-------------+------------------+---------------------------+-------------+

We can then calculate the survival rates, at that time, we have

$$ S( 1:25h ) = ((10-2)/10)*((8-5)/8) = 0.3 $$

Alternatively, if we use the estimated cumulative hazard rate in the table above, we can express survival rate as:

$$ S(t) = Pr(T> t) = e^{-H(t)} $$

Which if we simplify it, would be $e^{-0.82} = 0.4404317$.


Ultimately your model depends the various assumptions you might have, but outlined here is one possible way to approach the problem from a survival analysis perspective.

$\endgroup$
  • $\begingroup$ Sorry for my late response. Thank you for this answer. Could you please elaborate a bit more on what to do with this survival rate information to get updated information about my frequency distribution? At the moment I don't understand what to do with the 0.3 or 0.4404317. $\endgroup$ – user2653422 May 11 '15 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.