1
$\begingroup$

I have a time series dataset as follows (just 1 part out of 1000 obs). The data includes only the time and the outcome (1 - success, 0 - failure). Time here is not the amount of time but the date (For example, test 1 is performed at 15:18:56 on 10/9/2012, not that it took 15 hours to complete).

Tests are independent but it is assumed that lessons learned after a failure

enter image description here

Questions: (1) Do you know any package in R/ Python that could help predicting this data? (2) How the assumption of lesson learned impact the prediction model?

$\endgroup$
  • $\begingroup$ It would help to have some more context here. What are the tests? What determines the times? What does "it is assumed that lessons learned after a failure" mean? What is to be predicted (success or failure given the time? time of success?) $\endgroup$ – The Laconic Feb 9 '18 at 2:06
  • $\begingroup$ "Lessons learned after a failure" for example the result of test 2 is fail, testers would learn failure lesson from test 2 in order to perform test 3 better $\endgroup$ – Sophie Feb 9 '18 at 2:17
  • $\begingroup$ Need to predict: success or failure given the time. $\endgroup$ – Sophie Feb 9 '18 at 2:18
  • $\begingroup$ It is the test of aviation. The time here is not the amount of time but the date. For example, at 15:18:56, test 1 is performed (it is not that test 1 took 15 hours to succeed) $\endgroup$ – Sophie Feb 9 '18 at 2:23
0
$\begingroup$

If you want to predict the outcome based on time, where outcome is your response try using survival analysis. Your data looks like a good candidate for this analysis.

Survival regression model and Kaplan-maier estimate (probability function) are available in R. The pakage is survival. Here is a link to a good tutorial.

https://www.r-bloggers.com/survival-analysis-with-r/amp/

$\endgroup$
0
$\begingroup$

You might consider the model \begin{align*} y_t \mid x_t &\sim \text{Bernoulli}\left(p = \frac{e^{x_t}}{1+e^{x_t}}\right) \\ x_t - \mu &= \phi( x_{t-1} - \mu) + w_t \end{align*} where $y_t$ is your $0,1$ observation and $x_t$ is some hidden state. I think the KFAS package can handle this, but I've never used it. That vignette also mentions other software that you can use for this model.

It's kind of a fancy model, but if you can figure it out, you'll be able to see how the probability of a $1$ changes over time in different ways.

$\endgroup$
0
$\begingroup$

I've used a multi-order markov transition table with reasonable success in the past. This should capture the possible learning after a failure to some extent. It should be pretty easy to port this Matlab code to R or Python.

Multi-Order Markov Transition Table

Perhaps this R package does a good job of fitting high order transition tables:

MarkovChain package

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.