1
$\begingroup$

The system I'm trying to make predictions for is Markovian (similar setup to a state-space model). I observe a bunch of events (say 1000) over time. Each event can have up to 10 steps. At each time period, I need to predict one of 3 possible outcomes using a set of exogeneous variables: Success, Failure, Next Step. Here is an example:

Event 1
Timestamp: Jan 1, 2020
Final outcome: Success
Step: 1
Size: 123
# more covariates here

Event 1
Timestamp: Jan 4, 2020
Final outcome: Success
Step: 2
Size: 15 # covariates can change at each step
# more covariates here

Event 1
Timestamp: Jan 5, 2020
Final outcome: Success
Step: 2
Size: 16 # this is a revision to Event 1/Step 2
# more covariates here

Event 2
Timestamp: Nov 24, 2019
Final outcome: Failure
Step: 1
Size: 12
# more covariates here

This is how I'm thinking of building the classifier.

  1. At each time period, look back T months for all events/steps that occurred within the period
  2. For each Event/Step pair, take the latest available set (so if there's a revision, use the revision rather than the superseded record, i.e., drop the second record in the example above)
  3. Create a new "Response" field which is Success/Failure if the step is final, otherwise Next Step
  4. Try and predict Response using the pooled data set

So I have 2 questions to for this approach:

  1. Will my classification be biased by examples where there are a lot of steps?
  2. Should I keep the superseded copies in the data set?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.