I am assuming there is an optimal why to set up my data to achieve my goal of predicting who will retire next year.

I can think of two methods. Which do you think is the most appropriate and why? Or if you can think of another alternative please let me know. Would your answer be different if I wanted to know who is likely to retire in the next two years? three years?

Each method assumes 5 years of training data, for example 2008 - 2012 to predict who will retire in 2013.

Method 1

  • Include a row of data for each employee for each year that they are active (did not retire) and one row for when they retire. In the year that they retire they generally have two rows (one for active and one for retire).
  • Each active year is snapshot employee data as of January 1st. Retiree data is as of date of retirement.
  • Example scenarios are provided below.

Method 2

  • Include one row of data for each employee that was active (did not retire) during the entire time period and one row for each employee that retired. In this example each employee only appears once in the training set.


  • I'm trying to predict who is likely to retire next year.
  • Population: All employees who are 55 years of age or older and have at least 5 years of service.
  • I'm using Random Forest (I have began experimenting w/ logistic regression...not much experience though).
  • I have a binary outcome and my predictors are made of both continuous and factor variables. My binary outcome is retire or not-retire (continued working). Examples of continuous variables are age of employee, years they were with the company and salary. Examples of factor variables are retirement plan type (pension or 401k), job type (exempt vs. non-exempt),Highest Education Level, Department.

Method 1 Data Examples (predicting '13)

  • Person A was here '08 through '12 and retired on '13
  • Person B was here '08 through '11 and retired on '12
  • Person C was here '08 through '13
  • Person D retired in '08 (he was here previous years too but the training data in the example only goes back to '08)
  • 2
    $\begingroup$ It sounds like this might be handled as a survival-analysis problem with age as the time variable, retirement as the "event", and the other variables as predictors. Have you considered that approach? $\endgroup$
    – EdM
    Commented Jun 6, 2015 at 15:42

2 Answers 2


I agree with EdM. Your overarching question reads like survival analysis. CRAN has a whole Task View associated with the various flavors and packages for survival analysis.

Alternative, you might look into a mixed effects logistoc regression model. Logistic, because you have a dichotomous outcome (retired, not retired); mixed, because you likely have both fixed and random effects. There are repeated observations for the same individual. I do not use this technique myself, but I know of colleagues who use it for estimating probability of a student dropping out from school. Seems pretty analagous.

A random forest seems like an odd choice, to me; it'll build out, no doubt, but you're not really exploiting the repeated observations, and the time element.


Why not try them both and see which works best? I think method 1 is probably better (and will give you a much larger dataset to work with), but I think you should just try both methods and see which model performs better.

  • $\begingroup$ I'm a beginner. I sort of began doing predictive modeling earlier this year. I used method 1 but found that the model was overpredicting. But when I divided by two (I figured because I was using two years of history) the number was more in line with expectation. $\endgroup$ Commented Dec 5, 2013 at 22:08
  • $\begingroup$ I bought Max Kuhn's Applied Predictive Modeling Book a couple weeks ago and now am trying things like creating a train and test set and looking at ROC curve to pick different cutoff point (trade off between sensitivity and specificity), etc. But I am finding between trying all these different methods & trying the different number of years of records to use to train with, looking at variables and derived variables that this is really really time consuming. I know this is isn't suppose to be easy but I thought I could get at least get some solid advice regarding how to structure the data. $\endgroup$ Commented Dec 5, 2013 at 23:01
  • $\begingroup$ The ideal answer will either talk through why theoretically one method might be better then the other given the context and/or point to examples (preferably using R) that are analagous to my data . The examples I have experimented with for learning purposes are iris and spam. I would like to hear about examples that have a time component like mine: e.g. using x number of years of data to predict what will happen in the following year. $\endgroup$ Commented Dec 5, 2013 at 23:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.