Sorry for the somewhat ambiguous title but I was not sure how to describe the problem in one line. The issue I am having is the following:
In a supervised learning setting, I have instances that have features associated to them. However, for some instances I have several observations.
As a concrete example, I might want to predict future employee performance in a company according to previous performance (e.g, a bunch of measurements like productivity etc). So I would have an employee with only one year of data (say 2003) and another employee with 3 (2001, 2002, 2003). The features measured for each year are the same and let's assume that all the employees worked at the same company, such that comparison is easier.
Now, the questions becomes: how to end up with one observation row per employee. I have a few ideas:
Simply use the last year of data available for each employee and discard those before, so that I have exactly one line per employee. I would also use a numerical variable indicating the number of years the employee has been working at the company for as an additional feature. The idea is that the most recent year would be the most informative anyways. However, it seems to me that I might be throwing away potentially useful information.
Taking the mean (or kernel mean embedding, any kind of summary really ...) across all years. However, this looks wrong to me as people who worked for a different amount of time at a company get unfairly compared. Admittedly, they would be less 'productive' in their first year and gradually improve. Meaning that it would be better to be more productive in your first year than someone in their third or fifth year for instance.
I would use some sort of measure that computes the rate of improvement for each feature from year 1 to most recent year and also add the number of years worked as an extra feature (as in point 1)). However, I would have to come up with some fake values for those of worked only on year. I was thinking of a very unrealistic value. I think this might work in a tree like algorithm that does not multiply a feature with a parameter, but would give seriously wrong results when using neural nets or linear regression to name a few. What are your thoughts on the effect this would have on various learning algorithms?