Extract valuable information from a list of dates for Machine Learning

Question

I have to create a model to predict if a patient has a disease or not based on pharmaceutical prescription data. I created a neural network and a random forest using as features the number of prescriptions for each drug available in the dataset.

paz id	Drug 1	Drug 2	...	Disease
1	3	5	...	1
2	4	1	...	0

But, in the dataset date of prescription is available too and I wanted to include this information in the model, to eventually detect important pattern over time. The problem is that patients that used the same drug multiple times will have multiple prescription dates.

paz id	Drug 1	Drug 2	...	Disease
1	[2018-03-14, 2020-01-07, 2021-09-21]	[2012-01-15, 2014-06-13, 2016-05-01, 2018-11-21, 2020-05-27]	...	1
2	[2016-08-24, 2017-04-02, 2020-07-26, 2021-04-21]	[2015-01-12]	...	0

Is there any way to encode list of dates in a single useful feature that can be used by classification models using python? Or is there any way to reorganize the dataset to eventually detect important pattern over time taking also into consideration the number of prescriptions for the same drug without adding a feature for each date of each drug? Thanks in advance!

Answer 1

Feature engineering is an art, there is no single way how this can be done or the single way that would be universally best. You have many possibilities, for example:

• Binary variable for the drug being ever prescribed or alternatively if it happened in some time window (last 5 years, last year, last month, etc).
• Count how many times did it happen in some period of time, as above.
• Count of days since the last prescription of the drug.
• You can treat it as a binary time-series where each prescription is a positive signal.

There are probably many other possibilities. Some of the solutions may make more or less sense for the problem, this is something worth discussing with domain experts. In the end, this is something to be verified empirically: some of the features might prove to work better for your problem than others and it might not be able to tell this a priori.