Machine Learning - Data Pre-Processing I'm currently working on a machine learning project. I have data from over 50 sensors but the time at which data was recorded are not the same, the data is not synchronized and there are missing data.
Most of the sensors record data at an interval of 10 minutes, one record at an interval of 5 minutes and another record at an interval of 1 minute.
What is the best approach to pre-process the data for machine learning?
I'm trying to predict energy consumption which is being recorded at an interval of 1 minute.
 A: Let's assume the underlying physics of your system of interest permits us to consider the last sensor records to be the most relevant record to each record of your target (energy consumption/ 1 min interval sensor). You can create a dataframe in which every row consists of 1 min interval energy consumption record as the target and other sensors records filling other columns (predictors). Now, since they are not synchronized, you could repeat the latest of the 5 min and 10 min sensors readings into every row until there is a new reading. basically it will be filling the gaps until there is none. See schematic picture below. The only way to see if this makes sense would be to understand all physical parameters of your system. It may very much be the case that the last reading of the sensors are not relevant and that all data points need to be thrown out except the points at which there are synchronous original readings from all sensors. Again domain knowledge would dictate which way makes more sense.
 
