I'm getting data by connecting to database to get electricity consumptions. In my case there are lots of meter id's (they will increase to thousands in time) and each meter id has its own time series hourly data. Since some of them show similar patterns and some dont, i decided to train them separately. But the problem is while using XGBoost i must create the best features for each time series and thats a hard thing to do. What should be the approach? Maybe i should train them in a single model for easy feature engineering but this time model will have noise?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Clearly, meter ID is a categorical variable and if only you could bring it into your model in a "good way", you could perhaps just train a single model. One-hot-encoding the IDs is of course totally ridiculous, if there's 1000s, 10,000s or even more IDs. So, you're looking for a more compact representation. There's several options that might do something sensible:
- Instead of the ID use a set of features that describe the ID (if you have enough information that describes how they differ, because if the IDs end up looking the same in terms of features, they end up getting the same predictions...). This can be combined with other strategies.
- Embeddings e.g. generated by pre-training without using the target information (e.g. a model that is trained to return area, property size, type of property [commerical, private...], ...) and/or values of the target variable in the past (but not the future - would have to be define what the split is).
- Forms of target encoding, traditionally that's primarily done for the target variable of interest, but you could perhaps also do it targeting different future outcomes that you are not directly interested in. Like many of the things further down on the list, this need careful attention to detail to not leak target information. E.g. catboost has a version of this in-buildt.
- Embeddings generated by predicting what you're interested in (and perhaps additional things you are not interested in, but could provide additional signal about how the IDs differ) using a neural network, then throwing away the rest of the neural network and just using those embeddings.
There's a lot of scope for creativity here, but also a lot of scope to mess up (e.g. by leaking target information) and/or by messing up your cross-validation.
