I'm experimenting with catboost, predicting electricity demand from temperature, timeofday, dayofweek and if a day is a public holiday or not (and a few other continuous and categorical columns but they're the key ones).
I'm having issues with the holiday category. The feature importance of this value is very low and it's pretty much ignoring it. Demand on a holiday is usually very low, but there aren't many holidays in a year. Generally with linear regression a holiday category will still have a high importance as it locally reduces the mean squared error considerably on these days (demand is usually pretty low on a holiday). But this isn't happening using catboost. I'm wondering if there's anything I can do to boost the weight put on this feature?
I've tried a True / False category (i.e is or isn't a public holiday) and I've tried specific holiday names (Christmas Day, Easter Monday etc.) but in either case the predicted demand on public holidays has very high error.
To give some more info:
I have the following columns
TimeOfDay (1-96 representing 15 minute intervals) DayOfWeek (Monday - Sunday) MonthOfYear (Jan-Dec) Holiday (Easter, Christmas Day etc) Temperature
I'm training with 1 years data, cross validating (not using the catboost cross validation) with 5 cuts from a 2nd year of data. I don't have much data.
The model picks up on the temperature and time of day / day of week cycle fairly well. But it doesn't differentiate between holidays and non-holidays. Are trees poor at predicting low frequency events - there's only 10 in the training set and when you look at (MonthOfYear,DayOfWeek,Holiday) the data doesn't cover the space at all well (holidays tend to be clustered on Mondays around the first few months of the year.